💥 From Zero to ELK Stack on EC2 — Fully Automated
🔍 The Problem
Centralized logging is a must — especially when you’ve got multiple instances running, logs scattered everywhere, and no idea where that 500 error came from.
But standing up ELK (Elasticsearch, Logstash, Kibana) can be... a pain.
I wanted a single EC2 instance running the full ELK stack — logs routed and searchable — all with one command. No manual installs. No clicky walkthroughs.
🛠️ The Fix: A Bash-Based ELK Installer
I wrote a bash script that:
- ✅ Sets up an EC2 instance with IAM + Security Group preconfigured
- 📦 Installs Elasticsearch, Logstash, and Kibana with version compatibility handled
- 🔐 Opens the right ports (9200, 5601, etc.) so it works out of the box
- 🧪 Runs a test log to validate the pipeline end-to-end
Result: A working, log-ready ELK stack you can SSH into and start using immediately.
📬 GitHub Link
Full installer script and details:
👉
github.com/chinmaya-chhatre/elk-integration
📈 What Changed
- 📊 Gained full visibility into application logs in < 10 minutes
- 🛠️ Made log search and dashboarding possible without any third-party tools
- 🧹 Reduced log noise with filters and pipeline tuning
⚖️ Tradeoffs I Made
- Single-Instance Stack: No HA or cluster setup — perfect for demos and small apps, not production scale
- No Persistent Storage: Defaults to ephemeral disk — logs reset on restart unless customized
- Basic Security: Left auth open for testing — production use would need auth+HTTPS setup
🧠 What I'd Add Next
- 🔐 Add basic auth and TLS config for Kibana access
- 📤 Push logs from multiple instances via Filebeat or FluentBit
- 🔁 Package it as a reusable shell module or Terraform wrapper
🧵 Why I’m Sharing This
Because setting up ELK shouldn’t require a YouTube tutorial + 2 hours.
Because log visibility is too important to delay.
And because this script helped me debug real issues — fast.
📎 Bonus Links