What You'll Learn
How the ELK Stack (Elasticsearch, Logstash, Kibana) centralizes logs from hundreds of servers into a single, searchable dashboard for production debugging.
The Logging Problem
When you have 5 microservices running across 30 Kubernetes pods or EC2 instances, SSH-ing into servers to run tail -f /var/log/nginx.log is impossible. You need centralized logging.
What is the ELK Stack?
Elasticsearch
A NoSQL search engine based on Lucene. Stores the logs and provides lightning-fast full-text search capabilities.
Logstash
The data processing pipeline. Ingests logs from multiple sources, parses them (grok), and ships them to Elasticsearch.
Kibana
The web UI. Connects to Elasticsearch to let you search logs, create bar charts of HTTP 500 errors, and build dashboards.
* Note: Filebeat (the "F" in EFK) is a lightweight agent installed on your actual servers that reads log files and forwards them to Logstash/Elasticsearch.
Logstash Configuration Pipeline
Logstash pipelines consist of three stages: Input, Filter, and Output.
# 1. INPUT: Receive logs from Beats (Filebeat) on port 5044
input {
beats {
port => 5044
}
}
# 2. FILTER: Parse unstructured text into JSON fields
filter {
if [type] == "nginx_access" {
grok {
# This matches standard Nginx access logs and extracts IP, status, etc.
match => { "message" => "%{IPORHOST:clientip} %{USER:ident} %{USER:auth} \[%{HTTPDATE:timestamp}\] \"%{WORD:verb} %{DATA:request} HTTP/%{NUMBER:httpversion}\" %{NUMBER:response:int} (?:-|%{NUMBER:bytes:int}) %{QS:referrer} %{QS:agent}" }
}
date {
match => [ "timestamp" , "dd/MMM/YYYY:HH:mm:ss Z" ]
}
# Geolocate the IP address!
geoip {
source => "clientip"
}
}
}
# 3. OUTPUT: Send structured JSON to Elasticsearch
output {
elasticsearch {
hosts => ["http://elasticsearch:9200"]
index => "nginx-logs-%{+YYYY.MM.dd}" # Create a new index every day
user => "elastic"
password => "changeme"
}
}KQL (Kibana Query Language)
Once logs are in Kibana, you use KQL to search them rapidly.
response: 500# Find all HTTP 500 server errorsresponse >= 400 AND response < 500# All client errors (4xx)kubernetes.namespace: "production" AND message: "Exception"# Prod exceptionsclientip: "192.168.1.50"# Track what a specific user didrequest: *login*# Wildcard search on URL path
DevOps Pro Tip: JSON Logging
Writing complex Grok filters in Logstash is painful and CPU intensive. The modern best practice is to configure your applications (Node.js, Python, Nginx) to output logs natively in JSON format. When Filebeat sees JSON, it parses it automatically without any Logstash filters needed!