ELK（Elasticsearch + Logstash + Kibana）详细部署方法

ELK是Elasticsearch、Logstash、Kibana三个开源工具的简称，是目前最流行的日志分析和可视化解决方案。本文详细介绍从零部署ELK 8.x版本的完整步骤。

---

一、环境准备

1.1 系统要求

• 操作系统：CentOS 7+ / Ubuntu 20.04+ / Debian 11+
• 内存：建议 ≥ 8GB（Elasticsearch 最低 4GB）
• 磁盘：建议 ≥ 50GB（取决于日志量）
• JDK：Elasticsearch 8.x 自带 JDK，无需单独安装

1.2 安装前优化

# 修改系统参数
cat >> /etc/security/limits.conf << EOF
* soft nofile 65536
* hard nofile 65536
* soft nproc 4096
* hard nproc 4096
EOF

# 关闭swap（生产环境建议）
swapoff -a
sed -i '/swap/s/^/#/' /etc/fstab

# 修改vm.max_map_count
sysctl -w vm.max_map_count=262144
echo "vm.max_map_count=262144" >> /etc/sysctl.conf

二、部署 Elasticsearch

2.1 安装 Elasticsearch

# 导入 GPG key
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg

# 添加 yum 源（CentOS/RHEL）
cat > /etc/yum.repos.d/elasticsearch.repo << EOF
[elasticsearch]
name=Elasticsearch repository
baseurl=https://artifacts.elastic.co/packages/8.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=0
autorefresh=1
type=rpm-md
EOF

# 安装
yum install --enablerepo=elasticsearch elasticsearch -y

2.2 配置 Elasticsearch

vim /etc/elasticsearch/elasticsearch.yml

# 关键配置项：
cluster.name: my-elk-cluster
node.name: node-1
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
network.host: 0.0.0.0
http.port: 9200
discovery.type: single-node
xpack.security.enabled: true

2.3 启动并设置密码

systemctl daemon-reload
systemctl enable elasticsearch
systemctl start elasticsearch

# 查看状态
curl -X GET "localhost:9200/"

# 重置密码
/usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic -i

三、部署 Kibana

3.1 安装

yum install --enablerepo=elasticsearch kibana -y

3.2 配置

vim /etc/kibana/kibana.yml

server.port: 5601
server.host: "0.0.0.0"
elasticsearch.hosts: ["http://localhost:9200"]
elasticsearch.username: "kibana_system"
elasticsearch.password: "your_password"
i18n.locale: "zh-CN"

3.3 启动

systemctl enable kibana
systemctl start kibana

# 防火墙放行
firewall-cmd --add-port=5601/tcp --permanent
firewall-cmd --reload

浏览器访问 http://服务器IP:5601，用 elastic 用户登录。

四、部署 Logstash

4.1 安装

yum install --enablerepo=elasticsearch logstash -y

4.2 配置 Nginx 日志采集

vim /etc/logstash/conf.d/nginx.conf

input {
  file {
    path => "/var/log/nginx/access.log"
    start_position => "beginning"
    sincedb_path => "/dev/null"
  }
}

filter {
  grok {
    match => { "message" => "%{COMBINEDAPACHELOG}" }
  }
  date {
    match => ["timestamp", "dd/MMM/yyyy:HH:mm:ss Z"]
    target => "@timestamp"
  }
}

output {
  elasticsearch {
    hosts => ["localhost:9200"]
    index => "nginx-logs-%{+YYYY.MM.dd}"
    user => "elastic"
    password => "Elastic123!"
  }
}

4.3 启动

systemctl enable logstash
systemctl start logstash
tail -f /var/log/logstash/logstash-plain.log

五、Filebeat 轻量采集方案

yum install --enablerepo=elasticsearch filebeat -y

vim /etc/filebeat/filebeat.yml

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /var/log/*.log

output.elasticsearch:
  hosts: ["localhost:9200"]
  username: "elastic"
  password: "Elastic123!"

systemctl enable filebeat
systemctl start filebeat

六、常用维护命令

# 查看集群健康
curl -u elastic:密码 localhost:9200/_cluster/health?pretty

# 查看索引
curl -u elastic:密码 localhost:9200/_cat/indices?v

# 删除7天前索引
curl -u elastic:密码 -X DELETE "localhost:9200/nginx-logs-$(date -d '7 days ago' +%Y.%m.%d)"

# 重启全部服务
systemctl restart elasticsearch kibana logstash

七、常见问题

• ES启动失败： 内存不足，检查 ES_HEAP_SIZE 设为物理内存50%
• Kibana连不上ES： 检查密码和xpack.security配置
• Logstash收不到日志： 检查文件路径和权限
• 磁盘空间不足： 配置ILM自动清理旧索引

八、生产环境建议

1. 至少3个ES节点组成集群
2. 配置索引生命周期管理（ILM）
3. 开启xpack安全认证和TLS加密
4. 冷热架构：SSD存热数据，HDD存冷数据
5. 配合 Watcher 或 Prometheus 做监控告警

---

本文介绍了ELK 8.x完整部署流程，适合运维工程师参考。如有问题欢迎通过本站联系交流。