1. Tổng quan.
Bài nói về Telegraf và InfuxDB mình đã nói quá nhiều rồi, các bạn tự tìm bài của mình đọc lại nhé. Phần này mình chỉ nói về cách config Telegraf để output metrics về InfluxDB mà thôi.
2. Cài đặt Telegraf.
Chạy các lệnh sau bằng apt-get để cài đặt Telegraf từ kho lưu trữ InfluxData:
curl --silent --location -O \
https://repos.influxdata.com/influxdata-archive.key \
&& echo "943666881a1b8d9b849b74caebf02d3465d6beb716510d86a39f6c8e8dac7515 influxdata-archive.key" \
| sha256sum -c - && cat influxdata-archive.key \
| gpg --dearmor \
| tee /etc/apt/trusted.gpg.d/influxdata-archive.gpg > /dev/null \
&& echo 'deb [signed-by=/etc/apt/trusted.gpg.d/influxdata-archive.gpg] https://repos.influxdata.com/debian stable main' \
| tee /etc/apt/sources.list.d/influxdata.list
apt-get update && apt-get install telegrafĐây là cây thư mục lưu config file của Telegraf sau khi cài xong.
shell> tree /etc/telegraf
/etc/telegraf
├── telegraf.conf
├── telegraf.conf.sample
└── telegraf.d
1 directory, 2 files3. Cấu hình Telegraf đẩy về InfluxDB.
Bạn có thể chỉnh sửa lại file /etc/telegraf/telegraf.conf tham khảo theo mẫu của mình như dưới. Hãy tham khảo bài biết https://wiki.hoanghd.com/tim-hieu-ve-file-cau-hinh-telegraf-de-thu-thap-so-lieu-he-thong/ để xem hiểu rõ hơn về cách config file /etc/telegraf/telegraf.conf.
Bạn có thể tham khảo config Telegraf để đẩy metric về InfluxDB như sau:
[agent]
interval = "30s"
round_interval = true
metric_buffer_limit = 10000
flush_buffer_when_full = true
collection_jitter = "0s"
flush_interval = "10s"
flush_jitter = "0s"
debug = false
quiet = false
logfile = "/var/log/telegraf/telegraf.log"
[[outputs.influxdb]]
urls = ["http://10.237.7.71:8086"]
database = "elasticsearch-metrics"
precision = "s"
timeout = "10s"
# username = "monitor_user"
# password = "password"
insecure_skip_verify = true
[[inputs.elasticsearch]]
## specify a list of one or more Elasticsearch servers
## you can add username and password to your url to use basic authentication:
## servers = ["http://user:pass@localhost:9200"]
servers = ["http://127.0.0.1:9200"]
http_timeout = "5s"
username = "elastic"
password = "Password@123"
## When local is true (the default), the node will read only its own stats.
## Set local to false when you want to read the node stats from all nodes
## of the cluster.
local = false
## Set cluster_health to true when you want to obtain cluster health stats
cluster_health = true
## Adjust cluster_health_level when you want to obtain detailed health stats
## The options are
## - indices (default)
## - cluster
cluster_health_level = "indices"
#cluster_health_level = "cluster"
## Set cluster_stats to true when you want to obtain cluster stats.
cluster_stats = true
## Only gather cluster_stats from the master node. To work this require local = true
cluster_stats_only_from_master = truePhần này bạn để ý thông tin input và output nhập cho chính xác nhé.
Thông tin input từ Elasticsearch.
[[inputs.elasticsearch]]
## specify a list of one or more Elasticsearch servers
## you can add username and password to your url to use basic authentication:
## servers = ["http://user:pass@localhost:9200"]
servers = ["http://127.0.0.1:9200"]
http_timeout = "5s"
username = "elastic"
password = "Password@123"Thông tin input từ InfluxDB.
[[outputs.influxdb]]
urls = ["http://10.237.7.71:8086"]
database = "elasticsearch-metrics"
precision = "s"
timeout = "10s"
# username = "monitor_user"
# password = "password"
insecure_skip_verify = trueCấu hình xong thì khởi động lại Telegraf.
systemctl restart telegrafCheck lại trạng thái Telegraf sau khi khởi động.
shell> systemctl status telegraf
● telegraf.service - Telegraf
Loaded: loaded (/lib/systemd/system/telegraf.service; enabled; vendor preset: enabled)
Active: active (running) since Mon 2024-09-16 06:29:37 UTC; 3s ago
Docs: https://github.com/influxdata/telegraf
Main PID: 461479 (telegraf)
Tasks: 11 (limit: 19086)
Memory: 27.0M
CGroup: /system.slice/telegraf.service
└─461479 /usr/bin/telegraf -config /etc/telegraf/telegraf.conf -config-directory /etc/telegraf/telegraf.d
Sep 16 06:29:37 es-master2-82 systemd[1]: Starting Telegraf...
Sep 16 06:29:37 es-master2-82 telegraf[461479]: time="2024-09-16T06:29:37Z" level=warning msg="DBUS_SESSION_BUS_ADDRESS envvar looks to be not set, this can lead to runaway dbus-daemon processes. To avoid this, set envvar DBUS_SESSION_BUS_ADDRESS=$XDG_RUNTIME_DIR/bus (if>
Sep 16 06:29:37 es-master2-82 telegraf[461479]: 2024-09-16T06:29:37Z I! Loading config: /etc/telegraf/telegraf.conf
Sep 16 06:29:37 es-master2-82 telegraf[461479]: 2024-09-16T06:29:37Z W! DeprecationWarning: Option "precision" of plugin "outputs.influxdb" deprecated since version 1.0.0 and will be removed in 1.35.0: option is ignored
Sep 16 06:29:37 es-master2-82 telegraf[461479]: 2024-09-16T06:29:37Z W! DeprecationWarning: Option "http_timeout" of plugin "inputs.elasticsearch" deprecated since version 1.29.0 and will be removed in 1.35.0: use 'timeout' instead
Sep 16 06:29:37 es-master2-82 systemd[1]: Started Telegraf.Bạn cũng có thể check log để chắc chắn quá trình authentication thành công.
shell> tail -f /var/log/telegraf/telegraf.log
2024-09-16T06:35:30Z I! Available plugins: 235 inputs, 9 aggregators, 32 processors, 26 parsers, 62 outputs, 6 secret-stores
2024-09-16T06:35:30Z I! Loaded inputs: elasticsearch
2024-09-16T06:35:30Z I! Loaded aggregators:
2024-09-16T06:35:30Z I! Loaded processors:
2024-09-16T06:35:30Z I! Loaded secretstores:
2024-09-16T06:35:30Z I! Loaded outputs: influxdb
2024-09-16T06:35:30Z I! Tags enabled: host=es-master2-82
2024-09-16T06:35:30Z W! Deprecated inputs: 0 and 1 options
2024-09-16T06:35:30Z W! Deprecated outputs: 0 and 1 options
2024-09-16T06:35:30Z I! [agent] Config: Interval:30s, Quiet:false, Hostname:"es-master2-82", Flush Interval:10sHãy vào server InfluxDB check DB elasticsearch-metrics và truy vấn thử metrics nhé.
shell> influx -precision rfc3339
Connected to http://localhost:8086 version 1.8.10
InfluxDB shell version: 1.8.10
> SHOW DATABASES
name: databases
name
----
_internal
mydb
database-name
elasticsearch-metrics
> use elasticsearch-metrics
Using database elasticsearch-metrics
> SHOW MEASUREMENTS
name: measurements
name
----
elasticsearch_breakers
elasticsearch_cluster_health
elasticsearch_cluster_health_indices
elasticsearch_clusterstats_indices
elasticsearch_clusterstats_nodes
elasticsearch_fs
elasticsearch_http
elasticsearch_indices
elasticsearch_jvm
elasticsearch_os
elasticsearch_process
elasticsearch_thread_pool
elasticsearch_transportVà đây là kết quả.
> SELECT * FROM elasticsearch_cluster_health
name: elasticsearch_cluster_health
time active_primary_shards active_shards active_shards_percent_as_number delayed_unassigned_shards host initializing_shards name number_of_data_nodes number_of_in_flight_fetch number_of_nodes number_of_pending_tasks relocating_shards status status_code task_max_waiting_in_queue_millis timed_out unassigned_shards
---- --------------------- ------------- ------------------------------- ------------------------- ---- ------------------- ---- -------------------- ------------------------- --------------- ----------------------- ----------------- ------ ----------- -------------------------------- --------- -----------------
2024-09-16T06:36:00Z 9 18 100 0 es-master1-81 0 new-meta 4 0 7 0 0 green 1 0 false 0
2024-09-16T06:36:00Z 9 18 100 0 es-master3-83 0 new-meta 4 0 7 0 0 green 1 0 false 0
2024-09-16T06:36:00Z 9 18 100 0 es-master2-82 0 new-meta 4 0 7 0 0 green 1 0 false 0
2024-09-16T06:36:30Z 9 18 100 0 es-master3-83 0 new-meta 4 0 7 0 0 green 1 0 false 0