fluentbit+influxdb+chronograf收集日

安装 fluentbit

## 添加 GPG
wget -qO - https://packages.fluentbit.io/fluentbit.key | sudo apt-key add -
# 更新 sources lists
echo "deb https://packages.fluentbit.io/ubuntu/xenial xenial main" >> /etc/apt/sources.list

sudo apt-get update
sudo apt-get install td-agent-bit

这里安装的 td-agent-bit 就是 fluentbit

安装 influxdb + chronograf

为了方便，我使用 docker 部署， docker-compose.yml 如下：

influxdb:
  image: "influxdb:1.6-alpine"
  ports: ['127.0.0.1:8086:8086']
  volumes:
    - "/var/docker/influxdb:/var/lib/influxdb"
  environment:
    - TZ=Asia/Shanghai
    - GOGC=10
    - INFLUXDB_DATA_INDEX_VERSION=tsi1
  restart: "always"

chronograf:
  image: "chronograf:1.6-alpine"
  net: host
  restart: "always"

docker-compose up -d

配置 fluentbit

默认的配置在 /etc/td-agent-bit 目录下面

# td-agent-bit.conf
[SERVICE]
    Flush        1
    Daemon       Off
    Log_Level    info
    Log_File     /var/log/fluent-bit.log
    Parsers_File parsers.conf

[INPUT]
    Name cpu
    Tag  stats.cpu

[INPUT]
    Name disk
    Tag  stats.disk

[INPUT]
    Name mem
    Tag  stats.mem

[INPUT]
    Name          netif
    Tag           
    Interface     eth0

[INPUT]
    Name tail
    Tag tengine
    Path /var/log/tengine/source/*.log
    DB /var/log/tengine/source/cursor.db
    Parser nginx

[OUTPUT]
    Name  influxdb
    Match tengine
    Host 127.0.0.1
    Port 8086
    Database log
    Tag_Keys host http_version method remote

[OUTPUT]
    Name  influxdb
    Match stats.*
    Host 127.0.0.1
    Port 8086
    Database stats

# parsers.conf
[PARSER]
    Name   nginx
    Format regex
    Regex ^(?<remote>[^ ]*) \[(?<time>[^\]]*)\] "(?<host>\S*)" "(?<method>\S+) (?<path>[^\"]*?) (?<http_version>\S+)" (?<code>\S*) (?<size>[-0-9\.]+) "(?<referer>[^\"]*)" "(?<agent>[^\"]*)" (?<upstream_time>[-0-9\.]+) (?<request_time>[-0-9\.]+)$
    Time_Key time
    Time_Format %Y-%m-%dT%H:%M:%S%z
    Types code:integer size:integer upstream_time:float request_time:float

启动 fluent-bit

sudo systemctl start td-agent-bit

从 chronograf 查看数据

打开 Data Explorer 查看数据

在 Dashboards 里面可以添加 Cell ，显示各种指标。

Dashboards 的功能也可以使用 grafana 实现。