ELK (Elasticsearch-Logstash-Kibana)

More info: https://www.elastic.co/what-is/elk-stack

‍You can install the ELK tool with the attached docker-compose-elk.yml file.

To install it you should create a new stack and copy the content of the configured elk compose file then Deploy the new stack.

‍Before compose-up execute the following command on the host machine:

‍sudo echo "vm.max_map_count=262144" >> /etc/sysctl.d/elasticsearchSpecifications.conf && sudo sysctl --system

‍Configuration:

‍You can change the default storage path of the ELK application. For that, you have to configure the left side from the colon that lines which are starting with /srv/docker at volume sections.

Curator

Elasticsearch Curator helps you curate, or manage, your Elasticsearch indices and snapshots.

More info: https://www.elastic.co/guide/en/elasticsearch/client/curator/current/about.html

‍Configuration:

• CURATOR_SLEEP_SECS: This amount of secs will the Curator sleeps between 2 curations - configured for 24 hours, 86400 sec
• CURATOR_INDEX_PATTERN: Index pattern that curator will search - configured for filebeat-*
• You should set the filebeat index maximum age to 1 (Kibana->Management->Elascticsearch->Index Lifecycle Policies->filebeat)
• CURATOR_RETENTION_DAYS: Delete older indexes (in days) - configured to 30 days

Filebeat

Filebeat is a lightweight shipper for forwarding and centralizing log data.

More info: https://www.elastic.co/guide/en/beats/filebeat/7.10/filebeat-overview.html

‍Configuration:

‍You can change the default storage path of the Filebeat application. For that, you have to configure the left side from the colon that lines which are starting with /srv/docker at volume sections.