Setting Up Grafana Loki

This is a continuation of a story published about centralized logging . Let’s talk about more about configuration of loki.

Grafana Loki is an open-source log aggregation system and log processing solution designed to work seamlessly with Grafana, a popular open-source monitoring and observability platform. Loki is particularly well-suited for collecting, storing, and querying log data generated by distributed and microservices-based applications. It is part of the larger ecosystem of tools for observability, alongside Prometheus, Grafana, and others.

Here are some key features and components of Grafana Loki:

1. Log Aggregation: Loki collects log data from various sources, such as application logs, system logs, and infrastructure logs. It is designed to handle high volumes of log data efficiently.

2. Prometheus Integration: Loki integrates seamlessly with Prometheus, a widely-used monitoring system. This integration allows users to correlate metrics and logs easily, providing a holistic view of their applications and infrastructure.

3. Minimal Storage Requirements: One of Loki’s standout features is its efficient use of storage. It uses a technique called “log labels” to store log data in a compressed and structured format. This approach significantly reduces storage costs compared to traditional log storage solutions.

4. Query Language: Loki provides a query language called LogQL, which is similar to PromQL (Prometheus Query Language). It allows users to perform powerful queries on their log data, filter logs based on various criteria, and aggregate log lines.

5. Distributed Architecture: Loki is designed to be horizontally scalable and can run in a distributed architecture. This scalability allows it to handle large volumes of log data in a clustered setup.

6. Grafana Integration: Loki works seamlessly with Grafana, a popular open-source dashboard and visualization tool. Grafana can be used to create custom dashboards and visualizations for log data stored in Loki.

Loki Components:

1. Distributor: The Distributor component helps distribute incoming log data across multiple Loki instances, ensuring even load distribution.

2. Ingester: The Ingester is responsible for writing incoming log data to storage backends.

3. Querier: The Querier handles query requests from users and retrieves log data from storage for querying.

Here you can find the docker compose file to spin up a loki container.

version: '3.8'

services:

  loki:
    image: grafana/loki:2.8.4
    ports:
      - 3100:3100
    volumes:
      - loki-data:/loki
      - ./config/loki-config.yml:/etc/loki/local-config.yaml
    command: -config.file=/etc/loki/local-config.yaml
    restart: unless-stopped
    networks:
      - loki

volumes:
  loki-data:

networks:
  app:
    name: loki

Loki configuration used to configure various settings and behaviors of the Loki log aggregation system. Here you can find sample configuration.

auth_enabled: false

server:
  http_listen_port: 3100
  
common:
  path_prefix: /loki
  storage:
    filesystem:
      chunks_directory: /loki/chunks
      rules_directory: /loki/rules
  replication_factor: 1
  ring:
    kvstore:
      store: inmemory

schema_config:
  configs:
    - from: 2020-10-24
      store: boltdb-shipper
      object_store: filesystem
      schema: v11
      index:
        prefix: index_
        period: 24h

storage_config:
  boltdb:
    directory: /loki/index
  
compactor:
  working_directory: /loki/compactor
  shared_store: filesystem
  retention_enabled: true
  
chunk_store_config:
  max_look_back_period: 48h
  
limits_config:
  retention_period: 48h
  
table_manager:
  retention_deletes_enabled: true
  retention_period: 48h

Let’s break down the key sections and their meanings:

auth_enabled: false: This setting specifies whether authentication is enabled for Loki. You would typically enable authentication to secure access to your log data.

http_listen_port: 3100: Specifies the HTTP port on which Loki will listen for incoming requests.

path_prefix: /loki: Defines a path prefix

storage: Configures how log data is stored

filesystem: Loki is configured to use a filesystem-based storage backend for log data.

chunks_directory: /loki/chunks: Specifies the directory where log chunks are stored.

rules_directory: /loki/rules: Specifies the directory to store rules.

replication_factor: 1: Sets the replication factor for log data. In this configuration, data is not replicated (replication factor is 1), meaning there is only one copy of the data.

ring: Configures the ring configuration for distributing data across multiple instances of Loki. In this case, an in-memory key-value store (inmemory) is used for the ring configuration.

schema_config Section:

configs: Specifies the schema configuration for log data:

from: 2020-10-24: Defines the starting date from which the schema configuration applies.

store: boltdb-shipper: Specifies the store type for log data. In this case, it’s using BoltDB Shipper.

object_store: filesystem: Specifies the object store for log data, which is the filesystem in this case.

schema: v11: Specifies the schema version to use.

index: Configures the indexing of log data:

boltdb: Configures the BoltDB storage backend:

directory: /loki/index: Specifies the directory where the index data is stored.

compactor Section: Compacts index shards for performance

working_directory: /loki/compactor: Sets the working directory for the compactor component.

shared_store: filesystem: Specifies the shared store type for the compactor (filesystem in this case).

retention_enabled: true: Indicates that retention is enabled, meaning that old log data will be retained and potentially compacted.

max_look_back_period: 48h: Specifies the maximum look-back period for chunk data, which is 48 hours in this configuration.

retention_period: 48h: Sets the retention period for log data to 48 hours. After this period log chunks will be deleted.

table_manager Section: Table manager for retention

retention_deletes_enabled: true: Indicates that retention deletes are enabled, allowing old log data to be deleted based on the retention period.

retention_period: 48h: Sets the retention period for log data to 48 hours.

This configuration file defines how Loki should store, index, and manage log data. The specific values may vary depending on your use case and requirements, but this example provides a basic template for configuring Loki.

To better grasp the entire concept of centralized logging, I encourage you to explore the narrative regarding centralized logging in conjunction with this story.

That’s all for this story. Thank you!