How to Install and Configure Logstash on Rocky Linux

If you’re using the Elastic Stack for your data processing and analytics needs, one crucial component you must have is Logstash. Logstash is an open-source data processing pipeline that ingests, transforms, and ships data to various destinations such as Elasticsearch, Kafka, and more. In this guide, we’ll walk you through the process of how to install and configure Logstash on Rocky Linux.

Table of Contents

Prerequisites
Installing Java
Installing Logstash
Configuring Logstash
Running Logstash
Configuring Logstash as a Service
Testing Logstash
Conclusion

How to Install and Configure Logstash on Rocky Linux

Prerequisites

Before we begin, ensure that you have the following:

A Rocky Linux system with root access or sudo privileges
An updated system: run sudo dnf update -y
A working installation of Elasticsearch: check our guide on how to install and configure Elasticsearch on Rocky Linux

Installing Java on Rocky Linux

Logstash requires Java to run. You can install the OpenJDK 11 by executing the following command:

sudo dnf install java-11-openjdk -y

Verify the Java installation by running:

java -version

You should see the version details of the installed OpenJDK.

Installing Logstash on Rocky Linux

To install Logstash, first, import the Elastic GPG key:

sudo rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

Create a Logstash repository file:

sudo tee /etc/yum.repos.d/logstash.repo << EOL
[logstash-7.x]
name=Elastic repository for 7.x packages
baseurl=https://artifacts.elastic.co/packages/7.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md
EOL

Install Logstash using the dnf package manager:

sudo dnf install logstash -y

After the installation is complete, you can find the Logstash configuration files in the /etc/logstash directory.

Running Logstash on Rocky Linux

To run Logstash with the configuration file you just created, execute the following command:

sudo /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/basic-logstash.conf

Now, you can type any text into the terminal, and Logstash will process it and print the result to stdout.

To stop Logstash, press Ctrl + C.

Configuring Logstash on Rocky Linux

In order to make Logstash more useful, we need to configure it to process and analyze logs from various sources. Logstash uses a configuration file to define input, filter, and output plugins. The configuration file is written in the Logstash configuration language.

Create a new configuration file for Logstash:

bash

sudo nano /etc/logstash/conf.d/logstash.conf

Input Plugins

Input plugins are used to read data from various sources. In this example, we’ll use the file input plugin to read log files from a specific directory.

Add the following input configuration to your logstash.conf file:

plaintext

input {
  file {
    path => "/var/log/*.log"
    start_position => "beginning"
    sincedb_path => "/dev/null"
  }
}

This configuration will read all log files in the /var/log directory.

Filter Plugins

Filter plugins are used to process and manipulate the data. In this example, we’ll use the grok filter plugin to parse the logs and extract useful information.

Add the following filter configuration to your logstash.conf file:

plaintext

filter {
  grok {
    match => { "message" => "%{COMBINEDAPACHELOG}" }
  }
  date {
    match => [ "timestamp", "dd/MMM/yyyy:HH:mm:ss Z" ]
  }
}

This configuration will use the predefined COMBINEDAPACHELOG pattern to parse Apache logs and extract the timestamp.

Output Plugins

Output plugins are used to send the processed data to various destinations. In this example, we’ll use the stdout output plugin to print the processed logs to the terminal.

Add the following output configuration to your logstash.conf file:

plaintext

output {
  stdout {
    codec => rubydebug
  }
}

This configuration will print the processed logs to the terminal in a human-readable format.

Running Logstash with the Configuration File on Rocky Linux

To run Logstash with your configuration file, execute the following command:

bash

sudo systemctl start logstash

Now, Logstash will start processing the logs from the /var/log directory using the specified input, filter, and output plugins.

Note: If you want Logstash to start automatically at boot, run the following command:

bash

sudo systemctl enable logstash

Conclusion

In this tutorial, we have covered how to install and configure Logstash on Rocky Linux. We also learned how to create a basic configuration file for Logstash with input, filter, and output plugins. By following these steps, you can now use Logstash to process and analyze your log data on your Rocky Linux system.

For more information about Logstash, you can refer to the official Logstash documentation. Additionally, you can explore other tutorials on our website for further insights into various Linux applications and configurations: