Elasticsearch - Secure communication Logstash & File-beat - Part 3

-

For logging pipeline, Filebeat will establish a secure connection to Logstash. This article explains how to setup SSL mutual authentication using self signed certificate. 

Prerequisite

You should have docker and docker-compose installed in you machine. You may try Docker-Desktop in case you are using WINDOWS or MAC.

You need OpenSSL installed and configured in your machine.

Option1: You can install Git Bash in your machine and configure path in System environment variable. 

Environment Variables -> path -> "C:\Program Files\Git\usr\bin"

Option2: You can use Cmder is a software package. 

Add a custom domain in your host machine which points to localhost.

"host" File location - C:\Windows\System32\drivers\etc

#My custom domain

127.0.0.1 dev.experiment.com

#End of section

Creating Certificates

We will be using custom domain and self-signed certificates.

Create directory.

mkdir .\tls\certs

mkdir .\tls\private

openssl req -subj "/CN=dev.experiment.com/" -x509 -days 3650 -batch -nodes -newkey rsa:2048 -keyout private/logstash.key -out certs/logstash.crt
openssl req -subj '/CN=dev.experiment.com/' -x509 -days 3650 -batch -nodes -newkey rsa:2048 -keyout private/filebeat.key -out certs/filebeat.crt

We will configure these keys in both logstash and file-beat.

Configure logstash.conf file.


Configure filebeat yml file 
 

Comments

Post a Comment

Popular posts from this blog

Python Development Environment : pyenv & VS Code

-
Image
While working on Python projects, we need to support projects with multiple versions of Python. Managing multiple versions in one machine is adifficult task. But if you use pyenv it becomes easy to manage. What is pyenv? pyenv is a tool to manage multiple Python versions. You may not need to install System Python if you use pyenv. Installing pyenv [Windows] Follow instructions as mentioned in this link  . If you are using WINDOWS operating system the I would prefer Chocolatey package manager. Chocolatey Installation Use powershell with admin rights > choco install pyenv-win To test if the installation is successful  > pyenv To see available python installation > pyenv install -l If you do not see you python version then, > pyenv update Installing pyenv [WSL2 - LINUX/Ubuntu] Install Home Brew package manager. follow the the instructions HomeBrew  and HomeBrew Installations > /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew...
Read more

Open Distro for Elasticsearch - Installation - Part 2

-
Image
Elasticsearch ecosystem has been in the popular choice for data and analytics purposes. In recent development, Elasticsearch is also available as license based  deployments. Apache 2.0 Licensed Elasticsearch which is free of cost and provides very basic features compared to paid/licensed Elastic stack.   Open Distro for Elasticsearch is an alternative which is open source(apache 2.0 licensed) and maintained by Amazon. Apart from providing Elasticsearch and Kibana, this distribution also contains security, event monitoring & alerting, performance analysis, SQL query features, SQL JDBC. Open Distro for Elasticsearch and Kibana are available as RPM and Docker containers. Open Distro for Elasticsearch features -  Component Purpose Elasticsearch Data store and search engine Kibana Search frontend and visualizations Security Authentication and access control for your cluster Alerting Receive notifications when your data meets certain conditions SQL Use SQL ...
Read more

WSL 2 on Windows - LINUX based development environment and Docker

-
Image
Windows operating system provides "Windows Subsystem for Linux" which can be utilized to develop using Linux operating system and have Linux based tools. Windows operating system is providing a great interoperability so that it is very easy to operate between Linux distribution and base operating system. I still remember the old days where we had to do so many things to have Linux OS with the base operating system. Popular choices were to have dedicated partitions to have different Operating systems or have Virtual Machine to host any OS of your choice. For both the cases, the development experience was not that great. Modern development is based on Containerization technologies and for Windows operating system, Docker Desktop is a popular choice to create docker environment in Windows operating system. Docker Desktop was using Hyper-v based virtualization technology and Linux OS to provide required environment. Later it started supporting WSL to provide containerization capa...
Read more