Till Studer's Notes

Recent Notes

8 min read

Elastic Stack Setup

Overview

1

End To End Incident Response Using Elastic Security

To get a feeling for how the user flow of an Analyst might look like, check out this Video: https://www.youtube.com/watch?v=C5bYHCjF4qI

Setting up the Basics

In this guide, based on the Elastic tutorials 2 3, we’ll set up the following:

Step 1: Set up the First Elasticsearch Node

  1. Create the following VM: 
    NAME: "Target-IT-Elasticsearch1"
DISK: "500 GiB"
CORES: "8 Threads"
MEMORY: "32768 MiB"
NETWORK: "TarIT"
  2. Installing Elasticsearch from the APT repository

    NOTE

    Make sure you have the correct IP address for your server before installing Elasticsearch

    ⇒ Force DHCP Client Release

    wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg
sudo apt-get install apt-transport-https
echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-8.x.list
sudo apt-get update && sudo apt-get install elasticsearch
    4
  3. Copy the terminal output from the installation command to a local file (you’ll need that info later)
  4. Store the elastic password in your shell: 
    export ELASTIC_PASSWORD="your_password"
  5. Enable Elasticsearch to run as a service: 
    sudo systemctl daemon-reload
sudo systemctl enable elasticsearch.service

Step 2: Configure the First Elasticsearch Node for Connectivity

  1. Copy the host’s IP somewhere (using ip a)
  2. Uncomment and edit the following lines of the config: 
    sudo vim /etc/elasticsearch/elasticsearch.yml
    cluster.name: elasticsearch-infosec
network.host: 10.2.2.50
transport.host: 0.0.0.0

Step 3: Start Elasticsearch

Now, it’s time to start the service and make sure it’s running properly:

sudo systemctl start elasticsearch.service
sudo curl --cacert /etc/elasticsearch/certs/http_ca.crt -u elastic:$ELASTIC_PASSWORD https://localhost:9200
sudo systemctl status elasticsearch

Step 4: Set up Additional Elasticsearch Nodes

Elastic scales very well horizontally, but not very well vertically. So if you are experiencing performance issues, you need more nodes.5

To set up additional nodes, please have a look at steps 4 and 5 in the article this notes is based on:

Step 5: Install Kibana

  1. Create the following VM: 
    NAME: "Target-IT-Kibana1"
DISK: "16 GiB"
CORES: "1 Threads"
MEMORY: "4096 MiB"
NETWORK: "TarIT"
  2. Installing Kibana from the APT repository 
    wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg
sudo apt-get install apt-transport-https
echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-8.x.list
sudo apt-get update && sudo apt-get install kibana
    6
  3. Back on the Kibana host, enable Kibana: 
    sudo systemctl daemon-reload
sudo systemctl enable kibana.service
  4. Copy the host’s IP somewhere (using ip a)
  5. Generate a new encryption keys: 
    sudo /usr/share/kibana/bin/kibana-encryption-keys generate
  6. Uncomment and edit the following lines of the config: 
    sudo vim /etc/kibana/kibana.yml
    server.host: 10.2.2.51
xpack.encryptedSavedObjects.encryptionKey: "min-32-byte-long-strong-encryption-key"
xpack.reporting.encryptionKey: "min-32-byte-long-strong-encryption-key"
xpack.security.encryptionKey: "min-32-byte-long-strong-encryption-key"
    7
  7. Start the service and make sure it’s running properly: 
    sudo systemctl start kibana.service
sudo systemctl status kibana
  8. In the status, copy out the URL with a 6 digit verification code to get started. It should look something like this: 
    Go to http://10.2.2.51:5601/?code=000000 to get started.
  9. Generate a Kibana enrollment token on the first Elasticsearch Node: 
    sudo /usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibana
  10. Open the website and paste the enrolment token we generated previously on the Elasticsearch Node
  11. When you see the Welcome to Elastic page, provide the username elastic the password that you copied when we installed the first Elasticsearch Node (the ELASTIC_PASSWORD one)

Step 6: Securing the Elastic Stack

NOTE

Beginning with Elastic 8.0, security is enabled in the Elastic Stack by default, meaning that traffic between Elasticsearch nodes and between Kibana and Elasticsearch is SSL-encrypted. While this is suitable for testing non-production viability of the Elastic platform, most production networks have requirements for the use of trusted CA-signed certificates. These steps demonstrate how to update the out-of-the-box self-signed certificates with your own trusted CA-signed certificates.

⇒ https://www.elastic.co/guide/en/elastic-stack/current/install-stack-demo-secure.html

Step 7: Install Fleet Server

  1. Create the following VM: 
    NAME: "Target-IT-FleetServer1"
DISK: "16 GiB"
CORES: "1 Threads"
MEMORY: "4096 MiB"
NETWORK: "TarIT"
  2. Create a working directory: 
    mkdir fleet-install-files
cd fleet-install-files
  3. Copy the host’s IP somewhere (using ip a)
  4. Back to your web browser, open the Kibana menu and go to Management -> Fleet
  5. Click Add Fleet Server
  6. Specify a name for your Fleet Server host, for example FleetServer1.
  7. Specify the host URL where Elastic Agents will reach Fleet Server, for example: https://10.2.2.52:8220 (use the IP from the host we just set up)
  8. Click Generate Fleet Server policy
  9. On the Install Fleet Server to a centralized host step, select the Linux Tar tab
  10. Copy and paste the generated commands in the terminal on your Fleet Server host
  11. At the prompt, enter Y to install Elastic Agent and run it as a service. Wait for the installation to complete
  12. In the Kibana Add a Fleet Server flyout, wait for confirmation that Fleet Server has connected
  13. For now, ignore the Continue enrolling Elastic Agent option and close the flyout

Step 8: Install Elastic Agent

  1. Log in to the host where you’d like to set up Elastic Agent
  2. Create a working directory for the installation package: 
    mkdir agent-install-files
cd agent-install-files
  3. Open Kibana and go to Management -> Fleet
  4. Open the Settings tab
  5. Reopen the Agents tab and select Add agent
  6. In the flyout, choose a policy name, for example Demo Agent Policy.
  7. Leave Collect system logs and metrics enabled.
  8. Click Create policy.
  9. For the “Enroll in Fleet?” step, leave Enroll in Fleet selected
  10. On the Install Elastic Agent on your host step, for this example we select the Linux Tar tab
  11. Copy and paste the generated commands in the terminal on your Fleet Server host
  12. Append an --insecure flag at the end.

    Tutorial 2: Securing a self-managed Elastic Stack.

    If you want to set up secure communications using SSL certificates, refer to

  13. At the prompt, enter Y to install Elastic Agent and run it as a service. Wait for the installation to complete
  14. In the Kibana Add a Fleet Server flyout, wait for confirmation that Elastic Agent has connected

Setting up the Security

Step 1: Add Elastic Defend Integration

  1. Go to the Integrations page, by going to Kibana: Management -> Integrations
  2. Search for and select Elastic Defend, then select Add Elastic Defend
  3. Configure the Elastic Defend integration with an Integration name, for example: elastic-defend-policy
  4. Select Traditional Endpoints as the type of environment you want to protect
  5. Select Complete EDR as the configuration preset
  6. Click the Existing hosts tab and select the Demo one we just created.
  7. Save and continue

8

Step 2: Test if the Integration is Working

  1. Log into an Endpoint where the Elastic Agent has been deployed
  2. Download a EICAR file: wget https://secure.eicar.org/eicar.com.txt
  3. Under Security -> Alerts you should now see some alerts:

Step 3: Enable Detection Rules

  1. Go to the Rules page, by going to Kibana: Security -> Rules -> Detection rules
  2. Click on Add Elastic rules
  3. Click on Install all
  4. Now go back and click on Select all 1171 rules
  5. Under Bulk actions, select Enable

Step 4: Configure Auditd & Windows Integrations

Auditd Manager

Install Auditd on Your Clients
sudo apt update
sudo apt install auditd
Setup the Auditd Manager Integration

In this lab I’m going to use Audit Deamon Rules, so we’re going to copy and paste those rule into the Audit rules field:

Make sure to use the existing Demo Agent Policy:

Make Sure Your Agents Are Still Healthy

If not you might need to change the Auditd rules.

9

Windows

  1. Head to Managment -> Integrations and search for an Integration called Windows:
  2. Set the toggles you want and install it to the Demo Agent Policy we’ve already created:

Step 5: Configure Osquery Manager Integration

Make sure to use the existing Demo Agent Policy:

Once installed, enable all existing packs under Managment -> Osquery:

10 11

(Optional) Step 6: Enable pfSense/OPNsense Integration

Install the Integration on Kibana

  1. Go to the Integrations page, by going to Kibana: Management -> Integrations
  2. Search for and select pfSense.
  3. Give the Integration a name, for example: opnsense-1
  4. Set the Syslog Host to 0.0.0.0
  5. Set the Syslog Port to 9001
  6. Click the Existing hosts tab and select Fleet Server Policy

Configure Remote Logging on OPNsense

  1. Navigate to System -> Settings -> Logging
  2. Click on the Remote tab
  3. Add a new Logging/Target (Click the plus icon)
    • Transport = UDP
    • Applications = Nothing Selected (this will forwar all logs)
    • Levels = Nothing Selected
    • Facilities = Nothing Selected
    • Hostname = IP of Fleet Server 10.2.2.52
    • Port = 9001
    • Description = Syslog to Elasticsearch
  4. Click Save
  5. And don’t forget to hit Apply

(Optional) Step 7: Enable Additional Integrations

  • AbuseCH
  • Network Packet Capture
  • Defend for Containers
  • Lateral Movement Detection
  • Data Exfiltration Detection
  • Living off the Land Attack Detection
  • Domain Generation Algorithm Detection

Relevant Note(s):

Footnotes

  1. https://www.elastic.co/guide/en/starting-with-the-elasticsearch-platform-and-its-solutions/current/stack-components.html ↩

  2. https://www.elastic.co/guide/en/elastic-stack/current/installing-stack-demo-self.html ↩

  3. https://www.elastic.co/guide/en/elastic-stack/current/install-stack-demo-secure.html ↩

  4. https://www.elastic.co/guide/en/elasticsearch/reference/current/deb.html ↩

  5. https://blog.octoperf.com/elasticsearch-optimization-guide/ ↩

  6. https://www.elastic.co/guide/en/kibana/current/deb.html ↩

  7. https://www.elastic.co/guide/en/kibana/current/xpack-security-secure-saved-objects.html ↩

  8. https://www.elastic.co/guide/en/security/current/install-endpoint.html#add-security-integration ↩

  9. https://www.elastic.co/security-labs/linux-detection-engineering-with-auditd ↩

  10. https://www.elastic.co/guide/en/kibana/current/manage-osquery-integration.html ↩

  11. https://www.elastic.co/guide/en/kibana/current/osquery.html ↩

Graph View

Backlinks