~~Null Pointer Exception~~: centos

Though getting started with ELK itself is pretty straight-forward and simple; I ran into small hurdles lately and I thought it would make sense to make a note of it and share with all of you.

Step 1 : Missing JAVA
So the very first thing you will find missing is the JDK 8 which is mandatory for all these tools.
Please do not install OpenJDK1.8 as it has missing packages and will land you in trouble later. We need to install Oracle JAVA 8
You can download the RPM from here -
https://www.oracle.com/technetwork/java/javaee/downloads/jdk8-downloads-2133151.html

File downloaded: jdk-8u211-linux-x64.rpm

Make it executable : chmod +x jdk-8u211-linux-x64.rpm

Install: sudo yum install jdk-8u211-linux-x64.rpm

Set the JAVA_PATH: With Non Sudo user run these commands
vi .bash_profile

Add this line at end: export JAVA_HOME=/usr/java/jdk1.8.0_211-amd64/jre/bin

Step 2: Get Elastic Search
Please do not download the ZIP package. There is an RPM version available as well and it installs as a service. You can download it from here -
https://www.elastic.co/downloads/past-releases/elasticsearch-6-2-3

File downloaded: elasticsearch-6.2.3.rpm

Make it executable : chmod +x elasticsearch-6.2.3.rpm

Install: sudo yum install elasticsearch-6.2.3.rpm

Run:
sudo systemctl daemon-reload
sudo systemctl enable elasticsearch.service
systemctl start elasticsearch.service
systemctl status elasticsearch.service

Step 3: Get Kibana
Please do not download the ZIP package. There is an RPM version available as well and it installs as a service. You can download it from here -
https://www.elastic.co/downloads/past-releases/kibana-6-2-3

(NOTE: The version of Elastic Search and Kibana should be same)

File downloaded: kibana-6.2.3-x86_64.rpm

Make it executable : chmod +x kibana-6.2.3-x86_64.rpm

Install: sudo yum install kibana-6.2.3-x86_64.rpm

Run:
sudo systemctl daemon-reload
sudo systemctl enable kibana.service
systemctl start kibana.service
systemctl status kibana.service

Step 4: Configure Kibana Host in config file
Location of Config file: /etc/kibana/kibana.yml

Update these configurations -

server.host: "0.0.0.0"

elasticsearch.url: "http://localhost:9200"

elasticsearch.username: "admin"

elasticsearch.password: "password"

Step 5: Get Streamsets

You can download the zip from here:

https://archives.streamsets.com/datacollector/3.8.1/tarball/streamsets-datacollector-core-3.8.1.tgz

wget https://archives.streamsets.com/datacollector/3.8.1/tarball/streamsets-datacollector-core-3.8.1.tgz

Untar the .tgz file: tar -zxf streamsets-datacollector-core-3.8.1.tgz

Increase Ulimit:

vi /etc/security/limits.conf

Add these lines at end of file i.e.

* soft nofile 65000

* hard nofile 65000

Run: nohup streamsets-datacollector-3.8.1/bin/streamsets dc &

Step 6: Enable Ports in Firewall

Following ports need to be opened i.e. 5601, 9200

Hi,

If you are running low on swap memory in your centos environment; please follow these steps to extend the swap partition.

Step 1:
Since all the space in the physical disk would already be allocated to different partitions; there would not be any free space in your "Logical Volume" to give to swap partition.
You can see the volume group details by running the command > vgs

You can verify the logical volume details by running the command > lvs

You can see that there is no free space and my swap volume has 2GB space.

Step 2:

So, first logical step is to get some free space from other partition (probably from home partition as that has maximum space in the screenshot above) so that it can be allocated to swap partition

So, first we need to un-mount the home partition.

It can be done by following command > sudo umount -fl /home

Next to change the partition size, run the command > lvreduce -L 173.88G /dev/mapper/centos-home

NOTE: Please note that the size to be given here is the new total size i.e. if old size was 60GB and you want to reduce it to 50GB so that you get 10GB free space then give 50GB here.