ClickHouse DB Installation for Nectus Netflow & Syslog Storage

, ,

ClickHouse Database Installation for Nectus Netflow & Syslog Storage

Requirements: Ubuntu Server 18.04.2 LTS (with SSH access)

NOTE: Although ClickHouse can be installed on several different flavors of Linux, Ubuntu Server 16.04 & 18.04 are the only supported Linux distributions for Nectus at this point.

More information about installation on other OS’s can be found here: https://clickhouse.yandex/docs/en/getting_started/

Step 1

Import the public key:

apt-get update

sudo apt-key adv –keyserver keyserver.ubuntu.com –recv C8F1E19FE0C56BD4

NOTE: It is recommended to import the public key if it’s a fresh Ubuntu install.

Otherwise you may get the following error when adding the repository:GPG error: http://repo.yandex.ru/clickhouse/deb/stable main/ Release: The following signatures couldn’t be verified because the public key is not available: NO_PUBKEY C8F1E19FE0C56BD4

Optional commands to run:

sudo apt-key adv –keyserver keyserver.ubuntu.com –recv E0C56BD4

Step 2

Create Clickhouse repository:

sudo apt-get install dirmngr

sudo apt-add-repository “deb http://repo.yandex.ru/clickhouse/deb/stable/ main/”

NOTE: Please edit the sources.list file if you receive the following error:

“E: Mailformed entry 55 in list file /etc/apt/sources.list”.

Delete the entry XX and save/exit the file. Perform the update (and upgrade if you wish):

sudo apt-get update

Step 3

Get the packages list for the latest updates and the dependencies:

apt-get update && apt-get upgrade

Step 4

Download and Install the ClickHouse packages:

sudo apt-get install -y clickhouse-server clickhouse-server-common clickhouse-client

sudo apt-get install –allow-unauthenticated clickhouse-server-common clickhouse-client clickhouse-server

Step 5

Start the Clickhouse server as a daemon:

sudo service clickhouse-server start

Step 6

Now that we have installed the ClickHouse, it is time to test:

NOTE: TCP ports 8123 & 9000 must be open.

Start the Client:

clickhouse-client

Step 7

Create the Netflow/Syslog database.

In this example we are creating the netflow database named NETFLOW (database name is arbitrary):

clickhouse-client

create database NETFLOW

Step 8

We will now add the Clickhouse internal user “root” with password “nectus” to the Users.xml file located at: /etc/clickhouse-server/users.xml

NOTE: Paste the snippet of the code below starting at line 31 after “<users>” in the users.xml file. You can use vi or nano text editor to edit the file. WinSCP can also be used to accomplish this task. Use the file change owner command if needed “sudo chown -R xxxx users.xml”, where “xxxx” is the user that will take over the ownership of the file.

<root>

<password>nectus</password>

<networks incl=”networks” replace=”replace”>

<ip>::/0</ip>

</networks>

<profile>default</profile>

<quota>default</quota>

</root>

Save the file and exit.

Step 9

Restart the ClickHouse Server:

clickhouse-server restart

Step 10

Now that the ClickHouse Server has been restarted, we can start the ClickHouse client using the internal user that we created in step 8:

clickhouse-client –user default –password nectus

Step 11

We have completed the ClickHouse installation. This last step requires login to Nectus to finish the Netflow/Syslog integration.

Open to “Nectus Settings -> General Settings -> Netflow Integration” page

Enter the required information and click Test DB Connection (Remote Server IP is the IP address of the Ubuntu/ClickHouse server). The result should be “Test DB Connection OK”

Click “Run Integration Scripts” and finally Save.

 

Step 12

Restart Nectus NetFlow and Syslog Services.

0 replies

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply