search

ntop / ntopng

Web-based Traffic and Security Network Traffic Monitoring
http://www.ntop.org
GNU General Public License v3.0
6.16k stars 648 forks source link

Ntopng PFSense - Error with clickhouse #7927

Closed ycatoireDS closed 2 months ago

ycatoireDS commented 11 months ago

Environment:

What happened: We have a problem connecting ntopng Enterprise M with our clickhouse server. Clickhouse does not seem to be recognized.

How did you reproduce it? When we run ntopng with the following parameter:

ntopng -F "clickhouse;10.XX.XX.XXX@9000;ntopng;default;MYPASSWORD"

We have this error : 

19/Oct/2023 11:38:38 [Prefs.cpp:2029] WARNING: -F mysql/-F clickhouse not available: missing MySQL support in ntopng
19/Oct/2023 11:38:38 [Redis.cpp:169] Successfully connected to redis 127.0.0.1@0
19/Oct/2023 11:38:38 [Redis.cpp:169] Successfully connected to redis 127.0.0.1@0
19/Oct/2023 11:38:38 [NtopPro.cpp:340] [LICENSE] Reading license from /usr/local/etc/ntopng.license
19/Oct/2023 11:38:38 [NtopPro.cpp:503] [LICENSE] /usr/local/etc/ntopng.license: found valid Enterprise M license

Same thing if i try to start the service with the correct configuration : image

and the configuration is the following : -F="clickhouse;10.XX.XX.XXX@9000;ntopng;default;MYPASSWORD"

From the same server (PFSense) if i try to connect to clickhouse-server, it works : image

Thanks for your help

lucaderi commented 11 months ago

This is the problem 19/Oct/2023 11:38:38 [Prefs.cpp:2029] WARNING: -F mysql/-F clickhouse not available: missing MySQL support in ntopng

Due to MySQL restrictions on pfSense this is the problem. I believe you are using a pfSense based on FreeBSD 14 that is the problem as it does not support TLS and because this is used by MySQL, we had to disable MySQL support.

lucaderi commented 11 months ago

As soon as FreeBSD14 is officially released with an officially supported TLS implementation (it should be OpenSSL v3) we can re-add MySQL support

dkggpeters commented 11 months ago

PFSense 23.09 is currently in beta which should resolve this issue as it uses OpenSSL v3. I am hoping it gets released this year.

ycatoireDS commented 11 months ago

Hi, Thanks you for your help. But i have a question, i don't understand why we depend on MySQL while we use clickhouse client and a clickhouse server.... Maybe I misunderstood how clickhouse works, it's my first use

dkggpeters commented 11 months ago

I am assuming that it uses Mysql for the backend to insert and/or query records.

lucaderi commented 10 months ago

MySQL is necessary to query data from ClickHouse. Soon FreeBSD 14 should be released https://www.freebsd.org/releases/14.0R/schedule/

ycatoireDS commented 10 months ago

Hi Luca,

Thanks for your reply. We will wait FreeBSD now..

ycatoireDS commented 10 months ago

Hi @lucaderi

PFSense 23.09 as been released with OpenSSL 3.0.12. Is it possible to re add MySQL support in NtopNG ?

Thank you

ycatoireDS commented 9 months ago

HI, any news ?

pachardata commented 4 months ago

Hello @lucaderi Can we get an update on this issue please ? makes ntopng unusable for us, if this issue is fixed we can get more licences Regards

cardigliano commented 3 months ago

Today's version (nightly build) fixes this.

Steps to update ntopng and install latest clickhouse to the correct path:

(ntopng) pkg update pkg upgrade

(clickhouse) curl https://clickhouse.com/ | sh mv clickhouse /usr/local/bin/