Open FedmahnK opened 1 week ago
FedmahnK
commented
1 week ago I answer to myself.
The three instances which showed clickhouse problems had problem in ntopng configuration file.
I had this :
-F clickhouse;127.0.0.1;my_database;default;pass********
And I changed it for this :
-F="clickhouse;127.0.0.1;my_databse;default;pass********"
( the = and the quote fix the main issue )
The warning in the logs are still occurring thou.
Regards
cardigliano
commented
1 week ago The clickhouse failures do not seem to be directly related to disaggregated interfaces as they seem to work fine in our lab.
cardigliano
commented
1 week ago @FedmahnK do you have .dsc files under /var/lib/ntopng/infra-cd25/tmp/clickhouse/ ?
FedmahnK
commented
1 week ago Yes, I have .dsc files in this folder. But no dsc files in others instances folders
cardigliano
commented
6 days ago Could you send us (direct email) one of the dsc?
MatteoBiscosi
commented
1 day ago Hi @FedmahnK can you please update and let me know if the issue persists? I did some fixes lately that could possibly fix your issue. In case if you can send again the query failing
FedmahnK
commented
16 hours ago Hi,
Thanks for the update. Unfortunatly, now the issue is on all instances. I cleaned up all data from clickhouse database (the system one had 90 gig of data), in order to restart with clean instance, but no success so far.
Now, I have 0 .dsc files.
Regards
Environment:
Hi, I Have four daemons/instances running. Three of them use disaggregated interfaces with theses parameters :
I cannot display historical data on these instances :
I am using clickhouse for dumping flows. I created one database per instance, but I am using the default account for handling the connexion. I tried for one database to create a specific user, but no success.
I see a lot of errors related to clickhouse on all the four instances (on the working one too):
nov. 06 15:26:46 ntop ntopng[5891]: 06/Nov/2024 15:26:46 [MySQLDB.cpp:46] ERROR: SQL error (1): Code: 62. DB::Exception: Syntax error: failed at position 230 ('Co'): Co, Ltd', 1730902855, 1730902856 WHERE NOT EXISTS ( SELECT 1 FROM asset_management WHERE key = '0_172.16.76.33@1132' ). Expected one of: token, Comma, FROM, PREWHERE, WHERE, GROUP BY, WITH, HAVING, WINDOW, QUALIFY, ORDER BY, LIMIT, OFFSET, FETCH, SETTINGS, UNION, EXCEPT, INTERSECT, FORMAT, end of query. (SYNTAX_ERROR) (version 24.10.1.2812 (official build))ornov. 06 15:29:05 ntop ntopng[5891]: 06/Nov/2024 15:29:05 [ClickHouseImport.cpp:136] WARNING: Unable to execute '/usr/bin/cat /var/lib/ntopng/infra-cd25/tmp/clickhouse/clickhouse-2-233-6.1730903288.csv | /usr/bin/clickhouse-client --port 9000 --host "127.0.0.1" --user "default" --password "*********" -d "ntop_infra_cd25" --format_csv_delimiter="|" --format_csv_allow_double_quotes true --query="INSERT INTO flows (FLOW_ID,IP_PROTOCOL_VERSION,FIRST_SEEN,LAST_SEEN,VLAN_ID,PACKETS,SRC2DST_PACKETS,DST2SRC_PACKETS,TOTAL_BYTES,SRC2DST_BYTES,DST2SRC_BYTES,SRC2DST_DSCP,DST2SRC_DSCP,PROTOCOL,IPV4_SRC_ADDR,IPV6_SRC_ADDR,IP_SRC_PORT,IPV4_DST_ADDR,IPV6_DST_ADDR,IP_DST_PORT,L7_PROTO,L7_PROTO_MASTER,L7_CATEGORY,FLOW_RISK,INFO,PROFILE,NTOPNG_INSTANCE_NAME,INTERFACE_ID,STATUS,SRC_COUNTRY_CODE,DST_COUNTRY_CODE,SRC_LABEL,DST_LABEL,SRC_MAC,DST_MAC,COMMUNITY_ID,SRC_ASN,DST_ASN,PROBE_IP,OBSERVATION_POINT_ID,SRC2DST_TCP_FLAGS,DST2SRC_TCP_FLAGS,SCORE,CLIENT_NW_LATENCY_US,SERVER_NW_LATENCY_US,CLIENT_LOCATION,SERVER_LOCATION,SRC_NETWORK_ID,DST_NETWORK_ID,INPUT_SNMP,OUTPUT_SNMP,SRC_HOST_POOL_ID,DST_HOST_POOL_ID,SRC_PROC_NAME,DST_PROC_NAME,SRC_PROC_USER_NAME,DST_PROC_USER_NAME,ALERTS_MAP,SEVERITY,ALERT_CATEGORY,IS_CLI_ATTACKER,IS_CLI_VICTIM,IS_CLI_BLACKLISTED,IS_SRV_ATTACKER,IS_SRV_VICTIM,IS_SRV_BLACKLISTED,ALERT_STATUS,IS_ALERT_DELETED,MINOR_CONNECTION_STATE,MAJOR_CONNECTION_STATE,PRE_NAT_IPV4_SRC_ADDR,PRE_NAT_SRC_PORT,PRE_NAT_IPV4_DST_ADDR,PRE_NAT_DST_PORT,POST_NAT_IPV4_SRC_ADDR,POST_NAT_SRC_PORT,POST_NAT_IPV4_DST_ADDR,POST_NAT_DST_PORT,WLAN_SSID,WTP_MAC_ADDRESS,DOMAIN_NAME,ALERT_JSON) FORMAT CSV" 2>'Regards