near "s": syntax error

[pragnesh-highlevel]

Facing below error after scanning the whole log file downloaded from Atlas.

Error - 1 - 2023/09/28 23:58:31 mtools/hatchet v0.5.1-20230926 2023/09/28 23:58:31 using database ./data/hatchet.db 2023/09/28 23:58:31 processing ./../crm-standard-logs.gz 2023/09/28 23:58:31 hatchet name is crm_standard_logs_d80230 2023/09/28 23:58:31 fast counting ./../crm-standard-logs.gz ... 2023/09/28 23:58:32 counted 987200 lines 2023/09/28 23:58:32 creating hatchet crm_standard_logs_d80230 2023/09/28 23:58:32 CREATE TABLE IF NOT EXISTS hatchet ( name text not null primary key, version text, module text, arch text, os text, start text, end text ); 2023/09/28 23:58:32 DROP TABLE IF EXISTS crm_standard_logs_d80230; CREATE TABLE crm_standard_logs_d80230 ( id integer not null primary key, date text, severity text, component text, context text, msg text, plan text, type text, ns text, message text collate nocase, op text, filter text, _index text, milli integer, reslen integer ); 2023/09/28 23:58:32 DROP TABLE IF EXISTS crm_standard_logs_d80230_audit; CREATE TABLE crm_standard_logs_d80230_audit ( type text, name text, value integer ); 2023/09/28 23:58:32 DROP TABLE IF EXISTS crm_standard_logs_d80230_clients; CREATE TABLE crm_standard_logs_d80230_clients ( id integer not null primary key, ip text, port text, conns integer, accepted integer, ended integer, context text ); 2023/09/28 23:58:32 DROP TABLE IF EXISTS crm_standard_logs_d80230_drivers; CREATE TABLE crm_standard_logs_d80230_drivers ( id integer not null primary key, ip text, driver text, version text ); 2023/09/28 23:58:32 DROP TABLE IF EXISTS crm_standard_logs_d80230_ops; CREATE TABLE crm_standard_logs_d80230_ops ( op text, count integer, avg_ms numeric, max_ms integer, total_ms integer, ns text, _index text, reslen integer, filter text ); 2023/09/28 23:58:32 using 9 threads 2023/09/28 23:53:45 near "s": syntax error

[simagix]

Need more info to continue;,share either the log line or the log file. What MongoDB version?

[pragnesh-highlevel]

MongoDB Version 6.0.10 It was very large log file. More than 1GB zipped version.

Below are the last few lines from the log file.

{"t":{"$date":"2023-09-28T16:23:49.502+00:00"},"s":"I",  "c":"ACCESS",   "id":20249,   "ctx":"conn993702","msg":"Authentication failed","attr":{"mechanism":"SCRAM-SHA-256","speculative":true,"principalName":"crm-microservice","authenticationDatabase":"admin","remote":"10.13.7.32:35000","extraInfo":{},"error":"BadValue: SCRAM-SHA-256 authentication is disabled"}}
{"t":{"$date":"2023-09-28T16:23:49.504+00:00"},"s":"I",  "c":"ACCESS",   "id":20250,   "ctx":"conn993702","msg":"Authentication succeeded","attr":{"mechanism":"SCRAM-SHA-1","speculative":false,"principalName":"crm-microservice","authenticationDatabase":"admin","remote":"10.13.7.32:35000","extraInfo":{}}}
{"t":{"$date":"2023-09-28T16:23:56.752+00:00"},"s":"I",  "c":"NETWORK",  "id":22944,   "ctx":"conn993700","msg":"Connection ended","attr":{"remote":"192.168.255.249:48506","uuid":"a2a78d18-e40d-4a5b-8866-2d06e010ed38","connectionId":993700,"connectionCount":13925}}
{"t":{"$date":"2023-09-28T16:23:56.764+00:00"},"s":"I",  "c":"NETWORK",  "id":22943,   "ctx":"listener","msg":"Connection accepted","attr":{"remote":"192.168.255.249:48508","uuid":"0d2b61b5-609e-433e-8113-2e0db30d6868","connectionId":993703,"connectionCount":13926}}
{"t":{"$date":"2023-09-28T16:23:56.789+00:00"},"s":"I",  "c":"NETWORK",  "id":51800,   "ctx":"conn993703","msg":"client metadata","attr":{"remote":"192.168.255.249:48508","client":"conn993703","doc":{"driver":{"name":"mongo-go-driver","version":"v1.10.1-cloud"},"os":{"type":"linux","architecture":"amd64"},"platform":"go1.20.8","application":{"name":"MongoDB CPS Module v13.4.2.8420 (git: 1d515169c1cffaa436c650f565c658620c3fb600)"}}}}

[simagix]

I understand. Either you find out which line(s) failed or provide the entire file. From the lines you provided, the hatchet didn't fail. If you are familiar with Go, use debug mode to find the lines. Another way, which may take some effort, is to split the log file into many small ones. Run through them and you can narrow down the scope.

On Fri, Sep 29, 2023 at 8:32 AM pragnesh-highlevel @.***> wrote:

MongoDB Version 6.0.10 It was very large log file. More than 1GB zipped version.

Below are the last few lines from the log file.

{"t":{"$date":"2023-09-28T16:23:49.502+00:00"},"s":"I", "c":"ACCESS", "id":20249, "ctx":"conn993702","msg":"Authentication failed","attr":{"mechanism":"SCRAM-SHA-256","speculative":true,"principalName":"crm-microservice","authenticationDatabase":"admin","remote":"10.13.7.32:35000","extraInfo":{},"error":"BadValue: SCRAM-SHA-256 authentication is disabled"}} {"t":{"$date":"2023-09-28T16:23:49.504+00:00"},"s":"I", "c":"ACCESS", "id":20250, "ctx":"conn993702","msg":"Authentication succeeded","attr":{"mechanism":"SCRAM-SHA-1","speculative":false,"principalName":"crm-microservice","authenticationDatabase":"admin","remote":"10.13.7.32:35000","extraInfo":{}}} {"t":{"$date":"2023-09-28T16:23:56.752+00:00"},"s":"I", "c":"NETWORK", "id":22944, "ctx":"conn993700","msg":"Connection ended","attr":{"remote":"192.168.255.249:48506","uuid":"a2a78d18-e40d-4a5b-8866-2d06e010ed38","connectionId":993700,"connectionCount":13925}} {"t":{"$date":"2023-09-28T16:23:56.764+00:00"},"s":"I", "c":"NETWORK", "id":22943, "ctx":"listener","msg":"Connection accepted","attr":{"remote":"192.168.255.249:48508","uuid":"0d2b61b5-609e-433e-8113-2e0db30d6868","connectionId":993703,"connectionCount":13926}} {"t":{"$date":"2023-09-28T16:23:56.789+00:00"},"s":"I", "c":"NETWORK", "id":51800, "ctx":"conn993703","msg":"client metadata","attr":{"remote":"192.168.255.249:48508","client":"conn993703","doc":{"driver":{"name":"mongo-go-driver","version":"v1.10.1-cloud"},"os":{"type":"linux","architecture":"amd64"},"platform":"go1.20.8","application":{"name":"MongoDB CPS Module v13.4.2.8420 (git: 1d515169c1cffaa436c650f565c658620c3fb600)"}}}}

— Reply to this email directly, view it on GitHub https://github.com/simagix/hatchet/issues/31#issuecomment-1740817752, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEEPS6QZUNFWEZIQM3ZLATLX425UPANCNFSM6AAAAAA5LM7KHA . You are receiving this because you were assigned.Message ID: @.***>

[simagix]

I want to point out the last timestamp seemed invalid which was before Hatchet started. I am closing this one due to no activity. Feel free to reopen it if you have more info.

[simagix]

This can be an issue of single quote(s) is not escaped in the SQL statement. It happens after parsing logs is completed. @ninefyi FYI

[ninefyi]

I tested it and it worked. Thank you. @simagix