search

timeplus-io / proton-grafana-source

Grafana data source plugin for Timeplus
Apache License 2.0
1 stars 0 forks source link

cardemo is broken? #1

Closed ayewo closed 2 months ago

ayewo commented 2 months ago

I tried running the docker-compose.yaml in this repo locally using docker compose up but got an error from cardemo (which is in the separate timeplus-io/chameleon repo):

04_grafana-carsharing_datagen-1  | logger initialization with level 4
04_grafana-carsharing_datagen-1  | time="2024-07-04 13:27:11.541 +0000" level=info msg="loaded config is {[{proton map[host:proton interval:200]}]}" func=github.com/timeplus-io/chameleon/cardemo/sink.loadConfig file="/build/sink/sink.go:53"
04_grafana-carsharing_datagen-1  | time="2024-07-04 13:27:11.541 +0000" level=info msg="load sinks from /timeplus/sink.yaml success with count 1" func=github.com/timeplus-io/chameleon/cardemo/demo.NewCarSharingDemoApp file="/build/demo/app.go:117"
04_grafana-carsharing_datagen-1  | panic: runtime error: invalid memory address or nil pointer dereference
04_grafana-carsharing_datagen-1  | [signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0xa05126]
04_grafana-carsharing_datagen-1  | 
04_grafana-carsharing_datagen-1  | goroutine 1 [running]:
04_grafana-carsharing_datagen-1  | github.com/timeplus-io/chameleon/cardemo/demo.NewCarSharingDemoApp()
04_grafana-carsharing_datagen-1  |  /build/demo/app.go:120 +0x506
04_grafana-carsharing_datagen-1  | github.com/timeplus-io/chameleon/cardemo/demo.Run(0xc0000c4700?, {0xc0000e8940?, 0x0?, 0x4?})
04_grafana-carsharing_datagen-1  |  /build/demo/demo.go:9 +0x17
04_grafana-carsharing_datagen-1  | github.com/spf13/cobra.(*Command).execute(0x1117580, {0xc0001341a0, 0x4, 0x4})
04_grafana-carsharing_datagen-1  |  /go/pkg/mod/github.com/spf13/cobra@v1.5.0/command.go:872 +0x6aa
04_grafana-carsharing_datagen-1  | github.com/spf13/cobra.(*Command).ExecuteC(0x1117580)
04_grafana-carsharing_datagen-1  |  /go/pkg/mod/github.com/spf13/cobra@v1.5.0/command.go:990 +0x38d
04_grafana-carsharing_datagen-1  | github.com/spf13/cobra.(*Command).Execute(...)
04_grafana-carsharing_datagen-1  |  /go/pkg/mod/github.com/spf13/cobra@v1.5.0/command.go:918
04_grafana-carsharing_datagen-1  | github.com/timeplus-io/chameleon/cardemo/cmd.Execute()
04_grafana-carsharing_datagen-1  |  /build/cmd/root.go:36 +0x105
04_grafana-carsharing_datagen-1  | main.main()
04_grafana-carsharing_datagen-1  |  /build/main.go:6 +0xf
04_grafana-carsharing_datagen-1 exited with code 2
...

After poring over the code used to build the cardemo docker image, I applied the following patch to docker-compose.yaml: 

-           - type: proton
+           - type: timeplus
+               address: http://proton:3218

but now get code 400 errors when I run docker compose up:

04_grafana-carsharing_datagen-1  | logger initialization with level 4
04_grafana-carsharing_datagen-1  | time="2024-07-04 21:39:01.209 +0000" level=info msg="loaded config is {[{timeplus map[address:http://proton:3218 host:proton interval:200]}]}" func=github.com/timeplus-io/chameleon/cardemo/sink.loadConfig file="/build/sink/sink.go:53"
04_grafana-carsharing_datagen-1  | time="2024-07-04 21:39:01.209 +0000" level=info msg="load sinks from /timeplus/sink.yaml success with count 1" func=github.com/timeplus-io/chameleon/cardemo/demo.NewCarSharingDemoApp file="/build/demo/app.go:117"
04_grafana-carsharing_datagen-1  | time="2024-07-04 21:39:01.209 +0000" level=info msg="Calling Stream Init" func="github.com/timeplus-io/chameleon/cardemo/sink/timeplus.(*TimeplusSink).initStream" file="/build/sink/timeplus/sink.go:331"
04_grafana-carsharing_datagen-1  | time="2024-07-04 21:39:01.216 +0000" level=warning msg="DimCarStream failed to init" func="github.com/timeplus-io/chameleon/cardemo/sink/timeplus.(*TimeplusSink).Init" file="/build/sink/timeplus/sink.go:291"
04_grafana-carsharing_datagen-1  | time="2024-07-04 21:39:01.515 +0000" level=info msg="success to load routes from ./routes.json" func="github.com/timeplus-io/chameleon/cardemo/demo.(*CarSharingDemoApp).initRoutes" file="/build/demo/app.go:154"
04_grafana-carsharing_datagen-1  | time="2024-07-04 21:39:01.517 +0000" level=error msg="failed to initialize data to stream dim_car_info, failed to ingest data into stream dim_car_info: request failed with status code 400, response body Code: 62. DB::Exception: Syntax error: failed at position 1 ('{'): {\"columns\":[\"cid\",\"license_plate_no\",\"in_service\"],\"data\":[[\"c00000\",\"I9RP3V\",true],[\"c00001\",\"WPNE5X\",true],[\"c00002\",\"NAKD4Q\",true],[\"c00003\",\"ZVBCLV\",true],[. Expected one of: Query, Query with output, EXPLAIN, SELECT query, possibly with UNION, list of union elements, SELECT query, subquery, possibly with UNION, SELECT subquery, SELECT query, WITH, SELECT, SUBSCRIBE TO SELECT query, possibly with UNION, SUBSCRIBE, UNSUBSCRIBE FROM SELECT query, possibly with UNION SETTINGS query_id=uuid, UNSUBSCRIBE, RECOVER FROM SELECT query, possibly with UNION SETTINGS query_id=uuid, RECOVER, SHOW CREATE QUOTA query, SHOW CREATE, SHOW CREATE FORMAT SCHEMA query, SHOW, SHOW [TEMPORARY] TABLES|DATABASES|CLUSTERS|CLUSTER 'name' [[NOT] [I]LIKE 'str'] [LIMIT expr], EXISTS or SHOW CREATE query, EXISTS, DESCRIBE FILESYSTEM CACHE query, DESCRIBE, DESC, DESCRIBE query, SHOW PROCESSLIST query, SHOW PROCESSLIST, CREATE STREAM or ATTACH STREAM query, CREATE, ATTACH, CREATE EXTERNAL TABLE query, CREATE DATABASE query, CREATE VIEW query, CREATE DICTIONARY, CREATE MATERIALIZED VIEW query, ALTER query, ALTER STREAM, ALTER DATABASE, RENAME query, RENAME STREAM, DROP query, DROP, DETACH, TRUNCATE, DROP EXTERNAL TABLE query, CHECK TABLE, KILL QUERY query, KILL, OPTIMIZE query, OPTIMIZE TABLE, WATCH query, WATCH, SHOW FORMAT SCHEMAS query, SHOW ACCESS query, SHOW ACCESS, ShowAccessEntitiesQuery, SHOW GRANTS query, SHOW GRANTS, SHOW PRIVILEGES query, SHOW PRIVILEGES, INSERT query, INSERT INTO, USE query, USE, SET ROLE or SET DEFAULT ROLE query, SET ROLE DEFAULT, SET ROLE, SET DEFAULT ROLE, SET query, SET, SYSTEM query, SYSTEM, CREATE USER or ALTER USER query, ALTER USER, CREATE USER, CREATE ROLE or ALTER ROLE query, ALTER ROLE, CREATE ROLE, CREATE QUOTA or ALTER QUOTA query, ALTER QUOTA, CREATE QUOTA, CREATE ROW POLICY or ALTER ROW POLICY query, CREATE SETTINGS PROFILE or ALTER SETTINGS PROFILE query, CREATE FUNCTION query, CREATE FORMAT SCHEMA query, DROP FORMAT SCHEMA query, DROP FUNCTION query, DROP access entity query, GRANT or REVOKE query, REVOKE, GRANT, EXTERNAL DDL query, EXTERNAL DDL FROM, TCL query, BEGIN TRANSACTION, COMMIT, ROLLBACK, SET TRANSACTION SNAPSHOT, BACKUP or RESTORE query, BACKUP, RESTORE. (SYNTAX_ERROR) (version 1.5.12)\n" func="github.com/timeplus-io/chameleon/cardemo/sink/timeplus.(*TimeplusSink).InitCars" file="/build/sink/timeplus/sink.go:373"
04_grafana-carsharing_datagen-1  | time="2024-07-04 21:39:01.525 +0000" level=error msg="failed to initialize data to stream dim_user_info, failed to ingest data into stream dim_user_info: request failed with status code 400, response body Code: 62. DB::Exception: Syntax error: failed at position 1 ('{'): {\"columns\":[\"uid\",\"first_name\",\"last_name\",\"email\",\"credit_card\",\"gender\",\"birthday\"],\"data\":[[\"u00000\",\"Ashlynn\",\"Mueller\",\"lolajewess@haley.net\",\"644402866721. Expected one of: Query, Query with output, EXPLAIN, SELECT query, possibly with UNION, list of union elements, SELECT query, subquery, possibly with UNION, SELECT subquery, SELECT query, WITH, SELECT, SUBSCRIBE TO SELECT query, possibly with UNION, SUBSCRIBE, UNSUBSCRIBE FROM SELECT query, possibly with UNION SETTINGS query_id=uuid, UNSUBSCRIBE, RECOVER FROM SELECT query, possibly with UNION SETTINGS query_id=uuid, RECOVER, SHOW CREATE QUOTA query, SHOW CREATE, SHOW CREATE FORMAT SCHEMA query, SHOW, SHOW [TEMPORARY] TABLES|DATABASES|CLUSTERS|CLUSTER 'name' [[NOT] [I]LIKE 'str'] [LIMIT expr], EXISTS or SHOW CREATE query, EXISTS, DESCRIBE FILESYSTEM CACHE query, DESCRIBE, DESC, DESCRIBE query, SHOW PROCESSLIST query, SHOW PROCESSLIST, CREATE STREAM or ATTACH STREAM query, CREATE, ATTACH, CREATE EXTERNAL TABLE query, CREATE DATABASE query, CREATE VIEW query, CREATE DICTIONARY, CREATE MATERIALIZED VIEW query, ALTER query, ALTER STREAM, ALTER DATABASE, RENAME query, RENAME STREAM, DROP query, DROP, DETACH, TRUNCATE, DROP EXTERNAL TABLE query, CHECK TABLE, KILL QUERY query, KILL, OPTIMIZE query, OPTIMIZE TABLE, WATCH query, WATCH, SHOW FORMAT SCHEMAS query, SHOW ACCESS query, SHOW ACCESS, ShowAccessEntitiesQuery, SHOW GRANTS query, SHOW GRANTS, SHOW PRIVILEGES query, SHOW PRIVILEGES, INSERT query, INSERT INTO, USE query, USE, SET ROLE or SET DEFAULT ROLE query, SET ROLE DEFAULT, SET ROLE, SET DEFAULT ROLE, SET query, SET, SYSTEM query, SYSTEM, CREATE USER or ALTER USER query, ALTER USER, CREATE USER, CREATE ROLE or ALTER ROLE query, ALTER ROLE, CREATE ROLE, CREATE QUOTA or ALTER QUOTA query, ALTER QUOTA, CREATE QUOTA, CREATE ROW POLICY or ALTER ROW POLICY query, CREATE SETTINGS PROFILE or ALTER SETTINGS PROFILE query, CREATE FUNCTION query, CREATE FORMAT SCHEMA query, DROP FORMAT SCHEMA query, DROP FUNCTION query, DROP access entity query, GRANT or REVOKE query, REVOKE, GRANT, EXTERNAL DDL query, EXTERNAL DDL FROM, TCL query, BEGIN TRANSACTION, COMMIT, ROLLBACK, SET TRANSACTION SNAPSHOT, BACKUP or RESTORE query, BACKUP, RESTORE. (SYNTAX_ERROR) (version 1.5.12)\n" func="github.com/timeplus-io/chameleon/cardemo/sink/timeplus.(*TimeplusSink).InitUsers" file="/build/sink/timeplus/sink.go:396"
04_grafana-carsharing_datagen-1  | time="2024-07-04 21:39:01.525 +0000" level=info msg="initialize demo car/user with number 100 100" func="github.com/timeplus-io/chameleon/cardemo/demo.(*CarSharingDemoApp).Start" file="/build/demo/app.go:241"
...

Can anyone tell me what I might be doing wrong?

jovezhong commented 2 months ago

Thanks I will check it

jovezhong commented 2 months ago

I can reproduce this. Assigning this to @gangtao to check code/config change of cardemo

gangtao commented 2 months ago

@ayewo the issue has been fixed with the latest cardemo image, please pull timeplus/cardemo:latest and try again

ayewo commented 2 months ago

Hi @gangtao

Thanks for getting on top of this.

I just tested now using the current docker-compose.yaml in this repo and I still get the same errors.

Could you share your copy of docker-compose.yaml known to work with the changes you just pushed?

Thanks!

jovezhong commented 2 months ago

Hi @ayewo , I updated the docker-compose file to add pull_policy: always so that you can get the new image. Other changes are minor (such as removing version and extra whitespace)