AirToSupply / spark-connector-influxdb

Spark Connector InfluxDB Data Source
Apache License 2.0
2 stars 0 forks source link

does it support InfluxDB2? #17

Open ufoe opened 1 month ago

ufoe commented 1 month ago

it seems not working with InfluxDB2

InfluxDB2 need to auth with token ,but this connector only support auth with username & password.

image image
ufoe commented 1 month ago

the username & password works with influxdb-java-client

AirToSupply commented 1 month ago

the username & password works with influxdb-java-client

Just support access token for latest release version, you can refer to v3.2.3.1 , If you have any new ideas, please submit your code.

ufoe commented 1 month ago

the username & password works with influxdb-java-client

Just support access token for latest release version, you can refer to v3.2.3.1 , If you have any new ideas, please submit your code.

v3.2.3.1 works for source example

the sink example seems success, but the data didn't show in the database

image
AirToSupply commented 1 month ago

the username & password works with influxdb-java-client

Just support access token for latest release version, you can refer to v3.2.3.1 , If you have any new ideas, please submit your code.

v3.2.3.1 works for source example

the sink example seems success, but the data didn't show in the database

image

You can show your complete code, but theoretically it works!Or if it works after you switch to username password mode.

AirToSupply commented 1 month ago

the username & password works with influxdb-java-client

Just support access token for latest release version, you can refer to v3.2.3.1 , If you have any new ideas, please submit your code.

v3.2.3.1 works for source example

the sink example seems success, but the data didn't show in the database

image

Hi, I have tried it and it works. I built an influxdb2.x mirror environment with the following command:

docker run -d -p 8086:8086 \
-v "/data/influxdb2/data:/var/lib/influxdb2" \
-v "/data/influxdb2/config:/etc/influxdb2" \
-e DOCKER_INFLUXDB_INIT_MODE=setup \
-e DOCKER_INFLUXDB_INIT_USERNAME=influxdb \
-e DOCKER_INFLUXDB_INIT_PASSWORD=influxdb \
-e DOCKER_INFLUXDB_INIT_ORG=org \
-e DOCKER_INFLUXDB_INIT_BUCKET=test_bucket \
influxdb:2

Then you can create access toke by http://localhost:8066 UI, and run FluxStreamSinkApplication in example package,and enter your access token in specfied token option.

So, you can enter nc -lk 7777 build streaming data,

image

And pay attendation to you timestamp in Line Protocol, Finally, you can see the incoming data on the InfluxDB dashboard:

Data Explore
AirToSupply commented 1 month ago

the username & password works with influxdb-java-client

Just support access token for latest release version, you can refer to v3.2.3.1 , If you have any new ideas, please submit your code.

v3.2.3.1 works for source example the sink example seems success, but the data didn't show in the database

image

Hi, I have tried it and it works. I built an influxdb2.x mirror environment with the following command:

docker run -d -p 8086:8086 \
-v "/data/influxdb2/data:/var/lib/influxdb2" \
-v "/data/influxdb2/config:/etc/influxdb2" \
-e DOCKER_INFLUXDB_INIT_MODE=setup \
-e DOCKER_INFLUXDB_INIT_USERNAME=influxdb \
-e DOCKER_INFLUXDB_INIT_PASSWORD=influxdb \
-e DOCKER_INFLUXDB_INIT_ORG=org \
-e DOCKER_INFLUXDB_INIT_BUCKET=test_bucket \
influxdb:2

Then you can create access toke by http://localhost:8066 UI, and run FluxStreamSinkApplication in example package,and enter your access token in specfied token option.

So, you can enter nc -lk 7777 build streaming data, image

And pay attendation to you timestamp in Line Protocol, Finally, you can see the incoming data on the InfluxDB dashboard: Data Explore

Generally speaking, the numInputRows indicator fed back to the terminal by Spark streaming represents the number of data items written to the external storage. If your InfluxDB does not query the data, it means that the external storage is unreliable. At the same time, as time series data, the granularity of time is generally very fine. You can track the streaming data through the data dashboard in the InfluxDB WebUI through the Custom time range.

AirToSupply commented 1 month ago

How to build Influxdb by docker image, it refer to Influxdb Docker Official Image