chdb-io / chdb

chDB is an in-process OLAP SQL Engine 🚀 powered by ClickHouse
https://clickhouse.com/chdb
Apache License 2.0
2.13k stars 75 forks source link

query remote clickhouse instance #33

Closed alanpaulkwan closed 1 year ago

alanpaulkwan commented 1 year ago

Suppose I would want to read foreign Clickhouse data.

If I typed the following:

select * from remote('database:port','database.table','user','password')

If I understand correctly, it just returns "Ok". and doesn't process the query

Did I do something wrong? It seems like potentially desirable behavior to be able to read directly from Clickhouse instances.

lmangani commented 1 year ago

Hello @alanpaulkwan I can perform remote queries to clickhouse servers just fine using the latest release:

# python3 -m chdb "select * from remote('XXXXXXXX:9000','system.disks','default','XXXXXXXXXXX')"

"backups","/backups/",3505403396096,3910140764160,3505403396096,0,"local",0,0,0,0,0,""
"default","/var/lib/clickhouse/",3505403396096,3910140764160,3505403396096,0,"local",0,0,0,0,0,""
"s3","/var/lib/clickhouse/disks/s3_disk/",18446744073709551615,18446744073709551615,18446744073709551615,0,"s3",0,0,0,1,0,""

Same goes for the libchdb bindings, for example in go:

# ./chdbgo -query "select * from remote('XXXXXXXX:9000','system.disks','default','XXXXXXXXXXX')"

"backups","/backups/",3505439924224,3910140764160,3505439924224,0,"local",0,0,0,0,0,""
"default","/var/lib/clickhouse/",3505439924224,3910140764160,3505439924224,0,"local",0,0,0,0,0,""
"s3","/var/lib/clickhouse/disks/s3_disk/",18446744073709551615,18446744073709551615,18446744073709551615,0,"s3",0,0,0,1,0,""

It even works in the public demo 😄

image

Could you confirm your running environment and installed version?

alanpaulkwan commented 1 year ago

I was using the demo version on Github. My fault, can try in Python later.

lmangani commented 1 year ago

I was using the demo version on Github. My fault, can try in Python later.

Thanks @alanpaulkwan which demo version are you referring to? This should work from everywhere, including the public demo (which I used above, too) as long as credentials are fine. Perhaps this was a secure connection? if so remoteSecure should be used instead of remote

alanpaulkwan commented 1 year ago

ch.fly.dev

image

I entered your query which obviously shouldn't run, since I don't know the IP or real pw

lmangani commented 1 year ago

Here's a working query you can try using the public clickhouse play server:

select count(*) from remoteSecure('play.clickhouse.com:9440','default.covid','play','play')
alanpaulkwan commented 1 year ago

Let's close this - I got it working on my latest pip install!

lmangani commented 1 year ago

Thanks @alanpaulkwan I understand the report now: the web demo does NOT return full error messages (the library does) and perhaps this was the confusing bit all along. Glad you got this working, and welcome to our community!