search

jshinonome / kola

a Polars Interface to kdb+/q
BSD 3-Clause "New" or "Revised" License
4 stars 0 forks source link

Limit error when query big tables #1

Closed ChaunceyDong closed 2 days ago

ChaunceyDong commented 3 days ago

hi,

Thank you for public the repo. It helps a lot.

I met a Error like QKolaError: Internal Server Error - "limit"

when q.sync("tbl")

the tblis a Kdb table with rows like 10,000,000

is there any way to unlimit the settings?

Thanks!

jshinonome commented 3 days ago

Hi, what is the size of the table, use -22!tbl to get the size.

ChaunceyDong commented 3 days ago

hi,

thx for the prompt responds

q)-22!tbl
22672823693
jshinonome commented 3 days ago

Unfortunately, the max size per IPC is 2,147,483,647. To support more than 2,147,483,647 requires extra work. I would suggest you to make multiple requests for the table.

https://www.timestored.com/kdb-guides/errors/limit

jshinonome commented 3 days ago

Going to release a new version to support IPC message up to 2TB this weekend.