vertica / VerticaPy

VerticaPy is a Python library that exposes sci-kit like functionality to conduct data science projects on data stored in Vertica, thus taking advantage Vertica’s speed and built-in analytics and machine learning capabilities.
https://www.vertica.com/python/
Apache License 2.0
218 stars 44 forks source link

Excessive time duration on a vDF creation #1235

Open oualib opened 1 month ago

oualib commented 1 month ago

Discussed in https://github.com/vertica/VerticaPy/discussions/1234

Originally posted by **corentinlelannier** June 17, 2024 Hello, We detected a different behaviour of verticapy between the v0.12.0 and the v1.0.3. When we construct a vDF on the V0.12.0, each request sent have the parameter "limit 0". So, each sent request doesn't have to wait for an answer but it just tests if the request is correct. On the v1.0.3, the parameter seems to have been removed. So, each request waits for an answer. These simple requests only need few seconds to be executed on the V0.12.0, but a long time on the V1.0.3. base = vpy.vDataFrame(input_relation = "schema.base", ) base = base.select(['"COLUMNS1"', '"COLUMNS2"', '"COLUMNS3"']) base = base.search(PERIODE_FILTER) Here are the the durations of the query, on both versions : on v0.12.0 ![image](https://github.com/vertica/VerticaPy/assets/43996984/b12a2070-e3ae-407b-9c4d-fa0c9bd39ea7) on v1.0.3 ![image](https://github.com/vertica/VerticaPy/assets/43996984/f0f837c2-e396-4622-a0d8-e4816c340649) How can we create a vDF like the V0.12.0, with the parameter limit 0 or equivalent on V1.0.3 ? Regards, Corentin