Open goetas opened 1 year ago
ok, seems a PDO thing... https://www.php.net/manual/en/mysqlinfo.concepts.buffering.php
is there a chance to offer such feature for snowflake PDO.... since it mainly contains large datasets
Also running into this issue. With MySQL you can do $pdo->setAttribute(PDO::MYSQL_ATTR_USE_BUFFERED_QUERY, false);
to make it use unbuffered queries. Would be great to have an equivalent in Snowflake, if it doesn't already eixst.
With postgres , it is possible to use FETCH FORWARD 5 FROM cursor_name;
... but in snowflake FETCH
can not be used outside of a stored procedure (and it fetches always one row)
hi - apologies for leaving this unattended for so long, we're changing that going forward.
we'll consider this enhancement request and i'll keep this thread posted on the progress, if any. In the meantime, you can consider using LIMIT .. OFFSET .. as a workaround and 'paginate' through very large query results in multiple iterations to lower memory usage.
yea, that is what we are doing, but is a query takes a large time to complete, running it multiple times with limit offset
makes it worse
What version of PDO/PHP driver are you using? PHP 8.2, Snowflake pdo 1.2.6
What operating system and processor architecture are you using? Debian buster
What version of C/C++ compiler are you using? gcc version 8.3.0 (Debian 8.3.0-6)
What did you do?
What did you expect to see?
PHP as expected reported a very low memory usage (~8mb) but the total memory used by the process peaked at 2gb (reported by
top
asVIRT
).My guess that the memory is used (and not released?) for the data buffering/transfer by the php extension.
Can you set logging to DEBUG and collect the logs?
What is your Snowflake account identifier, if any? (Optional) gqa33272