malloydata / malloy

Malloy is an experimental language for describing data relationships and transformations.
http://www.malloydata.dev
MIT License
1.99k stars 76 forks source link

Snowflake pool sometimes persists a temporary table in one session then attempts to access it in another session #1815

Open christopherswenson opened 3 months ago

christopherswenson commented 3 months ago

There does not seem to be any system in place to make sure that, for a single Malloy compile, the Snowflake connection only uses one session. Because of this, when compiling a Malloy model, the Snowflake connection might persist a query result using one session, then attempt to access that saved result in a different session, which fails.

453646274_518404927320126_7411236290798720652_n

This is an example of such a failure

This seems to be relevant code

This is an example of a test that might exhibit this behavior

christopherswenson commented 3 months ago

Could possibly solve this by generating a "Malloy Compile ID" (UUID) in either the translator or the Runtime API, which would get passed to all connection operations, allowing the connection to ensure that the same session was used for the entirety of a Malloy compile lifecycle.