use_browser_authentication fails with "Connection is closed"

[larsrinn]

I have a pipeline working using username/password authentication. When I switch this to use_browser_authentication: true, it fails. It seems like the failure occurs at the very end. What happens is:

• The tap is executed and provides data
• A browser for Snowflake-SSO opens
• Data seems to be loaded into Snowflake
• Cleanup is executed
• The state is updated
• Failure due to closed connection

Relevant log messages

2024-07-17T11:49:45.428395Z [info     ] 2024-07-17 13:49:45,427 | INFO     | target-snowflake.<stream-name> | Cleaning up <stream-name>
2024-07-17T11:49:45.428644Z [info     ] 2024-07-17 13:49:45,427 | INFO     | target-snowflake     | Emitting completed target state 
2024-07-17T11:49:45.428861Z [info     ] 2024-07-17 13:49:45,428 | INFO     | snowflake.connector.connection | closed
2024-07-17T11:49:45.429008Z [info     ] 2024-07-17 13:49:45,428 | INFO     | snowflake.connector.connection | No async queries seem to be running, deleting session 
2024-07-17T11:49:45.448569Z [info     ] Incremental state has been updated at 2024-07-17 11:49:45.448552.
...
2024-07-17T11:49:45.723670Z [info     ] 2024-07-17 13:49:45,723 | ERROR    | sqlalchemy.pool.impl.QueuePool | Exception during reset or similar
<traceback>
2024-07-17T11:49:45.725255Z [info     ] snowflake.connector.errors.DatabaseError: 250002 (08003): None: Connection is closed
2024-07-17T11:49:45.783624Z [info     ] Block run completed.           block_type=ExtractLoadBlocks err=None set_number=0 success=True

[edgarrmondragon]

I'm not familiar with how browser authentication is by snowflake-connector-python under the hood, but perhaps we could try to connect early if use_browser_authentication.

[larsrinn]

Connecting works. Just looking at the logs (I don't know how the target works under the hood), it seems like it's trying to operate on an already closed connection

[edgarrmondragon]

Ah I see. The pipeline seems to have exited cleanly and the error is logged but doesn't crash the target. It might help to look at the debug level logs for snowflake.connector.connection in case there's an operation that it tries to execute after the connection is closed.

https://sdk.meltano.com/en/v0.38.0/implementation/logging.html#logging