I'm starting to experiment with the idea of connecting to a DuckDB database running in a remote process (e.g., using https://github.com/jwills/buenavista ) and realized that I would need a mechanism to handle certain bits of functionality that are currently done locally (e.g., loading a fsspec filesystem or executing a Python model) by sending code/instructions to the remote process.
To lay the foundation for this work, I'm refactoring the connections module (which has become a bit overloaded with code tbh) into a credentials module (for, well, the credentials) and an environments module that will allow me to define different kinds of execution environments for a particular dbt-duckdb run.
I'm starting to experiment with the idea of connecting to a DuckDB database running in a remote process (e.g., using https://github.com/jwills/buenavista ) and realized that I would need a mechanism to handle certain bits of functionality that are currently done locally (e.g., loading a
fsspecfilesystem or executing a Python model) by sending code/instructions to the remote process.To lay the foundation for this work, I'm refactoring the
connectionsmodule (which has become a bit overloaded with code tbh) into acredentialsmodule (for, well, the credentials) and anenvironmentsmodule that will allow me to define different kinds of execution environments for a particular dbt-duckdb run.