Feature request: Mock, Fake, or Dev Docker image

[veqryn]

I would like to be able to run tests while offline, or in a CI environment that is offline. I would also like to be able to develop locally on my laptop while offline.

For most other data stores, I am able to use a docker image in development and test/ci environments. It isn't fast, but it doesn't need to be fast.

Does Snowflake have anything like this? And if not, can it please be considered?

[lfalkner]

Did you find any solution to this? I'm looking for the same, but for node :)

[veqryn]

depending on how complex your queries are you could use a Postgres for MySQL container in Dev to mock out snowflake.

[mihaitodor]

I attempted to get this driver to connect to a MySQL Docker container, but, unfortunately, it doesn't work. Since this code doesn't allow the authenticator parameter to be set to tokenaccessor, this connection string: testuser:testpass@localhost:3306/testdb/testdb?account=testuser&authenticator=tokenaccessor errors out and none of the other allowed authenticator values will work without pinging some remote system, which defeats the goal... There might be other blockers too, but I haven't tried to patch the driver to bypass this one.

[sfc-gh-dszmolka]

thank you for raising this request with us; we'll take a look if and how it would be feasible to provide. can not estimate any timeline though, so thank you for bearing with us !

[sfc-gh-pfus]

Hello everyone! Snowflake is a cloud platform, can't be installed on premise. It was build natively for the cloud. It consists of many components internally and making it available in a docker image would be extremely difficult.

On the other hand, preparing mock is some idea, but has a huge drawback - it should be kept up-to-date with the Snowflake cloud, so we are not planing to do it. We are planning to do some mocking, but only to cover corner cases, especially errors, that may be difficult to trigger from tests - so it doesn't answer your request too.

The closest thing to what you need is that we are planning to split tests into slow and fast ones (you can call it real unit tests and integration tests using Snowflake or stages). Still, even if we had this, it would test only less relevant cases. Not sure when this will happen though.

One last suggestion here - if your concern is not "offline mode", but not using production Snowflake for tests, you can always use trial. It is a bit inconvenient, because it is time limited, but it is better than nothing.

[mihaitodor]

Thank you for the update! I still think this would be a valuable utility to provide in the future, even if it will be very limited in capabilities. Let me share some concrete examples:

• 3rd party BigQuery emulator that's based on an embedded SQLite engine: https://github.com/goccy/bigquery-emulator It handles some (most?) of the BigQuery syntax and not much else
• Official CosmosDB emulator from Microsoft: https://learn.microsoft.com/en-us/azure/cosmos-db/emulator
• Official Azure Storage emulator from Microsoft (which could be regarded as similar to Snowflake stages): https://learn.microsoft.com/en-us/azure/storage/common/storage-use-azurite

These are quite limited in terms of features, but that's just fine for a small insert into foo(...) values(...) / select * from foo smoke test or a Snowpipe roundtrip test.

The alternative is for developers to add a layer of indirection in their code which wraps the relevant parts of database/driver so they can use dependency injection in their unit tests to provide a mock implementation. Or, alternatively, implementing a mock SQL driver might be another good approach. However, having a Docker container that one can spin up quickly to run some smoke tests can be quite convenient and it can be used together with any of the client SDKs.

At the very least, consider making it easy for the community to build and maintain such containers, similar to https://github.com/goccy/bigquery-emulator, by having a mechanism to bypass Snowflake authentication and provide a custom endpoint.

[sfc-gh-dszmolka]

hi all - after discussing this internally again, i'm going to close this issue as it has nothing to do with gosnowflake, mocking a SAAS service like Snowflake for local usage. The request is driver-independent.

As mentioned by my colleague before, as a workaround today for any tests which require Snowflake, the trial accounts can be used.

We can still see how this would be valuable to implement but it definitely should not come from the driver side; as it is only another consumer for Snowflake. Therefore if you are already a Snowflake customer and having a 'mock' Snowflake backend available to you would be important, please do reach out to your Account Team to let them know about your requirement.

[sfc-gh-pfus]

Guys, this comment may be very helpful. I haven't tried it myself, but looks very promising. Kudos to Stathi for bringing it to our attention!