The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
We are running 3 Airbyte OSS clusters in Kubernetes with Helm. Each instance is in a separate Kubernetes cluster and they don't share any resources. All 3 of our environments run the same configuration and AIrbyte versions, but each instance shows different versions of various source and destination packages as being the latest versions. In some environments, the Upgrade all button for Sources and Destinations also does not always upgrade the packages to the same versions in each environment because it thinks the latest version is a different version. It seems that this is a function of when the helm chart is installed. If we nuke all 3 environments and reinstall them at the same time, the latest versions of the packages match. We expect this to be consistent between environments regardless of when it was deployed. Some examples:
MySQL source:
Building Airbyte determines your initial catalog. So build once and using those images across several environments SHOULD provide the same initial versions across instances
For latest versions, the Settings page will make an API request to our registry on page load.
There may be some cacheing on this call, so potentially if we aren't clearing that cache, we could serve different results for different instances
TLDR: We think our system should be working as requested by this user, so there may be a bug that is worth investigating
Helm Chart Version
0.50.20
What step the error happened?
Other
Revelant information
We are running 3 Airbyte OSS clusters in Kubernetes with Helm. Each instance is in a separate Kubernetes cluster and they don't share any resources. All 3 of our environments run the same configuration and AIrbyte versions, but each instance shows different versions of various source and destination packages as being the latest versions. In some environments, the
Upgrade all
button for Sources and Destinations also does not always upgrade the packages to the same versions in each environment because it thinks the latest version is a different version. It seems that this is a function of when the helm chart is installed. If we nuke all 3 environments and reinstall them at the same time, the latest versions of the packages match. We expect this to be consistent between environments regardless of when it was deployed. Some examples: MySQL source:Snowflake destination:
Relevant log output
No response