Connector version reset when rebooting Airbyte Server and impossible to re-rupdate version (saleforce connector)

[tdebroc]

What method are you using to run Airbyte?

Kubernetes

Platform Version or Helm Chart Version

0.50.5

What step the error happened?

Other

Revelant information

In a summary

1. Launch Airbyte
2. Upgrade a connector version (Saleforce) from the UI
3. Reboot the instance
4. Check on the connectors UI, the version has been back to initial one
5. When we try to update the newer one from the UI, we have an error (stacktrace attached)
6. We are blocked !

Note: Maybe the problem is only with SF connector

More details to reproduce it easily:

 - Install airbyte helm install airbyte-test airbyte/airbyte --version 0.45.40
 - Upgrade connector (i tested salesforce-source): initial version 2.0.14 and upgraded to 2.1.4
 - Deploy with helm again, changing something completely unrelated: helm upgrade airbyte-test airbyte/airbyte --version 0.45.40 --set webapp.podLabels.foo=bar
 - Airbyte connector is back to 2.0.14 and there is no way to upgrade it again

Relevant log output

2023-08-31 10:51:12 ERROR i.a.s.a.ApiHelper(execute):40 - Unexpected Exception
java.lang.IllegalStateException: Get Spec job failed.
        at com.google.common.base.Preconditions.checkState(Preconditions.java:502) ~[guava-31.1-jre.jar:?]
        at io.airbyte.commons.server.converters.SpecFetcher.getSpecFromJob(SpecFetcher.java:18) ~[io.airbyte-airbyte-commons-server-0.50.5.jar:?]
        at io.airbyte.commons.server.handlers.SourceDefinitionsHandler.getSpecForImage(SourceDefinitionsHandler.java:341) ~[io.airbyte-airbyte-commons-server-0.50.5.jar:?]                          at io.airbyte.commons.server.handlers.SourceDefinitionsHandler.updateSourceDefinition(SourceDefinitionsHandler.java:271) ~[io.airbyte-airbyte-commons-server-0.50.5.jar:?]
        at io.airbyte.server.apis.SourceDefinitionApiController.lambda$updateSourceDefinition$8(SourceDefinitionApiController.java:142) ~[io.airbyte-airbyte-server-0.50.5.jar:?]                    at io.airbyte.server.apis.ApiHelper.execute(ApiHelper.java:26) ~[io.airbyte-airbyte-server-0.50.5.jar:?]
        at io.airbyte.server.apis.SourceDefinitionApiController.updateSourceDefinition(SourceDefinitionApiController.java:142) ~[io.airbyte-airbyte-server-0.50.5.jar:?]
        at io.airbyte.server.apis.$SourceDefinitionApiController$Definition$Exec.dispatch(Unknown Source) ~[io.airbyte-airbyte-server-0.50.5.jar:?]
        at io.micronaut.context.AbstractExecutableMethodsDefinition$DispatchedExecutableMethod.invoke(AbstractExecutableMethodsDefinition.java:371) ~[micronaut-inject-3.9.3.jar:3.9.3]
        at io.micronaut.context.DefaultBeanContext$4.invoke(DefaultBeanContext.java:594) ~[micronaut-inject-3.9.3.jar:3.9.3]
        at io.micronaut.web.router.AbstractRouteMatch.execute(AbstractRouteMatch.java:303) ~[micronaut-router-3.9.3.jar:3.9.3]
        at io.micronaut.web.router.RouteMatch.execute(RouteMatch.java:111) ~[micronaut-router-3.9.3.jar:3.9.3]
        at io.micronaut.http.context.ServerRequestContext.with(ServerRequestContext.java:103) ~[micronaut-http-3.9.3.jar:3.9.3]
        at io.micronaut.http.server.RouteExecutor.lambda$executeRoute$14(RouteExecutor.java:659) ~[micronaut-http-server-3.9.3.jar:3.9.3]
        at reactor.core.publisher.FluxDeferContextual.subscribe(FluxDeferContextual.java:49) ~[reactor-core-3.5.0.jar:3.5.0]
        at reactor.core.publisher.InternalFluxOperator.subscribe(InternalFluxOperator.java:62) ~[reactor-core-3.5.0.jar:3.5.0]
        at reactor.core.publisher.FluxSubscribeOn$SubscribeOnSubscriber.run(FluxSubscribeOn.java:194) ~[reactor-core-3.5.0.jar:3.5.0]
        at io.micronaut.reactive.reactor.instrument.ReactorInstrumentation.lambda$init$0(ReactorInstrumentation.java:62) ~[micronaut-runtime-3.9.3.jar:3.9.3]
        at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:84) ~[reactor-core-3.5.0.jar:3.5.0]
        at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:37) ~[reactor-core-3.5.0.jar:3.5.0]
        at io.micronaut.scheduling.instrument.InvocationInstrumenterWrappedCallable.call(InvocationInstrumenterWrappedCallable.java:53) ~[micronaut-context-3.9.3.jar:3.9.3]
        at java.util.concurrent.FutureTask.run(FutureTask.java:317) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]
        at java.lang.Thread.run(Thread.java:1589) ~[?:?]

[AndreaGiardini]

Relevant Slack conversation: https://airbytehq.slack.com/archives/C01AHCD885S/p1693400388057829

[AndreaGiardini]

What happens if we try to upgrade it again:

[tdebroc]

We upgraded to latest airbyte version from 0.50.5 to 0.50.24 and as it's the last version of the connectors, we don't have the problem anymore.

Also another test done "It looks in the latest version (0.50.22) the issue is not present anymore. I tried with Amazon Ads where the default version is 3.1.0 and update to 3.1.2 and after restarting the server the version continue at the same updated by me. "

We might be able to close the issue then