search

airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
15.91k stars 4.09k forks source link

Latest 0.45.40 Helm Chart Results in POST 500 check_updates Error "Opps Something Went Wrong" In UI #27793

Open drewipsonhq opened 1 year ago

drewipsonhq commented 1 year ago

What method are you using to run Airbyte?

Kubernetes

Platform Version or Helm Chart Version

Helm Chart: 0.45.40 App Version: 0.50.5

What step the error happened?

Upgrading the Platform or Helm Chart

Revelant information

Currently have helm chart 0.45.38 deploying airbyte version 0.50.3. Attempting to upgrade to application 0.50.5 with helm chart 0.45.40 to leverage IRSA in S3 Destination.

Deployment of 0.45.38 from helm chart is clean. Webapp UI works as expected. Can create Sources, Upgrade Connectors via Settings, and I can visit the Settings page.

Once 0.45.40 is deployed the following errors occur in the Webapp Pod:

Connections: "POST /api/v1/web_backend/check_updates HTTP/1.1" 500 1127 "http://localhost:8000/workspaces/ca506f45-a88b-4516-a045-cbc6358a15e0/connections"

Adding Sources (Listing Sources gets 200): "POST /api/v1/source_definitions/list_for_workspace HTTP/1.1" 500

Destinations: 200 OK

I've included the relevant stack traces in the Logs Output section.

Relevant log output

Connections Stack Trace:
2023-06-28 07:27:36 ERROR i.a.s.a.ApiHelper(execute):40 - Unexpected Exception                                                                                                                                                 
io.airbyte.commons.server.errors.InternalServerKnownException: Unable to process retrieved latest source definitions list                                                                                                      
    at io.airbyte.commons.server.handlers.SourceDefinitionsHandler.buildSourceDefinitionRead(SourceDefinitionsHandler.java:120) ~[io.airbyte-airbyte-commons-server-0.50.5.jar:?]                                              
    at io.airbyte.commons.server.handlers.SourceDefinitionsHandler.lambda$toSourceDefinitionReadList$1(SourceDefinitionsHandler.java:149) ~[io.airbyte-airbyte-commons-server-0.50.5.jar:?]                                    
    at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) ~[?:?]                                                                                                                                        
    at java.util.AbstractList$RandomAccessSpliterator.forEachRemaining(AbstractList.java:722) ~[?:?]                                                                                                                           
    at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) ~[?:?]                                                                                                                                            
    at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) ~[?:?]                                                                                                                                     
    at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921) ~[?:?]                                                                                                                                       
    at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]                                                                                                                                            
    at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682) ~[?:?]                                                                                                                                           
    at io.airbyte.commons.server.handlers.SourceDefinitionsHandler.toSourceDefinitionReadList(SourceDefinitionsHandler.java:150) ~[io.airbyte-airbyte-commons-server-0.50.5.jar:?]                                             
    at io.airbyte.commons.server.handlers.SourceDefinitionsHandler.listSourceDefinitions(SourceDefinitionsHandler.java:134) ~[io.airbyte-airbyte-commons-server-0.50.5.jar:?]                                                  
    at io.airbyte.commons.server.handlers.WebBackendCheckUpdatesHandler.getSourceDiffCount(WebBackendCheckUpdatesHandler.java:83) ~[io.airbyte-airbyte-commons-server-0.50.5.jar:?]                                            
    at io.airbyte.commons.server.handlers.WebBackendCheckUpdatesHandler.checkUpdates(WebBackendCheckUpdatesHandler.java:49) ~[io.airbyte-airbyte-commons-server-0.50.5.jar:?]                                                  
    at io.airbyte.server.apis.ApiHelper.execute(ApiHelper.java:26) ~[io.airbyte-airbyte-server-0.50.5.jar:?]                                                                                                                   
    at io.airbyte.server.apis.WebBackendApiController.webBackendCheckUpdates(WebBackendApiController.java:70) ~[io.airbyte-airbyte-server-0.50.5.jar:?]                                                                        
    at io.airbyte.server.apis.$WebBackendApiController$Definition$Exec.dispatch(Unknown Source) ~[io.airbyte-airbyte-server-0.50.5.jar:?]                                                                                      
    at io.micronaut.context.AbstractExecutableMethodsDefinition$DispatchedExecutableMethod.invoke(AbstractExecutableMethodsDefinition.java:371) ~[micronaut-inject-3.9.3.jar:3.9.3]                                            
    at io.micronaut.context.DefaultBeanContext$4.invoke(DefaultBeanContext.java:594) ~[micronaut-inject-3.9.3.jar:3.9.3]                                                                                                       
    at io.micronaut.web.router.AbstractRouteMatch.execute(AbstractRouteMatch.java:246) ~[micronaut-router-3.9.3.jar:3.9.3]                                                                                                     
    at io.micronaut.web.router.RouteMatch.execute(RouteMatch.java:111) ~[micronaut-router-3.9.3.jar:3.9.3]                                                                                                                     
    at io.micronaut.http.context.ServerRequestContext.with(ServerRequestContext.java:103) ~[micronaut-http-3.9.3.jar:3.9.3]                                                                                                    
    at io.micronaut.http.server.RouteExecutor.lambda$executeRoute$14(RouteExecutor.java:659) ~[micronaut-http-server-3.9.3.jar:3.9.3]                                                                                          
    at reactor.core.publisher.FluxDeferContextual.subscribe(FluxDeferContextual.java:49) ~[reactor-core-3.5.0.jar:3.5.0]                                                                                                       
    at reactor.core.publisher.InternalFluxOperator.subscribe(InternalFluxOperator.java:62) ~[reactor-core-3.5.0.jar:3.5.0]                                                                                                     
    at reactor.core.publisher.FluxSubscribeOn$SubscribeOnSubscriber.run(FluxSubscribeOn.java:194) ~[reactor-core-3.5.0.jar:3.5.0]                                                                                              
    at io.micronaut.reactive.reactor.instrument.ReactorInstrumentation.lambda$init$0(ReactorInstrumentation.java:62) ~[micronaut-runtime-3.9.3.jar:3.9.3]                                                                      
    at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:84) ~[reactor-core-3.5.0.jar:3.5.0]                                                                                                                              
    at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:37) ~[reactor-core-3.5.0.jar:3.5.0]                                                                                                                              
    at io.micronaut.scheduling.instrument.InvocationInstrumenterWrappedCallable.call(InvocationInstrumenterWrappedCallable.java:53) ~[micronaut-context-3.9.3.jar:3.9.3]                                                       
    at java.util.concurrent.FutureTask.run(FutureTask.java:317) ~[?:?]                                                                                                                                                         
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]                                                                                                                                  
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]                                                                                                                                  
    at java.lang.Thread.run(Thread.java:1589) ~[?:?] 

Sources:
2023-06-28 07:28:53 INFO i.a.c.i.RemoteDefinitionsProvider(getRemoteConnectorRegistry):124 - Fetched latest remote definitions (-331956860)                                                                                    
2023-06-28 07:28:53 ERROR i.a.s.a.ApiHelper(execute):40 - Unexpected Exception                                                                                                                                                 
io.airbyte.commons.server.errors.InternalServerKnownException: Unable to process retrieved latest source definitions list                                                                                                      
    at io.airbyte.commons.server.handlers.SourceDefinitionsHandler.buildSourceDefinitionRead(SourceDefinitionsHandler.java:120) ~[io.airbyte-airbyte-commons-server-0.50.5.jar:?]                                              
    at io.airbyte.commons.server.handlers.SourceDefinitionsHandler.lambda$toSourceDefinitionReadList$1(SourceDefinitionsHandler.java:149) ~[io.airbyte-airbyte-commons-server-0.50.5.jar:?]                                    
    at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) ~[?:?]                                                                                                                                        
    at java.util.AbstractList$RandomAccessSpliterator.forEachRemaining(AbstractList.java:722) ~[?:?]                                                                                                                           
    at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) ~[?:?]                                                                                                                                            
    at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) ~[?:?]                                                                                                                                     
    at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921) ~[?:?]                                                                                                                                       
    at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]                                                                                                                                            
    at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682) ~[?:?]                                                                                                                                           
    at io.airbyte.commons.server.handlers.SourceDefinitionsHandler.toSourceDefinitionReadList(SourceDefinitionsHandler.java:150) ~[io.airbyte-airbyte-commons-server-0.50.5.jar:?]                                             
    at io.airbyte.commons.server.handlers.SourceDefinitionsHandler.listSourceDefinitionsForWorkspace(SourceDefinitionsHandler.java:170) ~[io.airbyte-airbyte-commons-server-0.50.5.jar:?]                                      
    at io.airbyte.server.apis.SourceDefinitionApiController.lambda$listSourceDefinitionsForWorkspace$6(SourceDefinitionApiController.java:122) ~[io.airbyte-airbyte-server-0.50.5.jar:?]                                       
    at io.airbyte.server.apis.ApiHelper.execute(ApiHelper.java:26) ~[io.airbyte-airbyte-server-0.50.5.jar:?]                                                                                                                   
    at io.airbyte.server.apis.SourceDefinitionApiController.listSourceDefinitionsForWorkspace(SourceDefinitionApiController.java:122) ~[io.airbyte-airbyte-server-0.50.5.jar:?]                                                
    at io.airbyte.server.apis.$SourceDefinitionApiController$Definition$Exec.dispatch(Unknown Source) ~[io.airbyte-airbyte-server-0.50.5.jar:?]                                                                                
    at io.micronaut.context.AbstractExecutableMethodsDefinition$DispatchedExecutableMethod.invoke(AbstractExecutableMethodsDefinition.java:371) ~[micronaut-inject-3.9.3.jar:3.9.3]                                            
    at io.micronaut.context.DefaultBeanContext$4.invoke(DefaultBeanContext.java:594) ~[micronaut-inject-3.9.3.jar:3.9.3]                                                                                                       
    at io.micronaut.web.router.AbstractRouteMatch.execute(AbstractRouteMatch.java:303) ~[micronaut-router-3.9.3.jar:3.9.3]                                                                                                     
    at io.micronaut.web.router.RouteMatch.execute(RouteMatch.java:111) ~[micronaut-router-3.9.3.jar:3.9.3]                                                                                                                     
    at io.micronaut.http.context.ServerRequestContext.with(ServerRequestContext.java:103) ~[micronaut-http-3.9.3.jar:3.9.3]                                                                                                    
    at io.micronaut.http.server.RouteExecutor.lambda$executeRoute$14(RouteExecutor.java:659) ~[micronaut-http-server-3.9.3.jar:3.9.3]                                                                                          
    at reactor.core.publisher.FluxDeferContextual.subscribe(FluxDeferContextual.java:49) ~[reactor-core-3.5.0.jar:3.5.0]                                                                                                       
    at reactor.core.publisher.InternalFluxOperator.subscribe(InternalFluxOperator.java:62) ~[reactor-core-3.5.0.jar:3.5.0]                                                                                                     
    at reactor.core.publisher.FluxSubscribeOn$SubscribeOnSubscriber.run(FluxSubscribeOn.java:194) ~[reactor-core-3.5.0.jar:3.5.0]                                                                                              
    at io.micronaut.reactive.reactor.instrument.ReactorInstrumentation.lambda$init$0(ReactorInstrumentation.java:62) ~[micronaut-runtime-3.9.3.jar:3.9.3]                                                                      
    at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:84) ~[reactor-core-3.5.0.jar:3.5.0]                                                                                                                              
    at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:37) ~[reactor-core-3.5.0.jar:3.5.0]                                                                                                                              
    at io.micronaut.scheduling.instrument.InvocationInstrumenterWrappedCallable.call(InvocationInstrumenterWrappedCallable.java:53) ~[micronaut-context-3.9.3.jar:3.9.3]                                                       
    at java.util.concurrent.FutureTask.run(FutureTask.java:317) ~[?:?]                                                                                                                                                         
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]                                                                                                                                  
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]                                                                                                                                  
    at java.lang.Thread.run(Thread.java:1589) ~[?:?]                                                                                                                                                                           
Caused by: java.lang.NullPointerException: Cannot invoke "io.airbyte.config.ActorDefinitionVersion.getDockerRepository()" because "sourceVersion" is null                                                                      
    at io.airbyte.commons.server.handlers.SourceDefinitionsHandler.buildSourceDefinitionRead(SourceDefinitionsHandler.java:109) ~[io.airbyte-airbyte-commons-server-0.50.5.jar:?]                                              
    ... 31 more
jkleinkauff commented 9 months ago

Same problem here. It seems this problem was not completely mitigated. Could someone help here?

marcosmarxm commented 5 months ago

Are you still experiencing the issue?