search

airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
16.2k stars 4.14k forks source link

Facebook Marketing API configuration setup failure with external secrets #46328

Open bgadrian opened 1 month ago

bgadrian commented 1 month ago

Connector Name

source-facebook-marketing

Connector Version

3.3.15

What step the error happened?

Configuring a new connector

Relevant information

Install Airbyte 1., OSS, Helm, via abctl with external AWS Secrets enabled and external database Try to setup a new source connector facebook marketing (oauth) Important step: Fill both inputs called ACCESS TOKENS (from auth and the other one) (PS I am not sure why there are 2 of them

Setup source

Internally the secret is created, example: airbyte_workspace_00000000-0000-0000-0000-000000000000_secret_8bdc4dc5-643e-4f4b-9bed-eeeb788e188e but somehow the plugin looks for the _v1 suffix airbyte_workspace_00000000-0000-0000-0000-000000000000_secret_8bdc4dc5-643e-4f4b-9bed-eeeb788e188e

Relevant log output

Internal message: io.airbyte.workload.launcher.pipeline.stages.model.StageError: java.lang.RuntimeException: That secret was not found in the store! Coordinate: airbyte_workspace_00000000-0000-0000-0000-000000000000_secret_8bdc4dc5-643e-4f4b-9bed-eeeb788e188e_v1 at io.airbyte.workload.launcher.pipeline.stages.model.Stage.apply(Stage.kt:46) at io.airbyte.workload.launcher.pipeline.stages.BuildInputStage.apply(BuildInputStage.kt:58) at io.airbyte.workload.launcher.pipeline.stages.$BuildInputStage$Definition$Intercepted.$$access$$apply(Unknown Source) at io.airbyte.workload.launcher.pipeline.stages.$BuildInputStage$Definition$Exec.dispatch(Unknown Source) at io.micronaut.context.AbstractExecutableMethodsDefinition$DispatchedExecutableMethod.invoke(AbstractExecutableMethodsDefinition.java:456) at io.micronaut.aop.chain.MethodInterceptorChain.proceed(MethodInterceptorChain.java:134) at io.airbyte.metrics.interceptors.InstrumentInterceptorBase.doIntercept(InstrumentInterceptorBase.kt:61) at io.airbyte.metrics.interceptors.InstrumentInterceptorBase.intercept(InstrumentInterceptorBase.kt:44) at io.micronaut.aop.chain.MethodInterceptorChain.proceed(MethodInterceptorChain.java:143) at io.airbyte.workload.launcher.pipeline.stages.$BuildInputStage$Definition$Intercepted.apply(Unknown Source) at io.airbyte.workload.launcher.pipeline.stages.BuildInputStage.apply(BuildInputStage.kt:39) at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:132) at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:158) at reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:2571) at reactor.core.publisher.MonoFlatMap$FlatMapMain.request(MonoFlatMap.java:194) at reactor.core.publisher.MonoFlatMap$FlatMapMain.request(MonoFlatMap.java:194) at reactor.core.publisher.MonoFlatMap$FlatMapMain.request(MonoFlatMap.java:194) at reactor.core.publisher.MonoFlatMap$FlatMapMain.request(MonoFlatMap.java:194) at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.set(Operators.java:2367) at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onSubscribe(FluxOnErrorResume.java:74) at reactor.core.publisher.MonoFlatMap$FlatMapMain.onSubscribe(MonoFlatMap.java:117) at reactor.core.publisher.MonoFlatMap$FlatMapMain.onSubscribe(MonoFlatMap.java:117) at reactor.core.publisher.MonoFlatMap$FlatMapMain.onSubscribe(MonoFlatMap.java:117) at reactor.core.publisher.MonoFlatMap$FlatMapMain.onSubscribe(MonoFlatMap.java:117) at reactor.core.publisher.FluxFlatMap.trySubscribeScalarMap(FluxFlatMap.java:193) at reactor.core.publisher.MonoFlatMap.subscribeOrReturn(MonoFlatMap.java:53) at reactor.core.publisher.Mono.subscribe(Mono.java:4560) at reactor.core.publisher.MonoSubscribeOn$SubscribeOnSubscriber.run(MonoSubscribeOn.java:126) at reactor.core.scheduler.ImmediateScheduler$ImmediateSchedulerWorker.schedule(ImmediateScheduler.java:84) at reactor.core.publisher.MonoSubscribeOn.subscribeOrReturn(MonoSubscribeOn.java:55) at reactor.core.publisher.Mono.subscribe(Mono.java:4560) at reactor.core.publisher.Mono.subscribeWith(Mono.java:4642) at reactor.core.publisher.Mono.subscribe(Mono.java:4403) at io.airbyte.workload.launcher.pipeline.LaunchPipeline.accept(LaunchPipeline.kt:50) at io.airbyte.workload.launcher.pipeline.consumer.LauncherMessageConsumer.consume(LauncherMessageConsumer.kt:28) at io.airbyte.workload.launcher.pipeline.consumer.LauncherMessageConsumer.consume(LauncherMessageConsumer.kt:12) at io.airbyte.commons.temporal.queue.QueueActivityImpl.consume(Internal.kt:87) at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) at java.base/java.lang.reflect.Method.invoke(Method.java:580) at io.temporal.internal.activity.RootActivityInboundCallsInterceptor$POJOActivityInboundCallsInterceptor.executeActivity(RootActivityInboundCallsInterceptor.java:64) at io.temporal.internal.activity.RootActivityInboundCallsInterceptor.execute(RootActivityInboundCallsInterceptor.java:43) at io.temporal.common.interceptors.ActivityInboundCallsInterceptorBase.execute(ActivityInboundCallsInterceptorBase.java:39) at io.temporal.opentracing.internal.OpenTracingActivityInboundCallsInterceptor.execute(OpenTracingActivityInboundCallsInterceptor.java:78) at io.temporal.internal.activity.ActivityTaskExecutors$BaseActivityTaskExecutor.execute(ActivityTaskExecutors.java:107) at io.temporal.internal.activity.ActivityTaskHandlerImpl.handle(ActivityTaskHandlerImpl.java:124) at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handleActivity(ActivityWorker.java:278) at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:243) at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:216) at io.temporal.internal.worker.PollTaskExecutor.lambda$process$0(PollTaskExecutor.java:105) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) at java.base/java.lang.Thread.run(Thread.java:1583) Caused by: java.lang.RuntimeException: That secret was not found in the store! Coordinate: airbyte_workspace_00000000-0000-0000-0000-000000000000_secret_8bdc4dc5-643e-4f4b-9bed-eeeb788e188e_v1 at io.airbyte.config.secrets.SecretsHelpers.getOrThrowSecretValue(SecretsHelpers.kt:288) at io.airbyte.config.secrets.SecretsHelpers.combineConfig(SecretsHelpers.kt:173) at io.airbyte.config.secrets.SecretsHelpers$combineConfig$1.invoke(SecretsHelpers.kt:183) at io.airbyte.config.secrets.SecretsHelpers$combineConfig$1.invoke(SecretsHelpers.kt:177) at io.airbyte.config.secrets.SecretsHelpers.combineConfig$lambda$2(SecretsHelpers.kt:177) at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133) at io.airbyte.config.secrets.SecretsHelpers.combineConfig(SecretsHelpers.kt:177) at io.airbyte.config.secrets.SecretsHelpers$combineConfig$1.invoke(SecretsHelpers.kt:183) at io.airbyte.config.secrets.SecretsHelpers$combineConfig$1.invoke(SecretsHelpers.kt:177) at io.airbyte.config.secrets.SecretsHelpers.combineConfig$lambda$2(SecretsHelpers.kt:177) at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133) at io.airbyte.config.secrets.SecretsHelpers.combineConfig(SecretsHelpers.kt:177) at io.airbyte.config.secrets.hydration.RealSecretsHydrator.hydrateFromDefaultSecretPersistence(RealSecretsHydrator.kt:21) at io.airbyte.config.secrets.SecretsRepositoryReader.hydrateConfigFromDefaultSecretPersistence(SecretsRepositoryReader.kt:60) at io.airbyte.workers.ConnectorSecretsHydrator.hydrateConfig(ConnectorSecretsHydrator.kt:33) at io.airbyte.workers.CheckConnectionInputHydrator.getHydratedStandardCheckInput(CheckConnectionInputHydrator.kt:13) at io.airbyte.workload.launcher.pipeline.stages.BuildInputStage.buildPayload(BuildInputStage.kt:90) at io.airbyte.workload.launcher.pipeline.stages.BuildInputStage.applyStage(BuildInputStage.kt:62) at io.airbyte.workload.launcher.pipeline.stages.BuildInputStage.applyStage(BuildInputStage.kt:39) at io.airbyte.workload.launcher.pipeline.stages.model.Stage.apply(Stage.kt:42) ... 51 more
Failure origin: airbyte_platform

Contribute

bgadrian commented 1 month ago

It seems that everytime I update the source token the following behaviours exists