GoogleCloudPlatform / DataflowTemplates

Cloud Dataflow Google-provided templates for solving in-Cloud data tasks
https://cloud.google.com/dataflow/docs/guides/templates/provided-templates
Apache License 2.0
1.14k stars 954 forks source link

cdc-change-applier INVALID_ARGUMENT: dataset id contains #310

Closed mitsuru-komaita closed 3 months ago

mitsuru-komaita commented 2 years ago

I get an error when I run the following https://github.com/GoogleCloudPlatform/DataflowTemplates/tree/master/v2/cdc-parent#running-the-pipeline

Input command

$ mvn exec:java -pl cdc-change-applier -Dexec.args="--runner=DataflowRunner \
              --inputSubscriptions=${PUBSUB_SUBSCRIPTIONS} \
              --updateFrequencySecs=300 \
              --changeLogDataset=${CHANGELOG_BQ_DATASET} \
              --replicaDataset=${REPLICA_BQ_DATASET} \
              --project=${GCP_PROJECT} \
              --useSingleTopic=true --region=US" -X

Error Description

[WARNING] 
com.google.api.gax.rpc.InvalidArgumentException: io.grpc.StatusRuntimeException: INVALID_ARGUMENT: dataset id contains '.' but is not under google, f1, napa, spanner, disa, or autotfx projects; Th
e two parts of the invalid dataset id are as follows: 'project=${PROJECT_ID}, dataset=cdc_${PREFIX}${DATABASE_NAME}. ${DB_NAME}.${TABLE_NAME}@us-central1. If you used the dotted string format for the dataset id and 
believe your dataset is actually valid, you may try the full resource name (projects/.../datasets/...).
    at com.google.api.gax.rpc.ApiExceptionFactory.createException (ApiExceptionFactory.java:49)
    at com.google.api.gax.grpc.GrpcApiExceptionFactory.create (GrpcApiExceptionFactory.java:72)
    at com.google.api.gax.grpc.GrpcApiExceptionFactory.create (GrpcApiExceptionFactory.java:60)
    at com.google.api.gax.grpc.GrpcExceptionCallable$ExceptionTransformingFuture.onFailure (GrpcExceptionCallable.java:97)
    at com.google.api.core.ApiFutures$1.onFailure (ApiFutures.java:68)
    at com.google.common.util.concurrent.Futures$CallbackListener.run (Futures.java:1349)
    at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute (MoreExecutors.java:398)
    at com.google.common.util.concurrent.AbstractFuture.executeListener (AbstractFuture.java:1024)
    at com.google.common.util.concurrent.AbstractFuture.complete (AbstractFuture.java:866)
    at com.google.common.util.concurrent.AbstractFuture.setException (AbstractFuture.java:711)
    at io.grpc.stub.ClientCalls$GrpcFuture.setException (ClientCalls.java:563)
    at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose (ClientCalls.java:533)
    at io.grpc.internal.DelayedClientCall$DelayedListener$3.run (DelayedClientCall.java:464)
    at io.grpc.internal.DelayedClientCall$DelayedListener.delayOrExecute (DelayedClientCall.java:428)
    at io.grpc.internal.DelayedClientCall$DelayedListener.onClose (DelayedClientCall.java:461)
    at io.grpc.internal.ClientCallImpl.closeObserver (ClientCallImpl.java:553)
    at io.grpc.internal.ClientCallImpl.access$300 (ClientCallImpl.java:68)
    at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal (ClientCallImpl.java:739)
    at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext (ClientCallImpl.java:718)
    at io.grpc.internal.ContextRunnable.run (ContextRunnable.java:37)
    at io.grpc.internal.SerializingExecutor.run (SerializingExecutor.java:123)
    at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:515)
    at java.util.concurrent.FutureTask.run (FutureTask.java:264)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run (ScheduledThreadPoolExecutor.java:304)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1128)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:628)
    at java.lang.Thread.run (Thread.java:829)

I'm having problems with the dot in the topic name, and I'd like to know how to work around it."

icang1694 commented 2 years ago

yes i also have this problem, please advise

github-actions[bot] commented 3 months ago

This issue has been marked as stale due to 180 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the issue at any time. Thank you for your contributions.

github-actions[bot] commented 3 months ago

This issue has been closed due to lack of activity. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.