Closed HariprasadAllaka1612 closed 4 years ago
trying to understand, are you concurrently writing to the same dataset using two writers?
@vinothchandar No. let me be more clear. Below is the complete process i am doing
The problem here is when i am writing the data set for the first time, its working. But when i am trying UPSERT the data in the 2nd run its giving this error
@HariprasadAllaka1612 Not sure if I completely understand the context here.
Questions inline related to your descriptions ?
In general, this could be eventual consistency issue too. Does the path s3a://gat-datalake-refined-dev/reports/player/dat/2020/04/23 belong to the CDC table ? Does it actually exist when you do aws s3 ls ? Did CDC pipeline ran with consistency guard enabled ?
@HariprasadAllaka1612 : Were you able to resolve the issue ?
Closing due to inactivity.
Hi, When i am trying to write upsert 2 datasets into hoodie in one execution, I am having an exception saying input path doesnt exist on S3.
Environment Description
Hudi version :0.5.0
Spark version : 2.4.0
Hive version : 2.3.4
Hadoop version : 2.8.5
Storage (HDFS/S3/GCS..) : S3
Running on Docker? (yes/no) : No
Stacktrace