search

awslabs / aws-glue-libs

AWS Glue Libraries are additions and enhancements to Spark for ETL operations.
Other
635 stars 300 forks source link

Glue 3.0: write_dynamic_frame.from_options with DocumentDB should allow to set retryWrites=False #111

Open Nitjsefni7 opened 2 years ago

Nitjsefni7 commented 2 years ago

After upgrading to Glue 3.0 an error occur during writing dynamic frame to DocumentDB. DocumentDB does not support retryable writes, but with current connection options that Glue provides, there is no possibility to set retryWrites to false.

An error occurred while calling o365.pyWriteDynamicFrame. Command failed with error 301: 'Retryable writes are not supported' on server ****.*****.docdb.amazonaws.com:27017.

aoc-ot-psis commented 1 year ago

If you write the connection url like taht, it works: Append "/?retryWrites=false" at the end of the write connection. IMPORTANT: With the slash "/".

Nitjsefni7 commented 1 year ago

I am no longer using Dynamic Frames, therefore I am not able to verify If this approach will work, but if it does, it will be very helpful. As far as I am concerned, the issue can be closed.