Handling errors caused by empty-file-object file placed in the Inbound bucket as part of the installation

aws-solutions / data-connectors-for-aws-clean-rooms

Simplify the process of selecting application sources and preparing data for collaborating in AWS Clean Rooms

Apache License 2.0

13 stars 4 forks source link

Handling errors caused by empty-file-object file placed in the Inbound bucket as part of the installation #19

Closed thyagram-aws closed 10 months ago

thyagram-aws commented 1 year ago

Is your feature request related to a problem? Please describe. The empty-file-object which is placed in the inbound bucket folder as part of the installation can cause issues with Databrew operations when they are invoked with no additional data files are available. Typical error messages:

Failure reading from input
Cannot process file

Describe the feature you'd like This file should either be automatically removed or an appropriate object should be placed to avoid errors.

thyagram-aws commented 1 year ago

To avoid these errors you can simply delete the file from the inbound bucket. A workaround is needed for this only when the InboundDataUploadType parameter is set to Incremental. No additional steps needed if the upload type is Bulk.