search

awslabs / aws-glue-libs

AWS Glue Libraries are additions and enhancements to Spark for ETL operations.
Other
623 stars 296 forks source link

Union transform #7

Open philipp94831 opened 6 years ago

philipp94831 commented 6 years ago

I would be really nice to have a union transform on DynamicFrames. Right now it is necessary to use DataFrames for this operation. With a dedicated union transform, visualization of jobs also becomes better. Right now, we have to use join type for this.

mohitsax commented 6 years ago

Thanks for using AWS Glue. We would add implementing union transform for dynamic frames to our list of feature requests for AWS Glue library.

Bingjiling commented 2 years ago

Hi, I'm wondering if there's any update on this request? It'll be wonderful to have this functionality.

csutphin commented 2 years ago

It appears there is a union transform added back in May of 2020, but the official documentation has not been updated to include it. It was added at the same time as mergeDynamicFrame. Could you advise if this is safe to use in a production setting? Thank you!