This is expected to produce a column named "uploaded_url" which contains paths to successfully uploaded data or null to indicate failure. This would allow API consistency with the download() method, which allows users to optionally raise errors in an eager manner, or ignore them and report a null value from the expression.
Describe alternatives you've considered
I currently maintain a custom StatefulUDF which does the same thing, however, the small API change would remove the need for this solution.
Is your feature request related to a problem?
Currently, users can only upload to a single prefix (local directory, or S3).
The feature request is to extent the upload functionality to work on a column expression.
Describe the solution you'd like
I would like to be able to run this example
This is expected to produce a column named "uploaded_url" which contains paths to successfully uploaded data or
null
to indicate failure. This would allow API consistency with thedownload()
method, which allows users to optionallyraise
errors in an eager manner, or ignore them and report anull
value from the expression.Describe alternatives you've considered
I currently maintain a custom
StatefulUDF
which does the same thing, however, the small API change would remove the need for this solution.Additional Context
Slack Thread: https://dist-data.slack.com/archives/C041NA2RBFD/p1731900336643709
Would you like to implement a fix?
No