search

dagster-io / dagster

An orchestration platform for the development, production, and observation of data assets.
https://dagster.io
Apache License 2.0
11.71k stars 1.48k forks source link

Configure the `io_manager_key` for assets in bulk #18218

Open benpankow opened 12 months ago

benpankow commented 12 months ago

What's the use case?

Right now, io_manager_key must be set at definition-time on assets. It would be useful to be able to group and set the io_manager_key at the job-level or load_assets_from_modules etc, similar to the ability to bulk-set freshness policies, auto-materialization policies etc.

Ideas of implementation

No response

Additional information

No response

Message from the maintainers

Impacted by this issue? Give it a 👍! We factor engagement into prioritization.

selene-eiq commented 9 months ago

This would be a nice quality of life thing for us, as well. It would work really cleanly with load assets from modules as usually the assets for one IOManager vs another are cleanly divided by the location they are stored in - and if not, easily moveable to make that true.