support multiple IO Managers to an asset

[dagsir[bot]]

Issue from the Dagster Slack

This issue was generated from the slack conversation at: https://dagster.slack.com/archives/C01U954MEER/p1662557614690959?thread_ts=1662557614.690959&cid=C01U954MEER

---

Conversation excerpt

U034Q7NCNHY: Is it possible to assign multiple IO Managers to an asset? I would like to materialize an asset to multiple backing storages. Thank you in advance! UULA0R2LV: cc <@U011CET83FG> UULA0R2LV: One workaround is to have a “master” IOManager which materializes an asset to multiple storages in its handle_output. You can make it depend on other io managers using <https://docs.dagster.io/concepts/resources#resource-to-resource-dependencies|resource-to-resource dependencies>, such as:

@io_manager(required_resource_keys={"other_io_manager_1", "other_io_manager_2"})
def master_io_manager(init_context):
    other_io_manager_1 = init_context.resources.other_io_manager_1
    other_io_manager_2 = init_context.resources.other_io_manager_2
    return MasterIOManager(other_io_manager_1, other_io_manager_2)

class MasterIOManager(IOManager):

    def handle_output(self, context, obj):
        context.resources.other_io_manager_1.handle_output(context, obj)
        context.resources.other_io_manager_2.handle_output(context, obj)

EDIT: we currently don’t have native support for multiple IO Managers to an asset. U034Q7NCNHY: Thank you for the workaround - this can work for now! :thankewe:

Is there any plan to have native support for multiple IO Managers to an asset? UULA0R2LV: <@U018K0G2Y85> issue support multiple IO Managers to an asset

---

Message from the maintainers:

Do you care about this too? Give it a :thumbsup:. We factor engagement into prioritization.

[datasalaryman]

Hi, is this issue still being worked on? I'd love to be able to use it

[jamiedemaria]

Hi there. We aren't actively working on this right now and given our current priorities, I don't think we'll take it on soon. The suggestion in the main post of having a single I/O manager that writes to multiple storage platforms should work though, if that's something you're willing to write yourself!