dagster-io / dagster

An orchestration platform for the development, production, and observation of data assets.
https://dagster.io
Apache License 2.0
10.85k stars 1.35k forks source link

Be able to access resources by string index #4471

Open dpeng817 opened 2 years ago

dpeng817 commented 2 years ago

Use Case

If constructing solids programmatically in a way that is parameterized by resource key, then it's more convenient to access resources by string.


def create_a_solid(my_resource_key: str, my_resource_def: ResourceDefinition) -> SolidDefinition:
    @solid(required_resource_keys={my_resource_key})
    def _inner_solid(context):
        return context.resources[my_resource_key] # This is cleaner
        return context.resources.__getattribute__(my_resource_key) # Not very elegant, also not documented anywhere.

    return _inner_solid

Ideas of Implementation

This implementation could be added relatively straightforwardly to our custom resources object.


Message from the maintainers:

Excited about this feature? Give it a :thumbsup:. We factor engagement into prioritization.

ahoneycutt commented 2 years ago

I currently employ my_resource = getattr(context.resources, my_resource_key) and think this proposed implementation would clean things up a bit.

yuhan commented 2 years ago

I believe we use getattr throughout the system but I do agreed the proposed api is cleaner -- marking this as backlog for now