search

dagster-io / dagster

An orchestration platform for the development, production, and observation of data assets.
https://dagster.io
Apache License 2.0
11.17k stars 1.4k forks source link

Support for lazy loading in Definitions API #12476

Open kaiserjpg opened 1 year ago

kaiserjpg commented 1 year ago

What's the use case?

Similar to the Repositories API capability of lazy loading definitions. Same use case: This form is intended to allow definitions to be created lazily when accessed by name, which can be helpful for performance when there are many definitions in a repository, or when constructing the definitions is costly.

For example in our case we have a complex repository that lazily constructs jobs, sensors from an API

Ideas of implementation

No response

Additional information

No response

Message from the maintainers

Impacted by this issue? Give it a 👍! We factor engagement into prioritization.

prismec commented 7 months ago

Is there any update on this issue ? We are also eagerly waiting for this feature!