search

dagster-io / dagster

An orchestration platform for the development, production, and observation of data assets.
https://dagster.io
Apache License 2.0
10.8k stars 1.35k forks source link

investigate libraries for making runtime type assertions based on typehints #3486

Open mgasner opened 3 years ago

mgasner commented 3 years ago

https://pypi.org/project/enforce/ https://pypi.org/project/pydantic/ https://pypi.org/project/pytypes/

gjeusel commented 3 years ago

Hello there, any ETA on this one ?

I would love to see the power of Dagster dynamic type checking unleashed by pydantic rich toolset !

sryza commented 3 years ago

Hey @gjeusel - Dagster currently already actually does runtime type assertions on solid inputs and outputs based on their type hints. Is that the functionality that you're interested in?

I believe that the intent of this Github issue, when filed, was to do automatic runtime type checking for arguments to all Dagster APIs. We currently implement enforce types all over our codebase with manual type assertions. I..e it wouldn't affect the experience of using of Dagster's APIs, but would make the Dagster internals cleaner.