A data pipeline orchestration library for rapid iterative development with automatic cache invalidation allowing users to focus writing their tasks in pandas, polars, sqlalchemy, ibis, and alike.
The warning that dematerialization may be inhibited by "conditions" such as missing installed packages is great! But for the parquet table cache, there should be a list of input_types for which it is obvious that they cannot be parquet cached.
Maybe the hook could return different responses for "I don't know this type" and for "I know this type, and I know that it is not suited for local table caching".
Other solution options are possible including a hand curated match list of ill-suited types for local table cache.
I can pick this one up - it will help me familiarize myself with the code and since it's just error/warning message related it's something for which I don't really need to worry too much about breaking.
The warning that dematerialization may be inhibited by "conditions" such as missing installed packages is great! But for the parquet table cache, there should be a list of input_types for which it is obvious that they cannot be parquet cached.
Maybe the hook could return different responses for "I don't know this type" and for "I know this type, and I know that it is not suited for local table caching". Other solution options are possible including a hand curated match list of ill-suited types for local table cache.