emdgroup / foundry-dev-tools

Foundry DevTools
https://emdgroup.github.io/foundry-dev-tools/
Apache License 2.0
115 stars 23 forks source link

Add typing to callables in transform decorators #78

Closed JimLundin closed 1 month ago

JimLundin commented 1 month ago

Summary

Hello!

We are utilising foundry-dev-tools for some local development, primarily for the added type hints when using transforms.

We use the "strict=True" option for mypy type checking during our development. This setting disallows unspecified Callable types as an error when checking types. I added more specific gradual types to the "transformation_polars" and "transformation_pandas" decorators so that mypy is satisfied.

I suspect that the same type hints could be added for the pyspark transforms aswell. I can add that to this PR if that is desirable.

In the process, I also added the "Input" type to the **kwargs in the decorator signature. If required I could split that into a separate PR.

Best regards, Jim

Checklist

CLAassistant commented 1 month ago

CLA assistant check
All committers have signed the CLA.

nicornk commented 1 month ago

Hi! 👋 Thanks for the contribution. Feel free to add additional typing that you think is required (e.g. to the pyspark transform)

JimLundin commented 1 month ago

Added the pyspark typing to transform_df! There are more typing improvements that I think could be made to the decorators. If you would like I can continue adding to this PR or start a new one.

nicornk commented 1 month ago

Feel free to add further typing - let me know when you think it’s ready to be merged. Thanks!

JimLundin commented 1 month ago

PR looks good now! I have focused on fixing the developer-facing type hints for the transform decorators. You are welcome to merge or give feedback on the changes. When I have more time, I could help set up a github workflow for type checking and add more "internal" type hints. If that fits into the philosophy of this library.

nicornk commented 1 month ago

feel free to contribute additional typing PRs where you think it makes sense.