search

apache / iceberg-python

Apache PyIceberg
https://py.iceberg.apache.org/
Apache License 2.0
381 stars 140 forks source link

Library public api isolation and import decoupling #499

Closed ndrluis closed 3 days ago

ndrluis commented 6 months ago

Feature Request / Improvement

Hello, when I was trying to solve the #497 issue, I noticed that we are exposing the private API and we have some imports through modules.

For example, in glue.py, we import Identifier and Properties from catalog instead of importing from typedef. Another case is in test_writes.py, where we import Table from catalog instead of from table.

HonahX commented 6 months ago

@ndrluis Thanks for reporting this! I think we should fix these to import from the right module. Do you want to work on this?

ndrluis commented 6 months ago

@HonahX Yes!

ndrluis commented 6 months ago

The #505 decouple imports from

kevinjqliu commented 6 months ago

Thanks for the PR! Do you know if there are ways to do this systemically, with linters / rules? I foresee this to be an issue again as we continue to do more development.

ndrluis commented 6 months ago

@kevinjqliu Agreed, I haven't found anything that addresses this issue or provides a warning when it occurs. It seems like something the community has accepted, but I believe it's a problem. I will open an issue on Ruff to ask about this and understand how we can write a rule.

ndrluis commented 6 months ago

Issue to follow up https://github.com/astral-sh/ruff/issues/10300

ndrluis commented 6 months ago

We have some news about the issue: mypy has implemented a rule that could protect us. You can find more details here: https://mypy.readthedocs.io/en/stable/config_file.html#confval-implicit_reexport. Furthermore, there is an ongoing discussion about implementing this rule in ruff.

github-actions[bot] commented 4 days ago

This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.

ndrluis commented 3 days ago

I'll close this issue because we added the mypy linter, which solves our problem with coupling, and we have #1099 to solve the public API definition.