Package(s) vendoring issues

[daskol]

What is supposed way to distribute orbax-checkpoint и orbax-export? It is totally unclear for me

1. how to get notification about new releases?
2. what is 'ground truth' package(s) sources: PyPI or GitHub?
3. what is supposed way to deal with conflicts due to package namespace in Linux distros like Arch?

In my perspective the current situation is quite out of the ordinary and tedious to manage it over time. It would be great to known maintainers opinion on the issue.

[cpgaffney1]

Thanks for the questions, we acknowledge that it's an atypical setup. That said, the separation between orbax-checkpoint and orbax-export is very important to many users. The packages have different dependencies, in particular Tensorflow, which users do not wish to include if they are only using checkpointing.

In the future, Orbax will likely include other subpackages beyond checkpoint and export, which have thematic similarity but a high degree of independence. We expect the separation of subpackages to continue to be useful in this context.

1. I am not personally aware of how to do this automatically, but I would recommend monitoring the PyPI package for whichever subpackage you are interested in.
2. The 'ground truth' is PyPI, although this is just a periodic snapshot of the Github. There is an ability to clone a subpackage like orbax-checkpoint from the Github head, if a newly added feature or bugfix is urgently needed.
3. Can you elaborate on the conflict? The package namespaces do not work on certain platforms?

[daskol]

3. Can you elaborate on the conflict? The package namespaces do not work on certain platforms?

I mean that orbax.__init__ is different in orbax-checkpoint [1] and orbax-export [2]. As I understand the main purpose of these __init__ is to provide compatibility layer for packages that still uses orbax.

The issue is that orbax/___init__.py is out of namespace. If one decides to install both packages orbax-(checkpoint|export) then orbax/__init__.py will be missing. On the other hand, a distro maintainer could manually add missing orbax/__init__.py but it would cause another issue: which __init_.py to choose?

Could you please elaborate a little bit more on decision making to split original package to namespaces? Why not just split package into two different packages, say checkpointax and exportax? I'm pretty curious since it is literally the second time for 10+ years when developers become to use package namespaces.

[ethanluoyc]

I found that the separate init file is causing trouble with editable installs.

Maybe we can make orbax a namespace package?

[daskol]

I found that the separate __init__.py file is causing trouble with editable installs.

This is exactly my point.