Closed AA-Turner closed 7 months ago
Thanks for the kind words, @AA-Turner!
I'm definitely interested in discussing moving the project under sphinx-doc, with a final decision depending quite a bit on the exact shape of things.
Completely agree about not duplicating efforts. But, sphobjinv is more than just lookup/search into objects.inv files, and I'm not sure if all of those other pieces of the project would be most naturally housed in sphinx-doc. So, e.g., it might make more sense to slice out some of the pieces of functionality you're wanting to bring into core Sphinx, and then sphobjinv would migrate over to using those sphinx-core pieces over time. Depends a lot on what you have in mind.
Other questions that come to mind include aspects of ownership/governance, and technical direction. For example, one of my big goal items is a plugin system to let users write and use custom suggest
scorers (see https://github.com/bskinn/sphobjinv/issues/207). If suggest
were to become a core Sphinx feature, then those scorers could be wired in using Sphinx's existing extensions system, which (at the moment, at least) seems very attractive.
I'm guessing this overture is coming after some internal discussion on the part of the Sphinx team -- if so, is that also something I could learn more about before making a decision?
Happy to find a time for a call if that would be more efficient. Else, async in the thread here is good too.
Thanks again!
Completely agree about not duplicating efforts. But, sphobjinv is more than just lookup/search into objects.inv files, and I'm not sure if all of those other pieces of the project would be most naturally housed in sphinx-doc. So, e.g., it might make more sense to slice out some of the pieces of functionality you're wanting to bring into core Sphinx, and then sphobjinv would migrate over to using those sphinx-core pieces over time. Depends a lot on what you have in mind.
We have several changes planned for the inventory format in general (see https://github.com/sphinx-doc/sphinx/pull/8929, https://github.com/sphinx-doc/sphinx/pull/10128, https://github.com/sphinx-doc/sphinx/pull/10127), so part of this is trying to future-proof a v3 serialised inventory format and the v2 internal representation, etc.
As part of the new sphinx
command work, a possibility (especially if sphobjinv
is more closely integrated) is a new sphinx inventory
or sphinx inv
sub-command, which would likely as a minimum contain the suggest
functionality, but may also expose the inventory editing features sphobjinv
currently has.
Other questions that come to mind include aspects of ownership/governance, and technical direction. For example, one of my big goal items is a plugin system to let users write and use custom
suggest
scorers (see #207). Ifsuggest
were to become a core Sphinx feature, then those scorers could be wired in using Sphinx's existing extensions system, which (at the moment, at least) seems very attractive.
I would envision (at least to start) very little changing from the sphobjinv
user's perspective -- the changes would be that the Sphinx team has commit access to the repo, and changes in the inventory format can be reflected when made upstream.
Another option is full integration into the core sphinx-doc/sphinx
repo, which I would see as less likely (if nothing else as Sphinx is BSD-2-Clause and sphobjinv
is MIT, and it would make licencing complicated!).
I'm guessing this overture is coming after some internal discussion on the part of the Sphinx team -- if so, is that also something I could learn more about before making a decision?
Happy to find a time for a call if that would be more efficient. Else, async in the thread here is good too.
I've got a very busy few weeks (& as such I'm taking a break from GH), but could speak at the end of April / start of May, if useful?
Thanks, Adam
I've got a very busy few weeks (& as such I'm taking a break from GH), but could speak at the end of April / start of May, if useful?
No problem to keep this async for a while. I don't specifically see a need for a chat in the short term.
On the topic of time, can I ask—what's the timeline on all of these improvements? sphobjinv
dates from my earlier days as a Python developer, and as such there's quite a bit in there that could really use cleanup. Would that cleanup need to happen before a move under the sphinx-doc
umbrella? Some of the kludgy stuff might interfere with integration—e.g., sphobjinv
desperately needs its own custom exception hierarchy (#118), and the CLI code is not pretty. Also, I've got a major docs rework on the radar (#270). I don't know how quickly I would be able to execute any cleanup plans, whether my own or developed in tandem with the Sphinx team.
As part of the new
sphinx
command work, a possibility (especially ifsphobjinv
is more closely integrated) is a newsphinx inventory
orsphinx inv
sub-command, which would likely as a minimum contain thesuggest
functionality, but may also expose the inventory editing featuressphobjinv
currently has....
I would envision (at least to start) very little changing from the
sphobjinv
user's perspective -- the changes would be that the Sphinx team has commit access to the repo, and changes in the inventory format can be reflected when made upstream.Another option is full integration into the core
sphinx-doc/sphinx
repo, which I would see as less likely (if nothing else as Sphinx is BSD-2-Clause andsphobjinv
is MIT, and it would make licencing complicated!).
If direct integration of sphobjinv
's Inventory
, suggest
, convert
, etc. functionality into Sphinx core—which truly would avoid the need to make coupled edits in multiple places—isn't the primary intent, due to the license collision... what are the functional advantages of pulling sphobjinv
under the sphinx-doc
umbrella? Without the sphobjinv
code living directly in the Sphinx codebase, it's not clear to me how Sphinx can leverage sphobjinv
's machinery other than by following a typical import-and-use pattern. And for that, the project relocation doesn't seem necessary to me.
Along these same lines, if the main thing is providing Sphinx maintainers with commit bits, wouldn't granting them commit bits on bskinn/sphobjinv
serve the same purpose?
Following the train of thought further... logistically, wouldn't I need to be involved with most or all of the changes to sphobjinv
machinery, at least at first, to help make sure that those will do the right things, and integrate correctly with the surrounding codebase? It seems to me that starting out with a standard open source PR flow for changes to sphobjinv
would be a more cautious, lower-impact path, versus granting direct commit rights. (I've had negative effects on the state of the codebase result from an otherwise positive interaction, after granting a commit bit to a collaborator on a different project, so I'm hesitant to open that door too quickly.)
I'm not that familiar with how GitHub organizations work, so I may well be ignorant of some obvious benefits. If nothing else, I could see some potential advantages in terms of logistical tooling—gaining access to sphinx-doc
's user roles, teams, etc. Perhaps there's more there than what I know of—I'm happy to be educated.
Ultimately, I think it boils down to wanting to learn more about what to expect from this experience and about what I would be opening the codebase and the project to, and wanting to get to know the team I'd be de facto joining a bit better, before making a decision.
A quick reply to some key points:
What's the timeline on all of these improvements? ... Would that cleanup need to happen before a move under the
sphinx-doc
umbrella?
Timeline: perhaps Summer 2023? The tentative release timeline is Sphinx 6.2 around early May, and Sphinx 7.0 shortly afterwards (only removing deprecations).
I'd be happy to help with any clean-up, as you feel is appropriate.
If direct integration of
sphobjinv
'sInventory
,suggest
,convert
, etc. functionality into Sphinx core—which truly would avoid the need to make coupled edits in multiple places—isn't the primary intent, due to the license collision... what are the functional advantages of pullingsphobjinv
under thesphinx-doc
umbrella? Without thesphobjinv
code living directly in the Sphinx codebase, it's not clear to me how Sphinx can leveragesphobjinv
's machinery other than by following a typical import-and-use pattern. And for that, the project relocation doesn't seem necessary to me.Along these same lines, if the main thing is providing Sphinx maintainers with commit bits, wouldn't granting them commit bits on
bskinn/sphobjinv
serve the same purpose?Following the train of thought further... logistically, wouldn't I need to be involved with most or all of the changes to
sphobjinv
machinery, at least at first, to help make sure that those will do the right things, and integrate correctly with the surrounding codebase? It seems to me that starting out with a standard open source PR flow for changes tosphobjinv
would be a more cautious, lower-impact path, versus granting direct commit rights. (I've had negative effects on the state of the codebase result from an otherwise positive interaction, after granting a commit bit to a collaborator on a different project, so I'm hesitant to open that door too quickly.)
A partial reply to all of these points:
I admit I haven't thoroughly reviewed the code, and didn't realise that true integration would be the 'best' way of de-duplicating efforts. If so (and conversely that a simple move would offer no real benefits), we could try and focus on that aspect.
Following from (1), There are four commits not by you in this repo[^1] (b2d2e18a3e5ee91aa1cf091fb5be74732deb3744, dba67e390f8f229916c110db81ab4d092b0288b1, 359a645eaea71ae7bb03d9ca623ea0811c9b36c4, and e3b7938542c79531f35b62889066e1fbf1ab4bdf). The first two are trivial enough so as not to be copyrightable, the latter two are more questionable -- therefore (if you want to) you might be able to re-licence the repo to BSD-2-Clause either unilaterally or with Oscar's agreement -- should we want to integrate properly.
A possible halfway-house position (as you alluded to in your original reply) would be to integrate the 'suggest' functionality into Sphinx, and explore providing better APIs for the 'convert' subcommand.
The benefits I see of the organisation are that should any one person be suddenly indisposed or unavailable, others can continue maintainership -- I entirely agree though that this can be achieved by granting users write/collaborator permissions on this repository.
The downside of integration would be that improvements become coupled to versions of Sphinx, and if one is using an older version then the benefits aren't seen.
Thanks, Adam
[^1]: git log --perl-regexp --author='^((?!Brian Skinn).*)$'
Sorry for the long delay on this, @AA-Turner—whole bunch of stuff, including PyCon US, hung me up.
In any event, while mulling this over, I thought of something that makes integration of sphobjinv
into Sphinx core a non-starter.
I've built sphobjinv
specifically to have a light install footprint, since it's working with a static artifact published by Sphinx, and not actual Sphinx source or live Sphinx data structures. It doesn't need Sphinx installed to function, and I know of multiple uses of sphobjinv
without Sphinx present. One example is inventory-search chatbots for Discord and such.
I don't want to have to install Sphinx everywhere I want to use sphobjinv
, and I'm confident that these users won't want to either.
On top of this, I agree—the coupling of Sphinx and sphobjinv
release cadences would almost certainly be undesirable.
Given these blockers to integration, I don't see any particular benefits to moving sphobjinv
under sphinx-doc
at this time. I truly appreciate the offer and what it signifies in terms of the perceived value of the project. But, for now, I think the best course forward is for Sphinx core devs to collaborate with sphobjinv
by submitting PRs. While my independent development pace on sphobjinv
is bursty but overall slow, I'll be able and motivated to work on any time-critical external contributions in prompt fashion.
That said, I wonder if vendoring might be a viable ~middle path for integration. I don't know what Sphinx's position is on vendoring dependencies, but it seems like it might work ok.
It would lock a given version of Sphinx to a given version of sphobjinv
... but that actually might provide more benefits than downsides. It would freeze the sphobjinv
API exposed internally to Sphinx while not pinning the sphobjinv
version installed in a containing environment. And, if I understand correctly how Python imports work, a vendored sphobjinv
inside Sphinx would not import-collide with a sphobjinv
in the environment, because they would be imported from different locations.
I'd be willing to refactor the intra-project imports to be relative in future releases, to simplify vendoring.
I'd also be willing to go to the work of a license change, if it were needed; but I'd think it would suffice to include the MIT LICENSE.txt
in the directory tree with the vendored code?
I recognize either of these approaches leaves some questions/challenges in place. One you've mentioned; the other may not be a big deal but bears considering:
sphobjinv
I'm willing to work toward bringing one or more Sphinx maintainers on as sphobjinv
maintainer(s), but I would insist on it following a ~typical arc: a series of successful PRs leading to a good working rapport, and then a commit bit.
While sphobjinv
doesn't use Sphinx directly in its functionality, its test suite does import Sphinx. If Sphinx takes on sphobjinv
as a dependency, then the risk emerges for circular imports in sphobjinv
's test suite. I'm not sure what the likelihood of this problem emerging is, or how challenging fixes would be, but it's something to keep in mind.
One approach to avoid this possibility would be to have Sphinx only declare sphobjinv
as part of an extras_require
dependency set. sphobjinv
could then install Sphinx without this extra in testing environments.
Hi Brian,
Would you consider moving this project to @sphinx-doc? I'd like to integrate support for intersphinx lookup into core Sphinx, and it makes sense not to duplicate efforts.
Regardless of your thoughts, thank you for the great project!
Thanks, Adam