Release on PyPI

[stevennic]

Since the bitbucket repo is abandoned, we should put future releases of whoosh from here to the PyPI repository.

[benzkji]

under what name would the package be published? as I see, it could be made available under whoosh again? see https://www.python.org/dev/peps/pep-0541/#continued-maintenance-of-an-abandoned-project

[nijel]

That process is non existing so far, so it is not immediate solution. See list of issues: https://github.com/pypa/warehouse/labels/PEP%20541

[stevennic]

It does look like the process is working, if one looks at closed PEP 541 issues.

We could declare the official release as abandoned. PEP 541 requires the following to be true:

1. Owner not reachable (see Reachability)
2. No releases within the past twelve months
3. No activity from the owner on the project's home page (or no home page listed).

In our case:

1. There have been various calls out to Matt from Bitbucket and this project's issues but he appears to have been unresponsive to Whoosh for about 2 years.
2. Last PyPI release was in April 2016
3. Last commit by Matt on Bitbucket was in July 2017. Last commit by Matt on his Github repo was in 2015. Last edit to the home page Wiki by Matt was in 2013

@mchaput, if you're out there, we thank you for all your work on Whoosh and would like to keep this project alive. We would like your permission to transfer ownership of the PyPI package to Whoosh Community. The community effort has already patched a number of bugs, and we'd like to be able to publish the improvements to PyPI. Without this, projects that depend on Whoosh, such as Django Haystack, will probably not benefit from or have any visibility into our work.

If we're not able to reach Matt, we could declare the project abandoned and have ownership transferred here. Conditions for transfer:

1. The project has been determined abandoned by the rules described above
2. The candidate is able to demonstrate their own failed attempts to contact the existing owner
3. The candidate is able to demonstrate improvements made on the candidate's own fork of the project
4. the candidate is able to demonstrate why a fork under a different name is not an acceptable workaround
5. the maintainers of the Package Index don't have any additional reservations.

I think we have good support for all 5 conditions.

[nijel]

I haven't found any issue which got resolved by the warehouse maintainers - in most closed issues in the end the original maintainer has added new one and AFAIK we were not able to get in touch with @mchaput . The process for this is till not defined, see https://github.com/pypa/warehouse/issues/1506. I know it doesn't work because I'm already there with other package, see https://github.com/pypa/warehouse/issues/4860.

[stevennic]

This looks like a successful resolution to me. In many of the other cases, the moderators managed to contact the owner and mediated for ownership transfer, so it seems to me the process is functioning. I think if the moderators are not able to hear back from @mchaput, they will qualify it as abandoned and do the transfer for us as in 5829 .

[stevennic]

I just tried contacting Matt by email. Let's see.

[Hiyorimi]

I just tried contacting Matt by email. Let's see.

how is it going?

[stevennic]

No response.

[stevennic]

I have formally requested the transfer. Feel free to weigh in.

[andreymal]

Last commit by Matt on Bitbucket was in July 2017

At today the last commit was 4 days ago https://bitbucket.org/mchaput/whoosh/commits/dd3d0e13b527ab79cf14ed199c2c18a79b7f5841

[stevennic]

How strange, he just committed after 2 years! But he's not responding to anyone.

[mchaput]

Hi all, really sorry about the terrible state of Whoosh. I haven't had time in years to work on it, and I stopped paying attention to the email address out of shame/embarrassment. That clearly was a very bad thing to do. If y'all can let me know what changes/fixes/pull requests (on BB) you need included I can make a release.

I've wanted/needed to move the Mercurial/BB repo over to GitHub for a long time, but I don't know the right way to do it, especially since I have this old attempt sitting here. What's the best tool to migrate Mercurial to Git now? Can I export the hg repo "on top of" what's already here? Or can/should I clear this repo and try to do a clean export? Does anyone know?

[stevennic]

Hi Matt! Welcome back and thank you so much for responding!

First of all, shame is the last thing that should be on your mind. You've built something that is so good and needed so badly that people are refusing to let it languish. So thank you and please don't apologize. You're not alone. There's a team behind Whoosh.

@fortable1999 has copied the code base to this repo here and it has since seen important bug fixes and improvements. In my opinion, this is the best place to go forward. I noticed you've made a few commits in BB since. If we can find a way to merge those into this repo, I think we'll have a good start.

If you would prefer to have the project under your Github account, then perhaps we can find a way to migrate everything from whoosh-community to you? Either way, if the workload / commitment for you is too much, I think it would be great if we could have a few others who have the authority to merge pull requests and post releases to PyPI so Whoosh can live on! :)

It is also exciting to have your eyeballs on this again because I know many of the contributors are eager to ask you about how some of the inner workings are designed. There's only so much we can understand just by stepping through a debugger.

[mchaput]

Oops, I didn't even notice the name of this repo, I assumed it was mchaput/whoosh :) I'm happy to help here.

Should I make a release from the current state of this repo? Or maybe I should I add maintainers on PyPI so you guys can do it?

[stevennic]

I would vote for adding maintainers so we can give ourselves time to decide if we'd like to close any other issues before making a release, and generally have more of a robust process in place for releases.

[mchaput]

When you're ready I can make a release and/or add PyPI usernames as maintainers.

[stevennic]

@fortable1999 and @nijel have been merging pull requests so I propose them for a start. I would also be happy to be included but I'm currently just a contributor, so I leave that decision to you and the two collaborators on the project.

@fortable1999 and @nijel, how do you feel about pushing a PyPI release from current master?

[nijel]

I think we should first agree where further development will continue, bring all changes there and then release from it including links to that location in the release. If @mchaput is fine with moving the development here, releasing from current master branch definitely makes sense.

[stevennic]

So @nijel, @mchaput do you have an opinion about where official development should be stored?

[nijel]

I don't think I'm the one to decide this ;-). @fortable1999 did most of the work here on GitHub and @mchaput is still the main author.

My preference is definitely Git. Bitbucket is removing Mercurial next year, so that move has to be done anyway. So I'd say let's stay here on GitHub as the migration was already done here.

[stevennic]

I agree it should be on Github, and I suppose there is a choice between Matt's personal account or whoosh-community. @mchaput ?

[mchaput]

I think this is the right place. My account is quite an old attempt at an import of the BitBucket repo. What I'd like to do is make sure this repo is up to date with the BitBucket hg repo (basically the next_gen branch that I've made a few commits to), and possibly import bugs/pulls from my personal repo and the BitBucket repo, then mark them as deprecated and point them to this one.

[stevennic]

Sounds good! Do you have all the information you need to move the repo and add the maintainers at this point?

[mchaput]

I need the PyPI usernames of the maintainers. Also if anyone happens to know about migrating data from other Github accounts and/or Bitbucket/hg, I could use some pointers :)

[stevennic]

It looks like everyone in this thread is using their Github username on PyPI.

I don't have any specific experience on migration, sorry. :(

[mchaput]

PyPI maintainers added.

[stevennic]

Thanks Matt!

[stevennic]

With Matt's help this issue is resolved! We can start helping him with migrating the rest of the code here and plan for a much needed release of the next version on PyPI.

[rsirres]

Do you plan to make a release on PyPi this year ? The last release is nearly 4 years old. :-|

[stevennic]

Do you plan to make a release on PyPi this year ? The lastest release is nearly 4 years old. :-|

Yes, that's why we started this thread. Fortunately Matt has kindly reconnected with us and we've received access to the PyPI repository, so the main job is done. It's just a matter of tagging the code appropriately and pushing a new release. There are plenty of bug fixes and improvements since the last release that desperately need to get published. I think we should be able to start this process soon, unless @mchaput wanted to merge changes from his own repo into the master branch here first.

[rsirres]

Looks like @mchaput is continuing to commit on the nexgen branch on the original bitbucket repo to release whoosh 3.0: https://bitbucket.org/mchaput/whoosh/commits/

From what I can see there are many differences between whoosh 2 and 3:

• Moving to a key value store backend (default blueline used by X1 codec)
• Dropping Python 2 support
• Adding type annotations

Maybe it makes more sense to apply the improvement of whoosh-community@github once the work on whoosh 3 has been completed on the bitbucket repo !?

[mchaput]

Actually, I’m hoping to merge my 3.0 work into the community github before release, but my first attempt didn’t go very well. So I’m still committing to the BB repo, which I really should be deprecating (I really need to learn git!).

I made an effort over Christmas holidays to relearn the 3.0 codebase after not working on it for a long time. Version 3.0 is much faster, especially at indexing. But the fact that it requires a re-index, and as of now is Python 3 only, will probably limit its appeal/usefulness at first. I even wonder if I should give it a new name and treat it as a successor project instead of a new version. (At this point it’s a pretty major rewrite.)

(As an aside, at work we are going to have keep Py2.7 going for at least another year, but I want the benefits of Whoosh 3.0, so I’ve been looking into ways to “un-forward port” the Py3 code back to Py2.7. However, 3to2 turned out to be as useful as I was hoping.)

Before I do a release I’d like to (1) move the code over to the community github as mentioned, and (2) use the code in production at work for a while to find bugs. Oh, and I just remembered I’ll have to (3) rewrite a bunch of docs! :/

Meanwhile, I haven’t been following the old OR new issue trackers, but if there are big bugs in the current code that you need my help to fix, I can try to help :)

Matt

On Jan 22, 2020, at 10:56 AM, Sirres notifications@github.com wrote:

Looks like @mchaput is continuing to commit on the nexgen branch on the original bitbucket repo to release whoosh 3.0: https://bitbucket.org/mchaput/whoosh/commits/

From what I can see there are many differences between the whoosh 2 and 3:

• Moving to a key value store backend (default blueline) used by X1 codec) • Dropping Python 2 support • Adding type annotations Maybe a merge of whoosh-community@github and whoosh@bitbucket makes sense once whoosh 3 has been released ?!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[stevennic]

Hi @mchaput, just checking in. How's your code migration going? Shall we make a plan for a release?