Two versions of the same dist can be indexed at the same time, causing deadlock

[wolfsage]

If you upload Foo@1.1 and Foo@1.2 at the same time, mldistwatch will be started for each of them. They can then deadlock when trying to update the primeur table.

We should prevent (if possible) indexing of the same dist in parallel to avoid this problem.

Alternatively, we can say "If you upload 1.1 and 1.2, we just won't index 1.1 since 1.2 will replace it". (The risk here is if 1.1 has modules in it 1.2 doesn't, but does that matter?)

[ap]

does that matter?

Not practically, I think.

But the indexer has two cases it must handle sensibly:

1. the normal case where Foo@1.1 is uploaded first and then Foo@1.2 is uploaded later
2. the rare reverse case where Foo@1.2 is uploaded first and then Foo@1.1 is uploaded later

I think the end result is identical in both cases, in which case uploading both at the same time very definitely should not lead to a different result.

But if the end results of these cases are different from each other, then uploading both at the same time should yield the same result as one or the other, not an entirely separate third type. (Even at random, if that makes this case easier to handle – still better than a separate type of result.)

[neilb]

There are really three cases:

1. The "usual" case where Foo 1.2 is uploaded after 1.1 has already been indexed
2. The "race condition" case, where Foo 1.1 and 1.2 are uploaded so close together that PAUSE might try to index them in parallel. This happened again this morning, and there were 11 seconds between the first and second upload.
3. The "decreasing" version case, where 1.2 is uploaded first, indexed, and then 1.1 is uploaded. This is not as rare as you might think. This most often happens when someone switches between classic version numbers and semver version numbers. In this case PAUSE tells them it wasn't indexed because the version number went down. Remember this is version numbers on packages, not the dist version number. If you don't keep your package version numbers inline with the dist version number, then your mileage may vary (if the package version number doesn't change, then the lowered numbered release, uploaded later, will end up indexed).

[ap]

Thanks. I can see the decreasing version case making sense in its own terms but it seems to me that it should not happen unless the PAUSE author explicitly causes it – which means that if at all possible the race condition case should have the same behavior as the usual case.

Is it possible to de-dupe the mldistwatch invocations, such that Foo@1.2 just won’t be picked up for indexing immediately?