Smooth the breakage of other crates in the rust-lang org

[reem]

Since these crates were moved out of the regular distribution, they have been subject to the same breakage as other crates in the ecosystem. However, unlike most other crates, some crates such as log and regex are depended on by the vast majority of crates in the ecosystem, and if their latest versions don't compile it's an almost equivalent blow to the ecosystem as if the standard library was broken.

We should really find a better system for ensuring these crates continue to build or at least work on having a faster reaction time for merging PRs and updating crates.io versions for these crates, as it can take a while now.

[steveklabnik]

My glib response: this is called "1.0". :wink:

[brson]

While I agree that after 1.0 these crates should never break, I too am concerned about the testing and maintenance policy around them. At the moment I believe @alexcrichton is doing almost all of the review and publishing here, and that's not sustainable.

Before 1.0 I intend to set up more regression testing for cargo crates, but that won't necessarily help here since these are already getting coverage on nightly (on linux), and iiuc the main problem is response time in getting fixes published (though maybe to a lesser extent stopping breakage on windows).

[reem]

Response time has been my primary concern as well. Perhaps the answer is to just divert a few more people with push access to monitor these crates or get members from the community to maintain individual crates?

In the same vein, can someone please merge these PRs and update crates.io for these crates:

• regex: https://github.com/rust-lang/regex/pull/7
• rustc-serialize (common offender): https://github.com/rust-lang/rustc-serialize/pull/11

[steveklabnik]

Yeah, absolutely. Getting a good turn around is helpful.

[gchp]

I'd be willing to help maintain some of these crates if you decide to go down that road, and if it would be helpful. What @reem says above is spot on, when these are broken it pretty much means I can't work on anything. Happy to help with the turnaround on it where I can.

[reem]

I'd also be happy to work on maintaining these crates, like @gchp I haven't been able to work on a multitude of projects today because these crates have been broken. (Again, would really appreciate if someone merged the two PRs I listed above).

[jfager]

Not to detract from the importance of keeping these up-to-date, but don't forget that you can fork the project, apply a waiting PR, and use .cargo/config to temporarily point to your local copy. Here's how I'm working today w/ bindgen and serialize busted:

paths = [
    "/path/to/local/rust-bindgen",
    "/path/to/local/rustc-serialize",
] 

[vhbit]

I'd be willing to help too.

[alexcrichton]

I'd like to emphasize what @jfager mentioned, this is one of the core aspects of overrides in Cargo. A global ~/.cargo/config with paths entries pointing to updated libraries will allow you to seamlessly work on all your own projects while you're waiting for updates to be merged and/or published (I do this quite often).

[vhbit]

@alexcrichton ohh, yesterday I've take a grip on myself to avoid replying to @jfager as it would be too emotional :smile:

Yep, we all know the trick and moreover use it almost everyday. But doesn't scale at all if you have something which depends on a lot of libraries and breaking changes (btw, in the last week I've probably seen more breaking changes than in half a year before) collapse building of foundation libraries.

It looks like this:

1. build project
2. get it failed because of a dependency
3. patch the dependency in a separate repo
4. try to test dependency
5. ohh, well, dependency actually builds but is failing to build tests (hello regex!)
6. what to do now? you can continue working with version which just builds but you know others will be kept stalled or you can patch tests too and create a PR.
7. mess with .cargo/config
8. patch tests
9. issue a PR
10. come back to your project
11. loop to p2

Sometimes this fails exactly because of already patched .cargo/config as it points to outdated version and you still have to navigate, change the branch and so on. Every action might take a small amount of time, but doing all of them multiple times seems absolutely insane and takes the time which otherwise would be spent on either improving libraries or hacking on Rust itself (especially if you want to get something to be available in 1.0).

Yesterday I've opened 7 pull requests for different open-source libraries and patched 3 more internal ones. When I saw that by the time I got to traverse and collect-rs they were already patched by someone else my though was - ohh my god, that's more than wonderful!

So yep - I completely agree with @reem that major libraries like log, regex should be kept up to date even if they are out of tree. If Rust team doesn't have enough time for it - it's perfectly OK, we all understand you have a lot of pressure now. But there are a lot of people out here who wants to help (at least for their own sanity). If the problem is in maintaining control - give just a temporary access until 1.0 hits.

[tomaka]

One of the worst thing that I did recently is introduce a segfault in one of my libraries after I merged several minor PRs. Travis would have detected the segfault, but dependencies were failing to compile and you can't override paths on travis. It then took me one week to figure out where the segfault came from.

I agree that this problem is incredibly annoying. Most Rust users already deal with 4-5 broken libraries each time that there are breaking changes in Rustc, so adding 3 or 4 more to this list is painful.

However since the alpha is in less than a week, I don't think it's worth dealing with this problem as it will solve itself.

[nrc]

I see scope for having priveleged crates (log, regex, at the least) which get tested as part of make check and on our infra. If you break them it';s your responsibility to fix them, just like with crates in the distro. Put another way, we should decouple 'responsibility to fix' from 'in the distro/rust repo'.

[huonw]

One possibility to improve this is making @rust-highfive also assign PRs in the other rust-lang crates to people, so they get someone's attention more quickly.

(I talked with @nick29581 on IRC, and understandably he didn't want to work on this until next week, after the alpha, but someone/anyone feel free to ping me/him with reminder about this then.)

[brson]

Maintainership is not just a problem for other crates, but e.g. rust-by-example as well.

[huonw]

@nick29581 has now activated @rust-highfive on essentially all the rust-lang repos (including the crates.io repos and rust-by-example), so PRs will now be assigned a reviewer immediately and automatically e.g. https://github.com/rust-lang/semver/pull/34. Hopefully we can reduce the pain of breakage this way by merging and publishing things more quickly. (If something is taking particularly long, @alexcrichton and I are both owners of all the rust-lang crates on crates.io so you can try to get our attention, on github or on IRC etc.)

Also, I'll note that multirust gives you maximum Rust. Specifically, you can install and switch between multiple compilers, so, e.g. you can switch between "real" work and upgrading work without having to do any manual version juggling.

Closing; if things are still not working we can reopen and/or investigate better procedures.