Running CI with a rust-toolchain file present

[dwijnand]

Quoting from https://twitter.com/Sunjay03/status/997336553528610816:

If you use a rust-toolchain file in your @rustlang project, make sure you delete it when you run CI. If you don't, you'll always build on a single version of Rust and your tests will keep passing even if they shouldn't b/c you aren't testing what you expect

Summary of the thread at this time:

• Suggestion: CI should fail if the file is present
• Suggestion: CI should auto-delete the file
• Concern: pinned versions in two locations in your VCS (in rust-toolchain, in .travis.yml)
• Suggestion: CI should run cargo +nightly build, overriding the pin from rust-toolchain
• Note: If only building one supported version (e.g internal company code) it's how the version is defined (i.e CI shouldn't always delete it)

/cc @sunjay, @manishearth, @llogiq, @anp

[Manishearth]

CI should fail if the file is present and the user has explicitly provided rust versions.

Pinning nighties is a common use case for this and CI should allow that.

[anp]

@Manishearth I agree, although I think that's probably a better thing to do in an editable template (like this repo) than in the application logic of the CI software. Sometimes you have to do weird things :P.