Make workspace member inclusion/exclusion more useful, consistent, and intuitive

Problem

As of now, Cargo supports glob syntax for workspace.member to include crates as workspace members. However, for workspace.exclude it is implemented by a plain starts_with comparing with manifest path of certain crates. This confuses people as they need to figure out which rules overrides the other. It is also inconsistent with package.include and package.exclude, as they choose gitignore syntax.

For workspace.member, it proactively checks the existence of Cargo.toml to include a member, and throws an errors if Cargo.toml doesn't exist, which makes globbing **/* often invalid.

Similar issues have been raised from time to time:

[ ] #4593
[ ] #6745
[ ] #6009
[ ] #11362

Proposed Solution

There are several possible ways to "fix" the situation (to some extent).

The first is implementing glob support for workspace.exclude. #11374 tries to do that. This is a relatively low effort approach but might confuse people still, and it won't fix #6745.
The other way is using gitignore syntax, as ehuss mentioned earlier https://github.com/rust-lang/cargo/issues/6009#issuecomment-519576849. This would be a breaking change and might need a transition period and provide migration steps.
In https://github.com/rust-lang/cargo/issues/11362#issuecomment-1311811774, we talked about using a single workspace.member and drop workspace.exclude. However, the re-include is not possible due to the limit of gitignore itself[^1].
We might want to avoid starts_with and instead check the existence of Cargo.toml to exclude a package.

There might be other possible solution I am not aware of here.

[^1]: An optional prefix "!" which negates the pattern; any matching file excluded by a previous pattern will become included again. It is not possible to re-include a file if a parent directory of that file is excluded. See https://git-scm.com/docs/gitignore#_pattern_format

Notes

I would keep this as the main issue and close some. If this is inappropriate, please say it out load, and we can reopen the original issues.

rust-lang / cargo