take_while doesn't work with slice of custom token

[adamchalmers]

Please complete the following tasks

• [X] I have searched the discussions
• [X] I have searched the open and rejected issues

rust version

1.73.0

winnow version

0.5.16

Minimal reproducible code

use winnow::{prelude::*, token::take_while};

/// Parse any number of lowercase ASCII tokens.
pub fn lowercase_tokens<'slice, 'input>(i: TokenSlice<'slice, 'input>) -> PResult<&'input [Token]> {
    take_while(0.., |token: Token| token.0 >= 'a' && token.0 <= 'z').parse_next(i)
}

/// The stream is a vec of custom tokens.
type TokenSlice<'slice, 'input> = &'slice mut &'input [Token];

fn main() {
    // This parser should fail, because it's trying to parse two tokens and there's only one available.
    let input_stream = [Token('a'), Token('b'), Token('Q')];
    let _result = lowercase_tokens.parse_next(&mut input_stream.as_slice());
}

#[derive(Debug, PartialEq, Eq, Clone)]
pub struct Token(char);

Steps to reproduce the bug with the above code

cargo check

Actual Behaviour

error[E0277]: the trait bound `Token: AsChar` is not satisfied
   --> src/main.rs:5:21
    |
5   |     take_while(0.., |token: Token| token.0 >= 'a' && token.0 <= 'z').parse_next(i)
    |     ----------      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `AsChar` is not implemented for `Token`
    |     |
    |     required by a bound introduced by this call
    |
    = help: the following other types implement trait `AsChar`:
              char
              u8
              &'a u8
              &'a char
    = note: required for `[closure@src/main.rs:5:21: 5:35]` to implement `ContainsToken<Token>`
note: required by a bound in `winnow::token::take_while`
   --> /Users/adamchalmers/.cargo/registry/src/index.crates.io-6f17d22bba15001f/winnow-0.5.16/src/token/mod.rs:490:8
    |
483 | pub fn take_while<T, I, Error: ParserError<I>>(
    |        ---------- required by a bound in this function
...
490 |     T: ContainsToken<<I as Stream>::Token>,
    |        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ required by this bound in `take_while`

For more information about this error, try `rustc --explain E0277`.

Expected Behaviour

It should compile and allow me to take the first two tokens, while rejecting the third.

Additional Context

I think this is because trait ContainsTokens is implemented for &[char] and &[u8] but not for other custom token types that meet the right trait bounds. AFAICT ContainsToken<C> is implemented only with C: AsChar.

Discussed at https://github.com/KittyCAD/modeling-app/pull/731#discussion_r1356957357

[epage]

While its not baked-in, you can support that by implementing ContainsToken for your type.

For an example of this, see #348.

Now, there is the question of whether we should support some more blanket impls for ContainsToken. The main loss is that you can't intermix bytes/chars as tokens. In particular, I have a parser that is generic over both.

[epage]

@adamchalmers could you confirm that this is resolved?

[adamchalmers]

LGTM!