search

rust-lang / rust

Empowering everyone to build reliable and efficient software.
https://www.rust-lang.org
Other
98.91k stars 12.78k forks source link

In the lexer, accept number suffixes that start with `e`. #67544

Open richard-uk1 opened 4 years ago

richard-uk1 commented 4 years ago

Currently, the lexer will reject tokens like 1.0etest expecting an exponent after the e. It could alternatively accept this token as LitFloat { value: 1.0, suffix: "etest" }. This would mean that proc macros could use suffixes that start with an e (although not the suffix e itself of course).

richard-uk1 commented 4 years ago

To go even further, if the lexer could not find an exponent after the e, it could return LitFloat { suffix: "e", .. } although personally I think this could produce confusing diagnostics.

I've found further inconsistencies here. Numbers like 2f will also fail the lexer, presumably because it looks for f32 or f64.

I think I would now argue that the lexer should accept all combinations of letters and numbers coming after the first number. This way, any sequence of numbers and letters is a valid token: an Ident if it starts with a letter and a numeric literal if it starts with a number. The parser could ensure the same diagnostics as currently.

Doing this would enable, for example, someone to write a procedural macro that turned 12floz into Weight::FluidOunces(12.0), or accept any html hex color (3fc25a, for example).