davesnx / styled-ppx

Type-safe styled components for ReScript, Melange and native with type-safe CSS
https://styled-ppx.vercel.app
BSD 2-Clause "Simplified" License
399 stars 31 forks source link

Unify CSS parser and lexer #432

Open davesnx opened 4 months ago

davesnx commented 4 months ago

css_parser.mly parses stylesheets/selectors/declarations/properties and values

Parser.re currently parses (and type-checks) properties + values (there's code in Parser.re that might make it possible to parse stylesheets/selectors and everything but isn't used)

Parser.re uses value.rec ppx whichs generates the parser fns for each property, while css_parser.mly uses menhir. I'm more inclined on having a hand-made parser or push for menhir Incremental API in order to call the value.rec's parsers, but it's up to see if it's a good idea.

The driver calls one and the other depending at which stage of the parsing happens, in order to have success with https://github.com/davesnx/styled-ppx/issues/429 it's necessary to have more control over the parsing phase.

Resources

Mix Lexer and Parser like https://github.com/mnxn/eon and take a look at Sedlexing from here: https://github.com/FStarLang/FStar/pull/2203/files

davesnx commented 4 months ago

Currently we use 2 lexers that live inside Css_lexer, and run separate test suites: Tokenizer_test and css_lexer_test.

We need to unify those, expose a single API to tokenize a string of CSS and treat errors as an exception (Css_lexer.Error) (since menhir needs it), run a single test suite.

davesnx commented 1 week ago

After the unification of lexers, we have only one. We still have 2 separate parsers with 2 separate techniques (menhir and rules with combinators and the ppx).

Would be nice to join them into the same package. There's a few benefits of having them togeter:

davesnx commented 3 days ago

There's a bit of work left to unify the lexers which is the API, we have from_string and tokenize where one gets a recursive token structure, and the other a list of tokens. Both contain locations, it's a matter to only use one of them