nvim-neorg / rust-norg

A robust parser for Norg for tools that don't use tree-sitter.
12 stars 4 forks source link

feat: parse_tree #13

Closed benlubas closed 3 weeks ago

benlubas commented 2 months ago
previous post I'm stumped here. This change causes ``` @a <*a> @end ``` (without the trailing newline) to fail to parse with error: ``` thread 'main' panicked at src/stage_3.rs:221:84: called `Result::unwrap()` on an `Err` value: [Simple { span: 2..2, reason: Unexpected, expected: {Some(Special('<')), Some(Special('{')), Some(Special('['))}, found: None, label: None }] ``` With the trailing newline, it parses correctly. I feel like I've not changed anything that should effect this test case, but apparently I have. btw, this fails on the `parse` function, not just the new `parse_tree` function.

This PR adds stage 4 and the parse_tree method, which creates a non-flat AST.

Additionally, it adds support for delimiting modifiers

awegsche commented 1 month ago

Hi, as discussed on discord, there are still failing tests because of verbatim_ranged_tags not being able to handle the case where everything is in one line. The fix would be something along the lines of changing the let verbatim_ranged_tag = ... part in stage_2 to:

    let verbatim_ranged_tag = |c: char| {
        let parse_char = select! { Special(x) if x == c => x };
        let tag_end = select! {
                End(x) if x == c => x,
        };

        // tag_parameters also ends on encountering End(x)
        let tag_parameters = select! {
        Newlines(_) => (),
        SingleNewline => (),
        Whitespace(_) => (),
        Eof => (),
        End(x) if x ==c => ()
        }
        .not()
        .repeated()
        .at_least(1)
        .separated_by(whitespace.repeated().at_least(1));

        parse_char
            .ignore_then(newlines_whitespace_or_eof.not().repeated().at_least(1))
            .then(
                whitespace
                    .repeated()
                    .at_least(1)
                    .ignore_then(tag_parameters)
                    .or_not(),
            )
            // need to handle the `or_not()` cases
            .then_ignore(just(SingleNewline).or_not())
            .then_ignore(filter(|c| if let Newlines(_) = c { true } else { false }).or_not())
            .then(tag_end.not().repeated().or_not())
            .then_ignore(tag_end)
            .map(
NTBBloodbath commented 1 month ago

As discussed on Discord, there is a bug where carryover tags only search for headers currently. This might be tackled in another PR.

cc @benlubas

benlubas commented 1 month ago

@NTBBloodbath this has been addressed now