calmofthestorm
commented
3 years ago IMO not spam at all, thank you for sharing and interesting and completely topical project. The issues on this repo (both open and closed) have a number of edge cases @PoiScript and I have discussed.
I like the idea of a corpus of ambiguous/tricky/edge case/etc that can be used for testing, analogous to the big list of naughty strings. It would be a helpful artifact.
Ideally there would be one totally unambiguous Org grammar that all parsers would respect. Unfortunately at this point, I think that ship has sailed. That does not mean it cannot be done, but that it would be very hard. I wish you well but am not optimistic:-)
The next best thing is clear and precise documentation of edge case behavior. @PoiScript and I have disagreed on how a few edge cases should be interpreted, and that's fine. You could imagine that a parser could come with a test suite which clearly specifies its expected behavior for each of the edge cases in the corpus, thus preventing regressions, unexpected changes, etc, even if different parsers disagree. You could also imagine a linter tool that would warn about ambiguous cases in org files.
I think that it's frustrating to try to create a completely unambiguous parser for a language without an unambiguous formal specification. The approach I am currently taking is a bit different, and while it suffices for my needs, it is less satisfying. I would like to precisely specify a minimal grammar specifying a hierarchy of headlines and the "body" of each unambiguously (possibly a few other things like properties drawer, planning, etc). For the most part I am using "whatever Org mode does" as the source of truth there, but IIRC there are edge cases where different Org mode commands do different things.
This would then allow one to develop a parser which can unambiguously parse an Org file into a tree with the invariant that emit(parse(org)) == org for all strings org.
This parser would then be very safe, because you could be assured that if you parse a large file, only the headlines that you modify would be changed[1], limiting the scope of damage, unintended side effects, etc, and preventing mass changes that propagate far from the site. I understand Org files as a tree, and dislike anything which changes the tree structure unintentionally much more strongly than any unintended change limited to a single headline/body.
I lack time at the moment, but I would like to implement the unambiguous headline parser some day because I'm not aware of anything like it. I think that there are many parsers (including orgize) that would be sufficient for my needs for actually parsing the content of a headline, since I'm much more flexible on what behavior I would accept there.
The main thing I would suggest is going over the tests in this repository/other parsers and collect their unit test strings. I also suggest reading the issues (open and closed) on this repository, as I have raised a number of edge cases in them.
[1] I believe there are two ambiguous cases: document terminal newline and initial newline, that can't be avoided in all cases.
gitonthescene
appetrosyan
PoiScript
Hello, I found your project from the worg tools list. First, sorry for the semi-spam nature of this issue. I had a notion for a project that the org community might find useful and I'm looking for feedback. Feel free to close this issue if it doesn't sound useful to you.
My idea is to start a list of org-mode snippets which can serve as a test bed for people developing tools. The idea is that having a separate collection of examples makes it easier for others in the community to benefit from the examples developed through communication with users.
Users could use these samples to try to construct minimal examples of issues they're having and/or contribute examples there which others could benefit from. Exactly how it will take shape is still up in the air.
These samples could also serve as a place to discuss ideas about how to develop the grammar itself. According to worg, the spec is still in draft state.
There's not much there at the moment. Mostly because I don't want to commit too early to what seems like it might be useful. I'll add more examples as I go.
If you like the concept and/or want to contribute and/or just want to offer feedback, I'd very much appreciate it.
Again, sorry for the spam.