Update fixes

[tomtau]

@TheVeryDarkness I previously merged that original PR by rebasing it, because I wanted to preserve a linear history without merge commits... but that made a merge conflict between those two master branches, so I created a new branch off the current master (update-fixes) and cherry-picked the new commits from your master. How do you prefer to continue working on it going forward? Continue on that diverged master and I'll cherry-pick the new commits, or work directly on a branch?

Summary by CodeRabbit

• New Features

  • Renamed function parse_with to parse_with_partial and added check_with_partial in repetition.rs.
  • Added methods try_parse_with_partial, try_parse_partial, check_with_partial, check_partial, parse_partial to TypedNode.
  • Introduced FullRuleStruct trait for rules with full capacity.
  • Defined PairContainer methods for handling pairs and children pairs.
  • Defined PairTree trait extending PairContainer with pair tree operation methods.
  • Introduced TypedParser trait for typed syntax tree production with parsing methods.

• Enhancements

  • Simplified handling of documentation comments in grammar generation for improved readability.
  • Added box_all_rules configuration flag for controlling rule boxing behavior in the parser.
  • Enhanced code documentation with detailed attributes for enum variants and struct fields.

• Refactor

  • Revised logic for generating rule enums and content type handling based on configuration flags.
  • Restructured documentation comment storage for better management.
  • Renamed functions and added methods to traits for improved functionality and clarity.

• Documentation

  • Updated grammar and rule documentation for clarity and understanding.
  • Enriched library documentation with detailed comments and attributes.

• Bug Fixes

  • Fixed attribute parsing to prevent potential runtime errors.

[coderabbitai[bot]]

Walkthrough

This update introduces a comprehensive enhancement across the board for a JSON parser using the pest library. It includes a new JSON parser definition, streamlined handling of rule documentation, extended support for grammar and rule documentation, upgraded typed syntax tree functionality, and new traits for improved parsing operations. The changes aim to make the parser more robust, maintainable, and easier to understand.

Changes

Files	Summary
derive/tests/json.pest, derive/tests/json.rs	Introduced a parser for JSON with tests covering basic and structural parsing.
generator/src/..., meta/src/..., pest/src/...	Enhanced documentation handling, simplified rule enum generation, and improved typed syntax tree functionality. Added support for grammar and rule documentation.
generator/src/typed/..., pest/src/typed/...	Modified parsing logic with new traits and methods for better parsing operations and reachability analysis.
pest/src/{choice.rs, formatter.rs, lib.rs, sequence.rs}	Added documentation and attributes, introduced core parts of pest3, and updated sequence type generation.

"In the realm of code, where the parsers play,
🐇 A rabbit hopped, crafting JSON's way.
With pest in paw, it weaved through the night,
🌟 Enhancing each rule, with documentation bright.
Through sequences, choices, and trees it did hop,
Leaving behind code, that's top of the crop!"
🚀🐰✨

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share

- [X](https://twitter.com/intent/tweet?text=I%20just%20used%20%40coderabbitai%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20the%20proprietary%20code.%20Check%20it%20out%3A&url=https%3A//coderabbit.ai) - [Mastodon](https://mastodon.social/share?text=I%20just%20used%20%40coderabbitai%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20the%20proprietary%20code.%20Check%20it%20out%3A%20https%3A%2F%2Fcoderabbit.ai) - [Reddit](https://www.reddit.com/submit?title=Great%20tool%20for%20code%20review%20-%20CodeRabbit&text=I%20just%20used%20CodeRabbit%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20proprietary%20code.%20Check%20it%20out%3A%20https%3A//coderabbit.ai) - [LinkedIn](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fcoderabbit.ai&mini=true&title=Great%20tool%20for%20code%20review%20-%20CodeRabbit&summary=I%20just%20used%20CodeRabbit%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20proprietary%20code)

---

Tips

### Chat There are 3 ways to chat with CodeRabbit: - Review comments: Directly reply to a review comment made by CodeRabbit. Example: - `I pushed a fix in commit .` - `Generate unit-tests for this file.` - Files and specific lines of code (under the "Files changed" tab): Tag `@coderabbitai` in a new review comment at the desired location with your query. Examples: - `@coderabbitai generate unit tests for this file.` - `@coderabbitai modularize this function.` - PR comments: Tag `@coderabbitai` in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples: - `@coderabbitai generate interesting stats about this repository from git and render them as a table.` - `@coderabbitai show all the console.log statements in this repository.` - `@coderabbitai read src/utils.ts and generate unit tests.` - `@coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.` Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. ### CodeRabbit Commands (invoked as PR comments) - `@coderabbitai pause` to pause the reviews on a PR. - `@coderabbitai resume` to resume the paused reviews. - `@coderabbitai review` to trigger a review. This is useful when automatic reviews are disabled for the repository. - `@coderabbitai resolve` resolve all the CodeRabbit review comments. - `@coderabbitai help` to get help. Additionally, you can add `@coderabbitai ignore` anywhere in the PR description to prevent this PR from being reviewed. ### CodeRabbit Configration File (`.coderabbit.yaml`) - You can programmatically configure CodeRabbit by adding a `.coderabbit.yaml` file to the root of your repository. - The JSON schema for the configuration file is available [here](https://coderabbit.ai/integrations/coderabbit-overrides.v2.json). - If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: `# yaml-language-server: $schema=https://coderabbit.ai/integrations/coderabbit-overrides.v2.json` ### CodeRabbit Discord Community Join our [Discord Community](https://discord.com/invite/GsXnASn26c) to get help, request features, and share feedback.

[TheVeryDarkness]

Sorry, I'll switch to that branch later.

[TheVeryDarkness]

@tomtau I've cherry-picked all commits in my hand. But module system hasn't been finished yet, so please merge this later.

[TheVeryDarkness]

@tomtau and could you please explain how the module path (mod1::mod2::mod3::...) is mapped into the path in file system (dir1/dir2/dir3/...)? I found grammar.pest only accepts module path, and parser.rs simply uses module path as file path and ignores built-in modules. And when the grammar parser sees mod1::mod2, should it be converted to ./mod1/mod2.pest? And how could user access files in ancesters folders, say ../common.pest?

[tomtau]

@TheVeryDarkness the pest3 module system isn't yet properly defined, there are two aspects to it: syntax and semantics.

syntax

I saw two proposals for it:

1. specifying system paths directly: https://github.com/pest-parser/pest/issues/333#Modules use "cool.pest"; use "this.pest" as that;
2. Rust-like modules: https://github.com/pest-parser/pest/pull/660

I think the current pest3 implementation was leaning towards the latter (even though the former was in the original proposal by @dragostis ) which seems fair: it's familiar to Rust users, it doesn't have issues with different system path notations on different operating systems, and can potentially be abstracted to "pest_vm".

And how could user access files in ancesters folders, say ../common.pest?

If we go with that, I guess we can refer to the ancestor modules using the super as in Rust?

semantics

There are multiple issues related to semantics, off the top of my head (there are likely more):

• whether we add/allow mod {} syntax and its precedence over filesystem
• mapping to filesystem
• mapping to "virtual" modules in pest_vm (e.g. in the browser context where there's no OS filesystem)
• where there are any visibility modifiers
• scoping (especially of trivia operators)

could you please explain how the module path (mod1::mod2::mod3::...) is mapped into the path in file system (dir1/dir2/dir3/...)? And when the grammar parser sees mod1::mod2, should it be converted to ./mod1/mod2.pest?

I think for this initial prototype, we can go with simple decisions, e.g. the grammar parser sees mod1::mod2, it can convert it to ./mod1/mod2.pest, scope trivia operators to each module (if one wants to re-use the trivia operator, they can re-import it explicitly), no visibility operators...

[tomtau]

@TheVeryDarkness I rebase-merged this one, so feel free to start a new branch off the latest master!

[TheVeryDarkness]

Sorry, I'm busy these weeks :( Maybe let me implement that feature this or next weekend.

[tomtau]

No worries!