search

ThePrimeagen / ts-rust-zig-deez

627 stars 162 forks source link

Refactoring of Project with New Lexer, Parser, AST and REPL Implementation: rust_nom to rust_dr #229

Closed anis-dr closed 1 year ago

anis-dr commented 1 year ago

Summary:

This Pull Request includes major updates to the project, introducing new features and implementations. The project has transitioned from the previous name, rust_nom, to rust_dr, which reflects the username anis_dr. A new, feature-complete lexer, parser, abstract syntax tree (AST), and REPL have been implemented, marking key features of this refactor.

Changes:

The notable changes in this PR include:

  1. Project Rename: The project directory and the package in the Cargo.toml have been renamed from rust_nom to rust_dr. This new name reflects the username anis_dr, and distinguishes this specific implementation within the open source community.

  2. Lexer Refactoring: The lexer implementation has been enhanced and is now feature-complete. Token::Ident and Token::Int now take a String instead of &[u8], providing a more intuitive and flexible representation of these tokens.

  3. AST Introduction: A new module, ast, has been introduced. This module contains the definitions for the language's AST nodes, providing a structure for representing the syntax of our language and aiding in efficient parsing and interpretation.

  4. Parser Implementation: A new parser module has been added, which works in tandem with the lexer to generate an AST from the input source code. It provides the functionality to interpret the syntax of our language and also includes error handling and tests for parsing let statements.

  5. REPL Implementation: A new module, repl, has been introduced. This module implements a basic REPL for our language, which tokenizes user input and prints the corresponding tokens to the screen.

  6. Main Module Update: The main.rs file has been updated to launch the REPL when the program is run.

Testing:

New tests have been added to verify the functionality of the new lexer and parser, covering single-character tokens, multi-character tokens, and a complete sample program to ensure all tokens are correctly identified.

Impact:

This PR enhances the project with a feature-complete lexer, an interpreter in the form of a parser and AST, and an interactive user interface through the REPL.

Review Checklist:

  1. All tests pass.
  2. The new Lexer correctly identifies all token types.
vhladko commented 1 year ago

can you add yourself as codeowner for that folder ?

anis-dr commented 1 year ago

done

Spongman commented 1 year ago

you might want to add a Makefile and Dockerfile as described in the https://github.com/ThePrimeagen/ts-rust-zig-deez/blob/master/README.md.

vhladko commented 1 year ago

@anis-dr, yes, @Spongman is right, but you can do that in a new PR, I'll merge this one. The thing is, that I believe folders without Makefile and Dockerfile just won't be tested in the final race, so it's in your interests :)