Additional Lexer Test Cases

This issue is created to propose additional test cases for the Lexer component of the Compilator C++ Compiler project. These test cases aim to further enhance the test coverage and ensure the correctness of the Lexer's functionality.

Test Case 1: Different Operators

Input Code: Test the lexer's recognition of different operators.
```
int result = x * y - z / 2;
```
Expected Tokens:
TokenType::IDENTIFIER (int)
TokenType::IDENTIFIER (result)
TokenType::EQUAL
TokenType::IDENTIFIER (x)
TokenType::MULTIPLY
TokenType::IDENTIFIER (y)
TokenType::MINUS
TokenType::IDENTIFIER (z)
TokenType::DIVIDE
TokenType::INTEGER (2)
TokenType::SEMICOLON
TokenType::END_OF_FILE

Test Case 2: Keywords

Input Code: Test the lexer's recognition of C++ keywords.
```
if (condition) {
return 42;
}
```
Expected Tokens:
TokenType::IF
TokenType::LEFT_PAREN
TokenType::IDENTIFIER (condition)
TokenType::RIGHT_PAREN
TokenType::LEFT_BRACE
TokenType::RETURN
TokenType::INTEGER (42)
TokenType::SEMICOLON
TokenType::RIGHT_BRACE
TokenType::END_OF_FILE

Test Case 3: String and Character Literals

std::string greeting = "Hello, World!";
char ch = 'A';

Expected Tokens:

TokenType::IDENTIFIER (std)
TokenType::DOUBLE_COLON
TokenType::IDENTIFIER (string)
TokenType::IDENTIFIER (greeting)
TokenType::EQUAL
TokenType::STRING ("Hello, World!")
TokenType::SEMICOLON
TokenType::IDENTIFIER (char)
TokenType::IDENTIFIER (ch)
TokenType::EQUAL
TokenType::CHARACTER ('A')
TokenType::SEMICOLON
TokenType::END_OF_FILE

Test Case 4: Comments

// This is a single-line comment

/*
   This is a
   multi-line
   comment
*/

Expected Tokens:

TokenType::END_OF_FILE

Test Case 5: Identifiers with Digits cpp var123 = x42 + a1b2c3; Expected Tokens:

TokenType::IDENTIFIER (var123)
TokenType::EQUAL
TokenType::IDENTIFIER (x42)
TokenType::PLUS
TokenType::IDENTIFIER (a1b2c3)
TokenType::SEMICOLON
TokenType::END_OF_FILE

Test Case 6: Special Characters @variable $amount %value &name Expected Tokens: (All characters should be treated as errors)

Test Case 7: Empty Input (empty string) Expected Tokens:

TokenType::END_OF_FILE

Test Case 8: Non-ASCII Characters

你好, 世界
αβγδε

Expected Tokens: (All characters should be treated as errors)

Test Case 9: File with Line Endings Line 1\r\nLine 2\nLine 3 Expected Tokens: (List of tokens for each line) TokenType::END_OF_FILE

Test Case 10: Large Numbers

int bigNumber = 12345678901234567890;

Expected Tokens:

TokenType::IDENTIFIER (int)
TokenType::IDENTIFIER (bigNumber)
TokenType::EQUAL
TokenType::INTEGER (12345678901234567890)
TokenType::SEMICOLON
TokenType::END_OF_FILE

Test Case 11: Escaped Characters

std::string message = "Hello\nWorld";

Expected Tokens:

TokenType::IDENTIFIER (std)
TokenType::DOUBLE_COLON
TokenType::IDENTIFIER (string)
TokenType::IDENTIFIER (message)
TokenType::EQUAL
TokenType::STRING ("Hello\nWorld")
TokenType::SEMICOLON
TokenType::END_OF_FILE

diyorsattarov / 2

Additional Lexer Test Cases #4