Task Deefinition Without Braces

[lwgray]

Enhanced Newline Handling

Level: Knight

Description

Implement sophisticated newline handling to support indentation-based syntax while properly handling special cases like multi-line expressions, comments, and blank lines.

Technical Details

Changes Needed in parser.py

1. Update lexer state tracking:

   def __init__(self):
   self.paren_count = 0     # Track nested parentheses
   self.bracket_count = 0    # Track nested brackets
   # ... existing init code ...

2. Enhance newline handling:

   
   def t_NEWLINE(self, t):
   r'\n+'
   t.lexer.lineno += len(t.value)
   # Ignore newlines inside parentheses/brackets
   if self.paren_count == 0 and self.bracket_count == 0:
       return t

def t_LPAREN(self, t): r'(' self.paren_count += 1 return t

def t_RPAREN(self, t): r')' self.paren_count -= 1 return t

3. Add comment handling:
```python
def t_COMMENT(self, t):
    r'\#.*'
    # Comments don't generate tokens
    pass

Test Cases

Test Case 1: Multi-line Expression

task "MultiLine"
    x = (1 + 
         2 + 
         3)
    y = 4

Should treat the multi-line expression as a single statement.

Test Case 2: Comments and Blank Lines

objective "Test"

# Comment at root level
task "Comments"
    # Indented comment
    x = 1

    # Comment with blank line above
    y = 2

Should handle comments and blank lines properly.

Test Case 3: Nested Structures

task "Nested"
    x = [1, 2,
         3, 4]
    step "Inner"
        y = (5 +
             6)

Should handle nested structures with multi-line expressions.

Test Case 4: Invalid Newlines

task "Invalid"
    if x > 
        0:  # This should raise an error
        y = 1

Should detect invalid line breaks.

Completion Criteria

1. [ ] Newlines properly handled inside parentheses/brackets
2. [ ] Comments correctly ignored in token stream
3. [ ] Blank lines handled properly
4. [ ] Multi-line expressions parsed correctly
5. [ ] Line numbers tracked accurately
6. [ ] Error detection for invalid line breaks
7. [ ] All test cases pass
8. [ ] Existing brace syntax still works

Dependencies

• Issue #21 : Basic Indentation Tracking

Documentation Updates Needed

• Document multi-line expression rules
• Add section on comment handling
• Update line continuation guidelines
• Add examples of valid/invalid line breaks

Testing Instructions

1. Run provided test cases
2. Verify line number tracking
3. Test multi-line expressions
4. Validate comment handling
5. Check error detection
6. Verify integration with indentation tracking