lcompilers / lpython

Python compiler
https://lpython.org/
Other
1.5k stars 163 forks source link

Match statement is not recognized #1022

Closed Thirumalai-Shaktivel closed 2 years ago

Thirumalai-Shaktivel commented 2 years ago

Example:

def _get_slots(cls):
    match cls.__dict__.get('__slots__'):
        case None:
            return
        case str(slot):
            yield slot
        # Slots may be any iterable, but we cannot handle an iterator
        # because it will already be (partially) consumed.
        case iterable if not hasattr(iterable, '__next__'):
            yield from iterable
        case _:
            raise TypeError(f"Slots of '{cls.__name__}' cannot be determined")

Error:

syntax error: Token 'cls' (of type 'identifier') is unexpected here
    --> ./Lib/dataclasses.py:1130:11
     |
1130 |     match cls.__dict__.get('__slots__'):
     |           ^^^
Thirumalai-Shaktivel commented 2 years ago

Here, match and case are soft keywords: https://docs.python.org/3/reference/compound_stmts.html#the-match-statement

The match and case keywords are proposed to be soft keywords, so that they are recognized as keywords at the beginning of a match statement or case block respectively, but are allowed to be used in other places as variable or argument names.

https://peps.python.org/pep-0622/#the-match-statement @certik how do we handle this?

certik commented 2 years ago

So match is a keyword that is not reserved. See https://docs.python.org/3/reference/lexical_analysis.html#soft-keywords, they call it a soft keyword. In our tokenizer.re, we have the following sets of reserved keywords:

"as"      
"assert"  
"async"   
"await"   
"break"   
"class"   
"continue"
"def"     
"del"     
"elif"    
"else"    
"except"  
"finally" 
"for"     
"from"    
"global"  
"if"      
"import"  
"in"      
"is"      
"lambda"  
"None"    
"nonlocal"
"pass"    
"raise"   
"return"  
"try"     
"while"   
"with"    
"yield"   
"yield from"

These cannot be used as variable names such as in with = 5.

match, case are different (perhaps also _ as the docs say, not sure about how we handle that --- looks like _ can also be a variable name). We have to allow them as variable names.

certik commented 2 years ago

We have several options how to tackle the match and case non-reserved keywords (each has pros/cons):

certik commented 2 years ago

We should probably fix this one first, now when the new parser is merged.

Thirumalai-Shaktivel commented 2 years ago

Remaining issues:

match():
   # Comment
    case():
        x = 0

match ():     # Comment
    case ():
        pass
    case ['remove',
              *files]:
        print('Removing files: {}'.format(files))

match = {2:
    3}
match[2:
    8]