[Bug][pysrc2cpg] `Invalid Code` parsing issues

[AndreiDreyer]

Describe the bug When trying to generate a cpg of the following repo Pygoat, the python parser shows two warnings for lines of code it considers Invalid.

These relate to the following functions:

To Reproduce Steps to reproduce the behavior: 1) Create test.py with the following:

def register(request):
    if request.method == "POST":
        form = NewUserForm(request.POST)
        if form.is_valid():
            user = form.save()
            login(request, user)
            messages.success(request, "Registration successful." )
            return redirect('/')
        messages.error(request, "Unsuccessful registration. Invalid information.")
    form = NewUserForm()
    return render (request=request, template_name="registration/register.html", context={"register_form":form})

2) ./joern-parse test.py

1) Create test.py with the following:

class NewUserForm(UserCreationForm):
    def save(self, commit=True):
        user = super(NewUserForm, self).save(commit=False)
        user.email = self.cleaned_data['email']
        if commit:
            user.save()
        return user

2) ./joern-parse test.py

• From some limited testing, this parsing error only appears when the save function is defined in the class, but parses fine if it is defined in the top level of the program

Expected behavior No parsing warning shown for the above code.

Desktop (please complete the following information):

• OS: Ubuntu
• Joern Version: 2.0.375
• Java version: Java 17.0.10

[DavidBakerEffendi]

Very strange, if I re-indent the whitespace for user.save() I parse successfully. Perhaps a fix for this issue should be robust to the whitespace.

Specifically, the before user.save() is the problem. There are three tabs before it, but the third tab closest is the issue.

[DavidBakerEffendi]

Detailed error from the parser is

Encountered " <NAME> "user "" at line 7, column 4.
Was expecting:
    <INDENT> ...

From the message above, if I add another it also parses successfully

[DavidBakerEffendi]

Using a formatter seems to help here:

cd pygoat
pip3 install black
black .

Then I no longer get parser issues.

Not ideal, this is only a workaround, but should unblock this issue.