Importing symtable causes traceback

[gitlab-importer]

In Heptapod by @rouilj on Nov 30, 2021, 19:21

I am trying to run the Roundup Issue Tracker under PyPy. Part of the code uses the symtable module. When trying to import it I get:

$ pypy3 -c 'import symtable'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/opt/pypy/lib-python/3/symtable.py", line 3, in <module>
    import _symtable

This is with a dockerized version of PyPY:

% pypy -V
Python 3.7.12 (44db26267d0a, Oct 24 2021, 14:21:50)
[PyPy 7.3.7 with GCC 7.3.1 20180303 (Red Hat 7.3.1-5)]

The code is used by Zope Page Templates (ZPT). I think it's using the Python interpreter to exec Python expressions in the templates then using has_children() and get_children() to recursively walk the expression pulling out variable values.

This code in PythonExpr.py looks like the use cases:

    def _get_from_symtab(self):
        """
        Get the variables used in the 'f' function.
        """
        variables = set()
        table = symtable.symtable(self.f_code, "<string>", "exec")
        if table.has_children():
            variables.update(self._walk_children(table))
        return variables

    def _walk_children(self, sym):
        """
        Get the variables at this level. Recurse to get them all.
        """
        variables = set()
        for child in sym.get_children():
            variables.update(set(child.get_identifiers()))
            if child.has_children():
                variables.update(self._walk_children(child))
        return variables

I brought this up on irc #pypy and cfboltz recommended opening an issue.

[gitlab-importer]

In Heptapod by @mattip on Jan 12, 2022, 08:53

PyPy has a symtable module in pypy/interpreter/astcompiler/symtable.py, but it is structured very differently to the CPython one.

[gitlab-importer]

In Heptapod by @rouilj on May 12, 2022, 03:59

This file shows how trying to get symbols in cpython 2.7 differed from 3.x.

https://issues.roundup-tracker.org/file1708/python_bug.py

So the need to import symtable and walk the table to identify the variables was required.

This is tracked on the roundup-tracker issue tracker as well at: https://issues.roundup-tracker.org/issue2551174 that may have some additional info on why we need symtable.

There may be a different way to do it under pypy3. Any ideas welcome.

[gitlab-importer]

In Heptapod by @cfbolz on May 16, 2022, 06:54

I think the _symtablemodule was added in more recent releases of pypy, certainly not the 3.7 version. Is there a reason you're trying 3.7 instead of 3.9?

[gitlab-importer]

In Heptapod by @rouilj on May 16, 2022, 18:38

Thanks for your response. Interesting that _symtable may not exist in 3.7.

I am using the official pypy docker. docker pull pypy:3 downloads a newer build, but it is still pypy version 3.7 as of a few minutes ago.

# pypy -V
Python 3.8.13 (4b1398fe9d76ad762155d03684c2a153d230b2ef, Mar 29 2022, 07:08:24)
[PyPy 7.3.9 with GCC 10.2.1 20210130 (Red Hat 10.2.1-11)]

[gitlab-importer]

In Heptapod by @mattip on May 16, 2022, 19:12

The word "official" in that image refers to (quote)

the Git repo of the Docker "Official Image" for pypy (not to be confused with any official pypy image provided by pypy upstream).

I opened an issue there https://github.com/docker-library/pypy/issues/75

[gitlab-importer]

In Heptapod by @cfbolz on May 16, 2022, 19:17

oh no, I am super sorry, I confused _opcode with _symtable in my head (because both expose internals of the ast compiler). _symtable remains not supported on pypy at this point. As Armin notes in #2130, it would not be impossible to support.

[gitlab-importer]

In Heptapod by @rouilj on May 16, 2022, 20:18

@mattip thanks for opening the ticket. It would be good to get that updated anyway. I must have skipped over the "official" definition part....

@cfbolz no prob. They both start with _ so are similar enough in my book 8-). Supporting _symtable just requires somebody with the skills to do it.

I would be nice to see if I can get a faster Roundup 8-).

[gitlab-importer]

In Heptapod by @mattip on May 17, 2022, 20:00

docker pull pypy:3 downloads a newer build, but it is still pypy version 3.

@rouilj could you comment on the docker issue? They have a test that they are using pypy3.8 in the alias, something is off.

[gitlab-importer]

In Heptapod by @rouilj on May 17, 2022, 23:23

Hi @mattip:

The most recent docker I pulled yesterday was:

# pypy -V
Python 3.8.13 (4b1398fe9d76ad762155d03684c2a153d230b2ef, Mar 29 2022, 07:08:24)
[PyPy 7.3.9 with GCC 10.2.1 20210130 (Red Hat 10.2.1-11)]

as you can see above. I am not sure how PyPY versions its releases. There seems to be both a PyPY release (7.3.9) and a Python (language ??) version (3.8.13) as well. I am not sure which is supposed to be used to determine the version of PyPY.

I commented on the GitHub ticket as well.

[gitlab-importer]

In Heptapod by @mattip on May 18, 2022, 06:35

There are two versions: the version of the python language (3.8.13) and the version of the PyPy interpreter (7.3.9). Both are the updated ones from the latest release. Here is the entire matrix of downloads. It is a bit confusing because there are similar digits in both numbers.