Support for Python 3.11

[yelinaung]

Describe the bug you encountered:

Trying to integrate pyroscope into Python 3.11 and the application is not showing up on the dashboard. After enabling the logs, saw these messages

2022-11-29T09:45:57.603Z INFO  py_spy::python_spy > Got virtual memory maps from pid 2539574:
2022-11-29T09:45:57.608Z INFO  py_spy::python_spy > Getting version from python binary BSS
2022-11-29T09:45:57.608Z INFO  py_spy::python_spy > Failed to get version from BSS section: failed to find version string
2022-11-29T09:45:57.608Z INFO  py_spy::python_spy > Trying to get version from path: /home/yelinaung/.pyenv/versions/3.11.0/bin/python3.11
2022-11-29T09:45:57.608Z INFO  py_spy::python_spy > python version 3.11.0 detected
2022-11-29T09:45:57.608Z INFO  py_spy::python_spy > Failed to get interp_head from symbols, scanning BSS section from main binary
2022-11-29T09:45:57.608Z INFO  py_spy::python_spy > Failed to connect to process, retrying. Error: Unsupported version of Python: 3.11.0
2022-11-29T09:45:57.629Z INFO  py_spy::python_spy > Got virtual memory maps from pid 2539574:
2022-11-29T09:45:57.634Z INFO  py_spy::python_spy > Getting version from python binary BSS
2022-11-29T09:45:57.634Z INFO  py_spy::python_spy > Failed to get version from BSS section: failed to find version string
2022-11-29T09:45:57.634Z INFO  py_spy::python_spy > Trying to get version from path: /home/yelinaung/.pyenv/versions/3.11.0/bin/python3.11
2022-11-29T09:45:57.634Z INFO  py_spy::python_spy > python version 3.11.0 detected
2022-11-29T09:45:57.634Z INFO  py_spy::python_spy > Failed to get interp_head from symbols, scanning BSS section from main binary
2022-11-29T09:45:57.635Z INFO  py_spy::python_spy > Failed to connect to process, retrying. Error: Unsupported version of Python: 3.11.0

What did you expect to happen instead?

I expect the pyroscope to be working with Python 3.11.

How did you install pyroscope-rs? I installed it via pip and also as part of the requirements.txt.

---

pyroscope-rs version and environment

• pyroscope python version 0.8.0
• environment - ubuntu linux

[yelinaung]

I realized that the py-spy version used in pyroscope doesn't have the latest py-spy version py-spy version used: 0.3.12 https://github.com/pyroscope-io/pyroscope-rs/blob/main/pyroscope_backends/pyroscope_pyspy/Cargo.toml#L17-L20

Only py-spy 0.3.13 has Python 3.11 support https://github.com/benfred/py-spy/releases/tag/v0.3.13

so maybe bumping the py-spy version will fix this?

[korniltsev]

bumping py-spy should help Another problem is that one of our build dependencies (milksnakex) is not ready for 3.11

https://github.com/omarabid/milksnakex/blob/master/milksnake/ffi.py#L8

  File "/tmp/build-env-dxzj3a1v/lib/python3.11/site-packages/milksnake/setuptools_ext.py", line 226, in make_ffi
    from milksnake.ffi import make_ffi
  File "/tmp/build-env-dxzj3a1v/lib/python3.11/site-packages/milksnake/ffi.py", line 8, in <module>
    _directive_re = re.compile(r'^\s*#.*?$(?m)')
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/korniltsev/.asdf/installs/python/3.11.0/lib/python3.11/re/__init__.py", line 227, in compile
    return _compile(pattern, flags)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/korniltsev/.asdf/installs/python/3.11.0/lib/python3.11/re/__init__.py", line 294, in _compile
    p = _compiler.compile(pattern, flags)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/korniltsev/.asdf/installs/python/3.11.0/lib/python3.11/re/_compiler.py", line 743, in compile
    p = _parser.parse(p, flags)
        ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/korniltsev/.asdf/installs/python/3.11.0/lib/python3.11/re/_parser.py", line 980, in parse
    p = _parse_sub(source, state, flags & SRE_FLAG_VERBOSE, 0)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/korniltsev/.asdf/installs/python/3.11.0/lib/python3.11/re/_parser.py", line 455, in _parse_sub
    itemsappend(_parse(source, state, verbose, nested + 1,
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/korniltsev/.asdf/installs/python/3.11.0/lib/python3.11/re/_parser.py", line 841, in _parse
    raise source.error('global flags not at the start '
re.error: global flags not at the start of the expression at position 9

[korniltsev]

https://github.com/omarabid/milksnakex/pull/1

[korniltsev]

Bumping py-spy worked https://flamegraph.com/share/de380019-6ff1-11ed-a919-5a9d7314f5ec Waiting for milksnakex PR now

[yelinaung]

@korniltsev Seems like the PR is approved. Any ETA for when the new release will be out? Thanks!

[yelinaung]

Thanks @korniltsev!

[korniltsev]

It failed to build, I am not familiar with build system yet, will try to fix now

[yelinaung]

@korniltsev is the build system still failing to release the new Python package that supports 3.11?

[korniltsev]

no(almost no - windows tests are failing, I will disable them for now) I will try to release today

[korniltsev]

it failed again, at the very last step of publishing it https://github.com/pyroscope-io/pyroscope-rs/actions/runs/3948828459/jobs/6759290100

we will have to wait abit until we publish milksnakex, sorry for inconvenince

[korniltsev]

@yelinaung finally released it. Sorry for long wait. Could you please check if a new release fixes your problem?

[yelinaung]

@korniltsev Yep! it's working now! Thanksss 🙇🏾