ci: add boto3 framework tests

[P403n1x87]

This change adds the boto3 test suite run to check that when all the encountered functions are recompiled with the library the test suite still passes.

[P403n1x87]

@MatthieuDartiailh as a testing strategy I'm thinking of testing some popular libraries by decompiling and recompiling all code objects that are imported during test runs. With this approach I caught

Traceback (most recent call last):
  File "/home/gabriele/Projects/bytecode/tests/frameworks/function.py", line 26, in linenos
    return {instr.lineno for instr in Bytecode.from_code(f.__code__) if hasattr(instr, "lineno")}
                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gabriele/Projects/bytecode/src/bytecode/bytecode.py", line 190, in from_code
    return concrete.to_bytecode()
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gabriele/Projects/bytecode/src/bytecode/concrete.py", line 457, in to_bytecode
    arg = self.names[arg]
          ~~~~~~~~~~^^^^^
IndexError: list index out of range

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/tmp/boto-3.11/boto3/scripts/ci/run-tests", line 6, in <module>
    import argparse
  File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1149, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "/home/gabriele/Projects/bytecode/tests/frameworks/module.py", line 83, in _exec_module
    self.loader.exec_module(module)
  File "/usr/lib/python3.11/argparse.py", line 95, in <module>
    from gettext import gettext as _, ngettext
  File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1149, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "/home/gabriele/Projects/bytecode/tests/frameworks/module.py", line 85, in _exec_module
    callback(module)
  File "/home/gabriele/Projects/bytecode/tests/frameworks/sitecustomize.py", line 13, in after_import
    discovery = FunctionDiscovery(module)
                ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gabriele/Projects/bytecode/tests/frameworks/function.py", line 180, in __init__
    for lineno in linenos(cast(FunctionType, function)):
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gabriele/Projects/bytecode/tests/frameworks/function.py", line 28, in linenos
    raise RuntimeError("Cannot get line numbers for function '{}' from module '{}'".format(f.__name__, f.__module__)) from e
RuntimeError: Cannot get line numbers for function '_tokenize' from module 'gettext'

on Python 3.11.

[MatthieuDartiailh]

@MatthieuDartiailh as a testing strategy I'm thinking of testing some popular libraries by decompiling and recompiling all code objects that are imported during test runs. With this approach I caught

Traceback (most recent call last):
  File "/home/gabriele/Projects/bytecode/tests/frameworks/function.py", line 26, in linenos
    return {instr.lineno for instr in Bytecode.from_code(f.__code__) if hasattr(instr, "lineno")}
                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gabriele/Projects/bytecode/src/bytecode/bytecode.py", line 190, in from_code
    return concrete.to_bytecode()
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gabriele/Projects/bytecode/src/bytecode/concrete.py", line 457, in to_bytecode
    arg = self.names[arg]
          ~~~~~~~~~~^^^^^
IndexError: list index out of range

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/tmp/boto-3.11/boto3/scripts/ci/run-tests", line 6, in <module>
    import argparse
  File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1149, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "/home/gabriele/Projects/bytecode/tests/frameworks/module.py", line 83, in _exec_module
    self.loader.exec_module(module)
  File "/usr/lib/python3.11/argparse.py", line 95, in <module>
    from gettext import gettext as _, ngettext
  File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1149, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "/home/gabriele/Projects/bytecode/tests/frameworks/module.py", line 85, in _exec_module
    callback(module)
  File "/home/gabriele/Projects/bytecode/tests/frameworks/sitecustomize.py", line 13, in after_import
    discovery = FunctionDiscovery(module)
                ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gabriele/Projects/bytecode/tests/frameworks/function.py", line 180, in __init__
    for lineno in linenos(cast(FunctionType, function)):
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gabriele/Projects/bytecode/tests/frameworks/function.py", line 28, in linenos
    raise RuntimeError("Cannot get line numbers for function '{}' from module '{}'".format(f.__name__, f.__module__)) from e
RuntimeError: Cannot get line numbers for function '_tokenize' from module 'gettext'

on Python 3.11.

Can you try again I believe I fixed the underlying issue ?

[MatthieuDartiailh]

Can you re-run ? I fixed the formatting issue and I would like to see what weird offset we are dealing with.

[P403n1x87]

Sure. Should be easy to run locally for a quicker feedback loop:

bash scripts/frameworks/boto3/setup.sh /tmp/boto3 "3.11"
PYTHONPATH=tests/frameworks bash scripts/frameworks/boto3/run.sh /tmp/boto3 "3.11"

[MatthieuDartiailh]

Thanks. I will investigate but to me it looks either like a bug in dis or something super weird in boto. Offsets should be always increasing.

[MatthieuDartiailh]

I figured the cell name issue but I will need some time to address it properly and in particular add regression tests.

[MatthieuDartiailh]

This looks rather good to me. One thing that can improved maybe would be to collect all errors so that we get a global views of all the errors (but it may bloat the log).

Did you have anything else in mind and is that why the PR is still WIP ?

[P403n1x87]

Did you have anything else in mind and is that why the PR is still WIP ?

Ah no sorry, just forgot to un-draft it. There are some potential improvements, but I don't have the time to look into them just now, so they could go with a follow-up PR.

[MatthieuDartiailh]

Thanks. I will merge as soon as I get the CIs green on #107 to help me catch the last possible issues.