Traceback mismatch in a couple tests on 3.11+

mahmoud commented 1 year ago

Python 3.11 came with an extra helping of carets:

-> assert expected == actual
(Pdb) print(expected)
Traceback (most recent call last):
  File "test_error.py", line ___, in _make_stack
    glom(target, spec)
  File "core.py", line ___, in glom
    raise err
glom.core.GlomError.wrap(Exception): error raised while processing, details below.
 Target-spec trace (most recent last):
 - Target: [None]
 - Spec: {'results': [{'value': <function _raise_exc at
 - Spec: [{'value': <function _raise_exc at
 - Target: None
 - Spec: {'value': <function _raise_exc at
 - Spec: <function _raise_exc at
  File "<boltons.funcutils.FunctionBuilder-0>", line ___, in _raise_exc
Exception: unique message

(Pdb) print(actual)
Traceback (most recent call last):
  File "test_error.py", line ___, in _make_stack
    glom(target, spec)
  File "core.py", line ___, in glom
    raise err
glom.core.GlomError.wrap(Exception): error raised while processing, details below.
 Target-spec trace (most recent last):
 - Target: [None]
 - Spec: {'results': [{'value': <function _raise_exc at
 - Spec: [{'value': <function _raise_exc at
 - Target: None
 - Spec: {'value': <function _raise_exc at
 - Spec: <function _raise_exc at
           ^^^^^^^^^^^^
  File "<boltons.funcutils.FunctionBuilder-0>", line ___, in _raise_exc
Exception: unique message

mahmoud commented 1 year ago

Poked at the new exception object, no dice. My initial readthrough of PEP657 and the traceback module confirms that the carets aren't actually part of the exception like SyntaxError, but part of the frame. So we can't just clear some value from the exception and go back to the previous behavior.

I think the next step is to explore either taking over more traceback rendering ourselves, or seeing if we can tweak the originating frame.

boonhapus commented 10 months ago

This appears to totally break rendering of the GlomError on py>=3.11

python 3.11

                    │ C:\projects\dark-seer\src\darkseer\informants\stratz\client.py:575 in match                                                                                                                      │
                    │                                                                                                                                                                                                  │
                    │   572 │   │                                                                                                                                                                                      │
                    │   573 │   │   try:                                                                                                                                                                               │
                    │   574 │   │   │   r = await self.query(q, match_id=match_id)                                                                                                                                     │
                    │ ❱ 575 │   │   │   d = glom.glom(r.json(), ("data.match", MATCH_SPEC))                                                                                                                            │
                    │   576 │   │   │                                                                                                                                                                                  │
                    │   577 │   │   │   # care for relationships                                                                                                                                                       │
                    │   578 │   │   │   if _ := d.get("REL_tournament"):                                                                                                                                               │
                    │                                                                                                                                                                                                  │
                    │ C:\projects\dark-seer\.venv\Lib\site-packages\glom\core.py:2293 in glom                                                                                                                          │
                    │                                                                                                                                                                                                  │
                    │   2290 │   │   │   raise                                                                                                                                                                         │
                    │   2291 │                                                                                                                                                                                         │
                    │   2292 │   if err:                                                                                                                                                                               │
                    │ ❱ 2293 │   │   raise err                                                                                                                                                                         │
                    │   2294 │   return ret                                                                                                                                                                            │
                    │   2295                                                                                                                                                                                           │
                    │   2296                                                                                                                                                                                           │
                    ╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
                    GlomError.wrap(TypeError): <exception str() failed>

python 3.10

                    │ C:\projects\dark-seer\src\darkseer\informants\stratz\client.py:575 in match                                                                                                                      │
                    │                                                                                                                                                                                                  │
                    │   572 │   │                                                                                                                                                                                      │
                    │   573 │   │   try:                                                                                                                                                                               │
                    │   574 │   │   │   r = await self.query(q, match_id=match_id)                                                                                                                                     │
                    │ ❱ 575 │   │   │   d = glom.glom(r.json(), ("data.match", MATCH_SPEC))                                                                                                                            │
                    │   576 │   │   │                                                                                                                                                                                  │
                    │   577 │   │   │   # care for relationships                                                                                                                                                       │
                    │   578 │   │   │   if _ := d.get("REL_tournament"):                                                                                                                                               │
                    │                                                                                                                                                                                                  │
                    │ C:\projects\dark-seer\.venv310\lib\site-packages\glom\core.py:2293 in glom                                                                                                                       │
                    │                                                                                                                                                                                                  │
                    │   2290 │   │   │   raise                                                                                                                                                                         │
                    │   2291 │                                                                                                                                                                                         │
                    │   2292 │   if err:                                                                                                                                                                               │
                    │ ❱ 2293 │   │   raise err                                                                                                                                                                         │
                    │   2294 │   return ret                                                                                                                                                                            │
                    │   2295                                                                                                                                                                                           │
                    │   2296                                                                                                                                                                                           │
                    ╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
                    GlomError.wrap(TypeError): error raised while processing, details below.
                     Target-spec trace (most recent last):
                     - Target: {'data': {'match': {'actualRank': 80, 'didRadiantWin': T... (len=1)
                     - Spec: ('data.match', {'REL_match_draft': (S(patch_id=T['gameVers... (len=2)
                     - Spec: 'data.match'
                     - Target: {'actualRank': 80, 'didRadiantWin': True, 'direTeam': {... (len=20)
                     - Spec: {'REL_match_draft': (S(patch_id=T['gameVersionId']), S(ma... (len=17)
                     - Spec: Invoke(<function _parse_event_types at 0x0000021DDB47C700>).specs(T)
                      File "C:\projects\dark-seer\src\darkseer\informants\stratz\_parse.py", line 610, in _parse_event_types
                        *[{**_, "id": i} for i, _ in enumerate(glom(target, SPEC_HERO_KILL_EVT))],
                      File "C:\projects\dark-seer\src\darkseer\informants\stratz\_parse.py", line 610, in <listcomp>
                        *[{**_, "id": i} for i, _ in enumerate(glom(target, SPEC_HERO_KILL_EVT))],
                    TypeError: 'NoneType' object is not a mapping

mahmoud commented 10 months ago

Oof. Looks like this is on Windows, and with a match spec. Not sure if we have a trace + Windows + match spec test specifically, but will investigate. Thanks for the report @boonhapus!

ajohnston9 commented 10 months ago

Hello,

Is there a path forward on this? Happy to draft a PR, unsure how to begin.

Thanks,

Andrew

mahmoud commented 10 months ago

Hey @ajohnston9 thanks for the offer! Kurt and I hammered a fix out last night and seems to be working fine on master. If you or anyone in the thread would like to help, try installing from master and let us know if the issue is resolved.

boonhapus commented 10 months ago

This looks good to me. I don't have the same data/spec as before, but I'm getting expected output on py3.11.

Platform: "win-amd64"
Python version: "3.11"
Current installation scheme: "venv"

Python 3.11.0 (main, Oct 24 2022, 18:26:48) [MSC v.1933 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import glom
>>> glom.__version__
'23.3.1.dev'

mahmoud commented 10 months ago

Excellent! I'll try to get a release out tonight/tomorrow. Thanks!

mahmoud commented 9 months ago

Took a little longer to get to this, but finally out (released while at GitHub Universe 2023, no less)

mahmoud / glom

Traceback mismatch in a couple tests on 3.11+ #264