search

Bears-R-Us / arkouda

Arkouda (αρκούδα): Interactive Data Analytics at Supercomputing Scale :bear:
Other
243 stars 87 forks source link

Intermittent CI failures with python `3.x/3.12` due to `ForwardRef` #3346

Open stress-tess opened 4 months ago

stress-tess commented 4 months ago

Even after pinning the numpy version to <2.0 I'm still seeing CI failures:https://github.com/Bears-R-Us/arkouda/actions/runs/9572308279/attempts/1

It seems only the 3.x python version is affected. The python version was 3.12.4 with an error along the lines of

TypeError: type of argument "pda1" must be one of (pdarray, Strings, ForwardRef('Categorical'), Sequence); got arkouda.categorical.Categorical instead

So this appears to be a forward ref issue

The interesting bit is that python 3.12.0 didn't have this issue and python 3.12.4 was released on June 6th but we've had successful runs since then...

I think the most straightforward plan is to pin the python version as well until we can dig into it more

stress-tess commented 4 months ago

it seems rerunning the test worked? I'm going to leave this issue open since this doesn't seem like a one off kinda problem

stress-tess commented 4 months ago

I'm working on eliminating the ForwardRef since that seems to be the main source of the issue. This stackoverflow comment seems to give a much cleaner solution for avoiding circular imports

https://stackoverflow.com/questions/39740632/python-type-hinting-without-cyclic-imports

stress-tess commented 4 months ago

Testing locally this seems to only be breaking for python 3.12.4. I pinned python to be less than this and disabled the 3.12/3.x CI checks for this release until we can get to the bottom of this

stress-tess commented 2 months ago

python 3.12.5 was released 2 days ago https://www.python.org/downloads/release/python-3125/

It seems to make some changes to how they handle ForwardRef in particular the __eq__ function: https://docs.python.org/release/3.12.5/whatsnew/changelog.html#python-3-12-5

I'm hoping this will fix the problems we were having. It's not yet available through conda, but I'm impatient so I'm going to build from source

edit: I was struggling to get this work and then they added it to conda forge while I was trying this out lol

stress-tess commented 2 months ago

okay that didn't fix our problem, but I did find this: https://typeguard.readthedocs.io/en/stable/userguide.html#permanently-suppressing-type-checks-for-selected-functions

indicating we needed to switch from @no_type_check to @typeguard_ignore, but that caused errors. I realized the version of typeguard is way behind. Updating that gives the following error:

TypeError: ForwardRef._evaluate() missing 1 required keyword-only argument: 'recursive_guard'

This one is seen by others when moving from 3.12.3 to 3.12.4, so I feel like we're on the right track. It seems most other ppl are using pydantic so this was an upstream error for them