search

neuromorphic-paris / loris

python3 library to handle files from neuromorphic cameras
GNU General Public License v3.0
14 stars 3 forks source link

Incompatibility with NumPy 2.0 #11

Open Tobias-Fischer opened 2 months ago

Tobias-Fischer commented 2 months ago

Error:

loris_extension.cpp:241:42: error: no member named 'fields' in '_PyArray_Descr'

See https://github.com/conda-forge/loris-feedstock/pull/8

biphasic commented 2 months ago

thank you Tobias but I'm afraid neither Alex nor me have the time to maintain this project at the moment. Alex wrote a new version of a parser library that works with the .es format https://github.com/neuromorphicsystems/event_stream and also the aedat format https://github.com/neuromorphicsystems/aedat for info

aMarcireau commented 1 month ago

@Tobias-Fischer, @biphasic,

The problem comes from the use of direct accessors on numpy structures, since they are replaced with functions in numpy 2.0. https://github.com/neuromorphic-paris/loris/blob/f21c819cfb9754b48beff2f7c69463d996549496/source/cpp/loris_extension.cpp#L241

This is only a problem in event_stream_write since it needs to validate the input array's type. The fix recommended by numpy is pretty straightforward and I readily implemented it in event_stream (https://github.com/neuromorphicsystems/event_stream/commit/960bd634c91ce4d6df08b79ba6493da142117a76 and https://github.com/neuromorphicsystems/event_stream/commit/e3248b32b2cc6c60634f6e77f0afbcdfaa702d7e), which uses code that is very similar to Loris's.

The C++ extension used by Loris is there to support .dat and .es. I think that this leaves us with two options to support numpy 2:

Which option do you prefer?

aMarcireau commented 1 month ago

Another (potentially more problematic) issue for numpy 2 support is that numpy 1 and numpy 2 are not binary-compatible. Whilst we can make sure that event-stream / aedat / loris can be built with both, wheels built with GitHub Actions and shipped via PyPi or Conda need to be compiled with one or the other. The user will need to use the same version of numpy at runtime to avoid issues. We could simply mark these libraries as depending on numpy 2, but this could be an issue if users need other libraries that have not been upgraded and need numpy 1 in the same project.

I am tempted to continue compiling with numpy 1 (at least for another month or so). Users that need numpy 2 will need to compile from source with the sdist in the mean time.

Tobias-Fischer commented 1 month ago

Hi @aMarcireau - thanks a lot for looking into this! Re strip vs port: I don't have a preference - it seems like a relatively minor change, so porting is probably the easier way to go? Re binary compatibility: For conda-forge, this is being taken care of. The migrator in https://github.com/conda-forge/loris-feedstock/pull/8 once merged will add a dependency on numpy 2, and thus the package going forward will not install in numpy 1 environments.

Tobias-Fischer commented 1 month ago

For a port see #12