Missing package: awkward

matthewfeickert commented 2 years ago

Package name

awkward

Package version

1.0.0+

PyPI URL

https://pypi.org/project/awkward/

piwheels URL

https://www.piwheels.org/project/awkward/

Python version

[ ] Python 3.5
[X] Python 3.7
[X] Python 3.9

I am the maintainer

[ ] Yes

More information

Wheels exists on piwheels up through v0.14.0 (last release before v1.0.0) and then no wheels exist for v1.0.0+. c.f. https://www.piwheels.org/project/awkward/
GitHub repo: https://github.com/scikit-hep/awkward-1.0
Maintainer is @jpivarski

bennuttall commented 2 years ago

It's running out of memory: https://paste.debian.net/1226576/

bennuttall commented 2 years ago

It built fine for me on an 8GB Pi, but not on the 4GB ones we have available. I'll build these ones offline and import them, and will see what we can do in future.

jpivarski commented 2 years ago

This will get easier to compile in version 2.0 later this year (probably spring). The majority of the C++ code is being ported to Python, leaving a much smaller C++ core to compile in limited memory situations. I don't know if you want to just wait for that, or what your intentions for this on Raspberry Pi is.

matthewfeickert commented 2 years ago

I'll build these ones offline and import them, and will see what we can do in future.

Thanks @bennuttall. That's very kind of you and I appreciate you taking time to do this for any of them. Just one release is fine for now.

This will get easier to compile in version 2.0 later this year (probably spring). ... I don't know if you want to just wait for that, or what your intentions for this on Raspberry Pi is.

Thanks @jpivarski. Yeah, this is just for fun and not for work so I think that we can in general wait for awkward v2. :+1:

bennuttall commented 2 years ago

I've imported the cp37m wheels, and currently building the cp39 wheels :+1:

bennuttall commented 2 years ago

Imported cp39 wheels :+1:

matthewfeickert commented 2 years ago

Thanks @bennuttall! Super appreciated. :)

matthewfeickert commented 2 years ago

oh. My guess is this is a missing library on my side (...actually maybe not), but I'm getting a

ImportError: /home/feickert/.local/venvs/base/lib/python3.9/site-packages/awkward/_ext.cpython-39-arm-linux-gnueabihf.so: undefined symbol: __atomic_fetch_add_8

Short summary of machine specs and example

(base) feickert@raspberrypi:~ $ cat /etc/os-release 
PRETTY_NAME="Raspbian GNU/Linux 11 (bullseye)"
NAME="Raspbian GNU/Linux"
VERSION_ID="11"
VERSION="11 (bullseye)"
VERSION_CODENAME=bullseye
ID=raspbian
ID_LIKE=debian
HOME_URL="http://www.raspbian.org/"
SUPPORT_URL="http://www.raspbian.org/RaspbianForums"
BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs"
(base) feickert@raspberrypi:~ $ uname -srvmo
Linux 5.10.63-v7l+ #1488 SMP Thu Nov 18 16:15:28 GMT 2021 armv7l GNU/Linux
(base) feickert@raspberrypi:~ $ python --version --version
Python 3.9.2 (default, Mar 12 2021, 04:06:34) 
[GCC 10.2.1 20210110]
(base) feickert@raspberrypi:~ $ python -m pip install --upgrade pip setuptools wheel
(base) feickert@raspberrypi:~ $ python -m pip install awkward
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting awkward
  Downloading https://www.piwheels.org/simple/awkward/awkward-1.7.0-cp39-cp39-linux_armv7l.whl (13.7 MB)
     |████████████████████████████████| 13.7 MB 443 kB/s            
Requirement already satisfied: setuptools in ./.local/venvs/base/lib/python3.9/site-packages (from awkward) (60.5.0)
Requirement already satisfied: numpy>=1.13.1 in ./.local/venvs/base/lib/python3.9/site-packages (from awkward) (1.22.0)
Installing collected packages: awkward
Successfully installed awkward-1.7.0
(base) feickert@raspberrypi:~ $ python -c 'import numpy as np; print(np.arange(10))'
[0 1 2 3 4 5 6 7 8 9]
(base) feickert@raspberrypi:~ $ python -c 'import awkward'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/feickert/.local/venvs/base/lib/python3.9/site-packages/awkward/__init__.py", line 23, in <module>
    import awkward.layout
  File "/home/feickert/.local/venvs/base/lib/python3.9/site-packages/awkward/layout.py", line 12, in <module>
    from awkward._ext import Index8
ImportError: /home/feickert/.local/venvs/base/lib/python3.9/site-packages/awkward/_ext.cpython-39-arm-linux-gnueabihf.so: undefined symbol: __atomic_fetch_add_8

@bennuttall, if you have time this week can you test if your build RPi is able to install the awkward wheels and then run the following? I appreciate that those wheels are probably deleted and that will require a rebuild, so apologies there.

import awkward as ak
array = ak.Array([
    [{"x": 1.1, "y": [1]}, {"x": 2.2, "y": [1, 2]}, {"x": 3.3, "y": [1, 2, 3]}],
    [],
    [{"x": 4.4, "y": [1, 2, 3, 4]}, {"x": 5.5, "y": [1, 2, 3, 4, 5]}]
])
print(array)  # [[{x: 1.1, y: [1]}, {x: 2.2, y: [1, 2]}, ... 4]}, {x: 5.5, y: [1, 2, 3, 4, 5]}]]

matthewfeickert commented 2 years ago

Ah, I see now that you've seen this before in Issue #59 and Issue #39 and @bennuttall's workaround in https://github.com/piwheels/packages/issues/59#issuecomment-611145391 does fix things here

(base) feickert@raspberrypi:~ $ LD_PRELOAD=/usr/lib/arm-linux-gnueabihf/libatomic.so.1 python
Python 3.9.2 (default, Mar 12 2021, 04:06:34) 
[GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import awkward as ak
>>> array = ak.Array([
...     [{"x": 1.1, "y": [1]}, {"x": 2.2, "y": [1, 2]}, {"x": 3.3, "y": [1, 2, 3]}],
...     [],
...     [{"x": 4.4, "y": [1, 2, 3, 4]}, {"x": 5.5, "y": [1, 2, 3, 4, 5]}]
... ])
>>> print(array)
[[{x: 1.1, y: [1]}, {x: 2.2, y: [1, 2]}, ... 4]}, {x: 5.5, y: [1, 2, 3, 4, 5]}]]

:+1:

Is this fixable in future releases?

piwheels / packages