CI: Linux aarch64/arm64

[ax3l]

Add native Linux aarch64/arm64 runners with CircleCI.

Adding CI here first before adding an entry to wheels for building Python (Pip) Wheels. Note: We already successfully build this architecture on conda-forge.

[ax3l]

Huh:

======================================================================
ERROR: testAttributes (API.APITest.APITest.testAttributes)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/circleci/linux-aarch64-wheels/test/python/unittest/API/APITest.py", line 381, in testAttributes
    self.attributeRoundTrip(ext)
  File "/home/circleci/linux-aarch64-wheels/test/python/unittest/API/APITest.py", line 267, in attributeRoundTrip
    self.assertEqual(bytes(series.get_attribute("pystring3")),
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: 'str' object cannot be interpreted as an integer

and in CLI.pipe.py:

    dest.set_attribute(key, attr, attr_type)
    run_pipe.run()
RuntimeError: Unable to cast Python instance of type <class 'str'> to C++ type '?' (#define PYBIND11_DETAILED_ERROR_MESSAGES or compile in debug mode for details)

[ax3l]

Time to fix our long-standing ODR issue...: https://github.com/openPMD/openPMD-api/pull/1521

Update: merged & rebased (good), but still the same error here.

[ax3l]

Let's see if #1547 helps...

[ax3l]

@franzpoeschel do you have an idea why the CLI.pipe.py test fails on aarch64?

It seems to have some issues matching a Python type to C++ for attributes in pybind11, I think:

    run_pipe.run()
    dest.set_attribute(key, attr, attr_type)
  File "/home/circleci/linux-aarch64-wheels/build/lib/python3.11/site-packages/openpmd_api/pipe/__main__.py", line 262, in __copy
RuntimeError: Unable to cast Python instance of type <class 'str'> to C++ type 'signed char'
    dest.set_attribute(key, attr, attr_type)
  File "/home/circleci/linux-aarch64-wheels/build/lib/python3.11/site-packages/openpmd_api/pipe/__main__.py", line 229, in run
    dest.set_attribute(key, attr, attr_type)
RuntimeError: Unable to cast Python instance of type <class 'str'> to C++ type 'signed char'
RuntimeError: Unable to cast Python instance of type <class 'str'> to C++ type 'signed char'
    self.__copy(inseries, outseries)
  File "/home/circleci/linux-aarch64-wheels/build/lib/python3.11/site-packages/openpmd_api/pipe/__main__.py", line 262, in __copy
    dest.set_attribute(key, attr, attr_type)
RuntimeError: Unable to cast Python instance of type <class 'str'> to C++ type 'signed char'
[~Series] An error occurred: [~Series] An error occurred: fileBased output can not be written with no iterations.
[~Series] An error occurred: fileBased output can not be written with no iterations.
[~Series] An error occurred: fileBased output can not be written with no iterations.
fileBased output can not be written with no iterations.

[ax3l]

Maybe missing here: https://github.com/openPMD/openPMD-api/blob/dev/src/binding/python/Attributable.cpp#L410-L448

[ax3l]

Looks like Unittest.py also has an issue related to strings in attributes:

======================================================================
ERROR: testAttributes (API.APITest.APITest.testAttributes)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/circleci/linux-aarch64-wheels/test/python/unittest/API/APITest.py", line 381, in testAttributes
    self.attributeRoundTrip(ext)
  File "/home/circleci/linux-aarch64-wheels/test/python/unittest/API/APITest.py", line 267, in attributeRoundTrip
    self.assertEqual(bytes(series.get_attribute("pystring3")),
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: 'str' object cannot be interpreted as an integer

write as bytes: https://github.com/openPMD/openPMD-api/blob/31e927b3fb90a828afd96e8357b359cda466cb44/test/python/unittest/API/APITest.py#L144

read: https://github.com/openPMD/openPMD-api/blob/31e927b3fb90a828afd96e8357b359cda466cb44/test/python/unittest/API/APITest.py#L267-L268

[ax3l]

Probably something is different on this platform with respect to fundamental char type defaults: https://pybind11.readthedocs.io/en/stable/advanced/cast/strings.html#passing-bytes-to-c

and we might lack an overload or so.

[ax3l]

https://github.com/openPMD/openPMD-api/blob/9ad33b07a9102abb2cd38ca1c7b524693c16de4e/src/binding/python/Attributable.cpp#L319-L344

[franzpoeschel]

For the error in openpmd-pipe, we use a special form of set_attribute that lets us explicitly specify the datatype:

dest.set_attribute(key, attr, attr_type)

This finally runs into this implementation where list and string types are handled separately, since the Numpy types in attribute_dtypes are the underlying types (i.e. char instead of string):

template <>
bool SetAttributeFromObject::call<char>(
    Attributable &attr, std::string const &key, py::object &obj)
{
    if (std::string(py::str(obj.get_type())) == "<class 'list'>")
    {
        using ListChar = std::vector<char>;
        using ListString = std::vector<std::string>;
        try
        {
            return attr.setAttribute<ListString>(key, obj.cast<ListString>());
        }
        catch (const py::cast_error &)
        {
            return attr.setAttribute<ListChar>(key, obj.cast<ListChar>());
        }
    }
    else if (std::string(py::str(obj.get_type())) == "<class 'str'>")
    {
        return attr.setAttribute<std::string>(key, obj.cast<std::string>());
    }
    else
    {
        return attr.setAttribute<char>(key, obj.cast<char>());
    }
}

It looks like on Arm64, <class 'str'> does not work to identify strings? Maybe there's a better way. The other error from APITest.py looks similar, but not the same.

[franzpoeschel]

I pushed something that might fix the openpmd-pipe problem. Aside from this PR, we should add a test that tests openpmd-pipe against the result of dtype_test, there were some broken char types.

[franzpoeschel]

This is the problem:

Setting attribute 'date' with type <class 'str'>' and requested type int8, i.e. openPMD type SCHAR
        'char' is unsigned on the platform

Char is represented as unsigned char, but Python reports signed char as type for strings. This might be since the datasets are the sample datasets written on another platform?

[franzpoeschel]

The openpmd-pipe thing seems to be fixed now. The remaining error is:

======================================================================
ERROR: testAttributes (API.APITest.APITest.testAttributes)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/circleci/linux-aarch64-wheels/test/python/unittest/API/APITest.py", line 381, in testAttributes
    self.attributeRoundTrip(ext)
  File "/home/circleci/linux-aarch64-wheels/test/python/unittest/API/APITest.py", line 267, in attributeRoundTrip
    self.assertEqual(bytes(series.get_attribute("pystring3")),
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: 'str' object cannot be interpreted as an integer

Reminder: This test on an AMD64 system:

>>> import openpmd_api as io                                                                                          
>>> s = io.Series("asdf.json", io.Access.create)                                                                      
>>> s.set_attribute("pystring3", b"howdy, again!")                                                                    
False                                                                                                                 
>>> s.close()     

Results in:

{
  "attributes": {
    "...": "...",
    "pystring3": {
      "datatype": "VEC_UCHAR",
      "value": [
        104,
        111,
        119,
        100,
        121,
        44,
        32,
        97,
        103,
        97,
        105,
        110,
        33
      ]
    },
   "...": "...",
  },
  "data": {}
}

Arm64 somehow seems to make a string out of this again. I've pushed a printf debugging commit.

[franzpoeschel]

The remaining error is HDF5-specific:

BACKEND: json
DEBUG: pystring3='[104, 111, 119, 100, 121, 44, 32, 97, 103, 97, 105, 110, 33]' of type '<class 'list'>'
openPMD series: unittest_py_API
openPMD standard: 1.1.0
openPMD extensions: 0

number of iterations: 0

number of meshes: 0

number of particle species: 0

Series.set_software_version is deprecated. Set the version with the second argument of Series.set_software
BACKEND: toml
DEBUG: pystring3='[104, 111, 119, 100, 121, 44, 32, 97, 103, 97, 105, 110, 33]' of type '<class 'list'>'
openPMD series: unittest_py_API
openPMD standard: 1.1.0
openPMD extensions: 0

number of iterations: 0

number of meshes: 0

number of particle species: 0

Series.set_software_version is deprecated. Set the version with the second argument of Series.set_software
BACKEND: bp
DEBUG: pystring3='[104, 111, 119, 100, 121, 44, 32, 97, 103, 97, 105, 110, 33]' of type '<class 'list'>'
openPMD series: unittest_py_API
openPMD standard: 1.1.0
openPMD extensions: 0

number of iterations: 0

number of meshes: 0

number of particle species: 0

Series.set_software_version is deprecated. Set the version with the second argument of Series.set_software
BACKEND: bp4
DEBUG: pystring3='[104, 111, 119, 100, 121, 44, 32, 97, 103, 97, 105, 110, 33]' of type '<class 'list'>'
openPMD series: unittest_py_API
openPMD standard: 1.1.0
openPMD extensions: 0

number of iterations: 0

number of meshes: 0

number of particle species: 0

Series.set_software_version is deprecated. Set the version with the second argument of Series.set_software
BACKEND: h5
DEBUG: pystring3='['h', 'o', 'w', 'd', 'y', ',', ' ', 'a', 'g', 'a', 'i', 'n', '!']' of type '<class 'list'>'

This is not really a surprise: HDF5 does not have an actual char type, so on a platform with unsigned chars, this will be returned as char and not as unsigned char.

TLDR: This is probably not an error in the openPMD-api, but just a hardcoded test that does not work on this platform.

[franzpoeschel]

New problem: series.set_attribute("pyint", 13) writes a char type. This bug is not introduced, by uncovered by this PR. Adding a workaround for now, proper fix should come along with a rework of set_attribute() type detection.

[franzpoeschel]

All issues fixed now. I'll push another commit that brings some further fixes, but might also bring regressions. If the CI shows regressions, I'll revert it.

[franzpoeschel]

The remaining failing tests are the appleclang14 tests that are currently failing everywhere. However one of them seems to be an Arm64 test, so we might want to wait for #1565 regardless (I think I've found a workaround there).

[franzpoeschel]

@ax3l now passing

[ax3l]

Thank you for the help :pray: :tada: