search

TileDB-Inc / tiledbsoma-feedstock

A conda-smithy repository for tiledbsoma.
BSD 3-Clause "New" or "Revised" License
3 stars 3 forks source link

Nightly feedstock build failed #153

Closed github-actions[bot] closed 1 month ago

github-actions[bot] commented 1 month ago

Nightly feedstock build failure for tiledbsoma-feedstock at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=43&_a=summary

jdblischak commented 1 month ago

All the builds failed last night. There is a solver-related problem with Python 3.12 and numpy. The linux-64 builds failed when trying to solve the test environment for Python 3.12. The osx-* builds failed during the initial metadata generation.

jdblischak commented 1 month ago

I see what happened. conda-smithy added a Python 3.12 build with numpy 1.26 (https://github.com/TileDB-Inc/tiledbsoma-feedstock/commit/fc3a467fb8317dade98dcc8162c0b30eca2044e6), but we limit the runtime upper bound to numpy 1.24

https://github.com/TileDB-Inc/tiledbsoma-feedstock/blob/c1dcea2715b0a26d9fc469309d0ce8e58fef3a9a/recipe/meta.yaml#L98-L99

jdblischak commented 1 month ago

I increased the numpy upper bound and pushed to "nightly-build" 🤞

https://github.com/TileDB-Inc/tiledbsoma-feedstock/commit/542b10167447113ad1c74578a1f3139ba9f707c0

jdblischak commented 1 month ago

I don't know if my idea will work because now the builds are failing when running version.py, presumably due to the recently merged https://github.com/single-cell-data/TileDB-SOMA/pull/2563. cc @ryan-williams 

  File "<string>", line 342, in <module>
  File "/home/conda/feedstock_root/build_artifacts/tiledbsoma_1715883755954/work/apis/python/version.py", line 229, in get_version
    version = get_git_version()
  File "/home/conda/feedstock_root/build_artifacts/tiledbsoma_1715883755954/work/apis/python/version.py", line 188, in get_git_version
    latest_tag = get_latest_remote_tag(tracked_remote)
  File "/home/conda/feedstock_root/build_artifacts/tiledbsoma_1715883755954/work/apis/python/version.py", line 124, in get_latest_remote_tag
    raise RuntimeError(f"No tags found in remote {remote}")
RuntimeError: No tags found in remote origin
jdblischak commented 1 month ago

Next I am trying an unshallow clone in https://github.com/TileDB-Inc/tiledbsoma-feedstock/commit/04a030bf2e3c93d9c9bc7b512a1c5d8388c1f895

jdblischak commented 1 month ago

Bumping the numpy upper bound fixed the linux-64 builds but not the osx-* builds

github-actions[bot] commented 1 month ago

Nightly feedstock build failure for tiledbsoma-feedstock at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=43&_a=summary

github-actions[bot] commented 1 month ago

Nightly feedstock build failure for tiledbsoma-feedstock at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=43&_a=summary

github-actions[bot] commented 1 month ago

Nightly feedstock build failure for tiledbsoma-feedstock at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=43&_a=summary

github-actions[bot] commented 1 month ago

Nightly feedstock build failure for tiledbsoma-feedstock at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=43&_a=summary

jdblischak commented 1 month ago

Quick update. For last night's failed builds, the osx-* failures are still the same conda solver error with Python 3.12 and numpy. However, the py312/numpy solver errors in the test section of the linux-64 builds, which I fixed in #154, are no longer the cause of the failure because the build doesn't make it to the test section.

The fmt10/spdlog1.13 job is failing to build libtiledbsoma. Below is the error (I replaced the long path to the conda host env with ${HOST_ENV}:

[ 65%] Building CXX object src/CMakeFiles/TILEDB_SOMA_OBJECTS.dir/utils/arrow_adapter.cc.o
In file included from ${HOST_ENV}/include/spdlog/fmt/fmt.h:32,
                 from ${HOST_ENV}/include/spdlog/common.h:50,
                 from ${HOST_ENV}/include/spdlog/spdlog.h:12,
                 from /home/conda/feedstock_root/build_artifacts/tiledbsoma_1716169644869/work/libtiledbsoma/src/utils/../utils/logger.h:54,
                 from /home/conda/feedstock_root/build_artifacts/tiledbsoma_1716169644869/work/libtiledbsoma/src/utils/arrow_adapter.cc:35:
${HOST_ENV}/include/fmt/core.h: In instantiation of 'constexpr fmt::v10::detail::value<Context> fmt::v10::detail::make_arg(T&) [with bool PACKED = true; Context = fmt::v10::basic_format_context<fmt::v10::appender, char>; T = nlohmann::basic_json<>; typename std::enable_if<PACKED, int>::type <anonymous> = 0]':
${HOST_ENV}/include/fmt/core.h:1842:51:   required from 'constexpr fmt::v10::format_arg_store<Context, Args>::format_arg_store(T& ...) [with T = {nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long int, long unsigned int, double, std::allocator, nlohmann::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> > >}; Context = fmt::v10::basic_format_context<fmt::v10::appender, char>; Args = {nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long int, long unsigned int, double, std::allocator, nlohmann::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> > >}]'
${HOST_ENV}/include/fmt/core.h:1860:18:   required from 'constexpr fmt::v10::format_arg_store<Context, typename std::remove_cv<typename std::remove_reference<T>::type>::type ...> fmt::v10::make_format_args(T& ...) [with Context = basic_format_context<appender, char>; T = {nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long int, long unsigned int, double, std::allocator, nlohmann::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> > >}]'
${HOST_ENV}/include/fmt/core.h:2835:44:   required from 'std::string fmt::v10::format(format_string<T ...>, T&& ...) [with T = {nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long int, long unsigned int, double, std::allocator, nlohmann::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> > >&}; std::string = std::__cxx11::basic_string<char>; format_string<T ...> = basic_format_string<char, nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long int, long unsigned int, double, std::allocator, nlohmann::adl_serializer, std::vector<unsigned char, std::allocator<unsigned char> > >&>]'
/home/conda/feedstock_root/build_artifacts/tiledbsoma_1716169644869/work/libtiledbsoma/src/utils/arrow_adapter.cc:342:24:   required from here
${HOST_ENV}/include/fmt/core.h:1600:63: error: 'fmt::v10::detail::type_is_unformattable_for<nlohmann::basic_json<>, char> _' has incomplete type
 1600 |     type_is_unformattable_for<T, typename Context::char_type> _;
      |                                                               ^
${HOST_ENV}/include/fmt/core.h:1604:7: error: static assertion failed: Cannot format an argument. To make type T formattable provide a formatter<T> specialization: https://fmt.dev/latest/api.html#udt
 1604 |       formattable,
      |       ^~~~~~~~~~~
${HOST_ENV}/include/fmt/core.h:1604:7: note: 'formattable' evaluates to false
[ 69%] Building CXX object src/CMakeFiles/TILEDB_SOMA_OBJECTS.dir/utils/logger.cc.o
make[5]: *** [src/CMakeFiles/TILEDB_SOMA_OBJECTS.dir/build.make:272: src/CMakeFiles/TILEDB_SOMA_OBJECTS.dir/utils/arrow_adapter.cc.o] Error 1
make[5]: *** Waiting for unfinished jobs....
make[4]: *** [CMakeFiles/Makefile2:100: src/CMakeFiles/TILEDB_SOMA_OBJECTS.dir/all] Error 2
make[3]: *** [Makefile:136: all] Error 2
make[2]: *** [CMakeFiles/libtiledbsoma.dir/build.make:86: libtiledbsoma-prefix/src/libtiledbsoma-stamp/libtiledbsoma-build] Error 2
make[1]: *** [CMakeFiles/Makefile2:85: CMakeFiles/libtiledbsoma.dir/all] Error 2
make: *** [Makefile:91: all] Error 2

In contrast, the fmt9/spdlog1.11 job was able to build libtiledbsoma and r-tiledbsoma, but then failed to build tiledbsoma-py due to the recent updates to version.py. Below is the error (${HOST_ENV} is the conda env with the host requirements installed and ${SRC_DIR} are the TileDB-SOMA source files):

Processing ${SRC_DIR}/apis/python
  Added file://${SRC_DIR}/apis/python to build tracker '/tmp/pip-build-tracker-wum_5kg0'
  Created temporary directory: /tmp/pip-modern-metadata-y9i08ehx
  Preparing metadata (pyproject.toml): started
  Running command Preparing metadata (pyproject.toml)
  Parsed tracked remote origin from branch origin/main
  Traceback (most recent call last):
    File "${HOST_ENV}/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
      main()
    File "${HOST_ENV}/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "${HOST_ENV}/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 149, in prepare_metadata_for_build_wheel
      return hook(metadata_directory, config_settings)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "${HOST_ENV}/lib/python3.12/site-packages/setuptools/build_meta.py", line 366, in prepare_metadata_for_build_wheel
      self.run_setup()
    File "${HOST_ENV}/lib/python3.12/site-packages/setuptools/build_meta.py", line 311, in run_setup
      exec(code, locals())
    File "<string>", line 342, in <module>
    File "${SRC_DIR}/apis/python/version.py", line 229, in get_version
      version = get_git_version()
                ^^^^^^^^^^^^^^^^^
    File "${SRC_DIR}/apis/python/version.py", line 188, in get_git_version
      latest_tag = get_latest_remote_tag(tracked_remote)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "${SRC_DIR}/apis/python/version.py", line 124, in get_latest_remote_tag
      raise RuntimeError(f"No tags found in remote {remote}")
  RuntimeError: No tags found in remote origin
  error: subprocess-exited-with-error
github-actions[bot] commented 1 month ago

Nightly feedstock build failure for tiledbsoma-feedstock at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=43&_a=summary

jdblischak commented 1 month ago

Latest update from last night's build:

@johnkerl I think the linux-64 fmt10 error is the most pressing concern. Was there any recent change to libtiledbsoma that could be the cause of this?

I am less concerned about the py312 solver error for the osx-* builds. While long-term we need to address this, we could unblock the next release by dropping the py312 build in conda_build_config.yaml

johnkerl commented 1 month ago

@johnkerl I think the linux-64 fmt10 error is the most pressing concern. Was there any recent change to libtiledbsoma that could be the cause of this?

@jdblischak AFAICT the issue is that at line https://github.com/single-cell-data/TileDB-SOMA/blob/eb1d0a769965564e89a42991ec1b42c635942c6a/libtiledbsoma/src/utils/arrow_adapter.cc#L343 we are getting -- within this build system --

/home/conda/feedstock_root/build_artifacts/tiledbsoma_1716256011562/work/libtiledbsoma/src/utils/arrow_adapter.cc:343:24/.../work/libtiledbsoma/src/utils/arrow_adapter.cc:343:24:   required from here
/.../include/fmt/core.h:1600:63: error: 'fmt::v10::detail::type_is_unformattable_for<nlohmann::basic_json<>, char> _' has incomplete type
 1600 |     type_is_unformattable_for<T, typename Context::char_type> _;
      |                                                               ^
/.../include/fmt/core.h:1604:7: error: static assertion failed: Cannot format an argument. To make type T formattable provide a formatter<T> specialization: https://fmt.dev/latest/api.html#udt
 1604 |       formattable,
      |       ^~~~~~~~~~~

Note that the value there has type json which is from https://github.com/single-cell-data/TileDB-SOMA/blob/eb1d0a769965564e89a42991ec1b42c635942c6a/libtiledbsoma/src/external/include/nlohmann/json.hpp and we can check with @nguyenv to see if she knows if:

jdblischak commented 1 month ago

AFAICT the issue is that at line

@johnkerl looks like our investigations led us to the same place. I just opened an Issue upstream with my findings https://github.com/single-cell-data/TileDB-SOMA/issues/2608

johnkerl commented 1 month ago

https://github.com/single-cell-data/TileDB-SOMA/pull/2609

jdblischak commented 1 month ago

I am less concerned about the py312 solver error for the osx-* builds. While long-term we need to address this, we could unblock the next release by dropping the py312 build in conda_build_config.yaml

I confirmed this is a valid short-term workaround. I dropped the py312 variant and both osx-* builds passed

https://github.com/jdblischak/tiledbsoma-feedstock/commit/81093a95ff76e9c1cdf16f8b6d71fdb7c66305a1 https://dev.azure.com/jdblischak/feedstock-builds/_build/results?buildId=813&view=results