TileDB-Inc / conda-forge-nightly-controller

Centralized nightly CI builds for TileDB conda feedstocks
2 stars 2 forks source link

Nightly feedstock build failed #133

Closed github-actions[bot] closed 2 weeks ago

github-actions[bot] commented 2 weeks ago

Nightly feedstock build failure for tiledbfeedstock_CI at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=4&_a=summary

github-actions[bot] commented 2 weeks ago

Nightly feedstock build failure for TileDB-Py%20Feedstock%20Testing at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=5&_a=summary

kounelisagis commented 2 weeks ago

It is the known dump problem. We are waiting for the 2.26.0 to merge and fix.

jdblischak commented 2 weeks ago

It is the known dump problem. We are waiting for the 2.26.0 to merge and fix.

Agreed. The tiledb-py-feedstock failures are the known dump problem because the Monday night build tests for the use of deprecated APIs.

However, the tiledb-feedstock win-64 build also failed. I restarted it

-- Running vcpkg install - done
CMake Error at D:/bld/tiledb_1724722492478/_build_env/Library/share/cmake-3.30/Modules/CMakeDetermineCCompiler.cmake:49 (message):
  Could not find compiler set in environment variable CC:

  cl.exe.
-- Configuring incomplete, errors occurred!
Call Stack (most recent call first):
  CMakeLists.txt:115 (project)

CMake Error: CMAKE_C_COMPILER not set, after EnableLanguage
CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage
KiterLuc commented 2 weeks ago

@teo-tsirpanis any idea what's going on with the windows build here?

teo-tsirpanis commented 2 weeks ago

On first glance Could not find compiler set in environment variable CC: cl.exe. looks like something outside of our control. 🤔

jdblischak commented 2 weeks ago

On first glance Could not find compiler set in environment variable CC: cl.exe. looks like something outside of our control.

Agreed. Your comment jogged my memory. We've seen this before when the GitHub Actions image (also used by Azure) is updated. I'll investigate

xref: https://github.com/TileDB-Inc/conda-forge-nightly-controller/issues/94#issuecomment-2122870213, https://github.com/TileDB-Inc/conda-forge-nightly-controller/issues/104#issuecomment-2150424716

jdblischak commented 2 weeks ago

Strangely however, both the last passing build on Sunday night and the failing build from Monday night used image version 20240811.1.0.

jdblischak commented 2 weeks ago

I think I've figured it out. The last passing build used vs2022_win-64 19.40.33808-h753ab0a_2, and the failing builds used vs2022_win-64 19.41.34120-h72b6792_20. vs2022 19.41 has since been marked as broken (https://github.com/conda-forge/admin-requests/pull/1056), so it should pass with a restart now.

xref: https://github.com/conda-forge/vc-feedstock/pull/82

jdblischak commented 2 weeks ago

Ha. Can't win. The third attempt installed vs2022_win-64 19.40 (19.40.33808-h753ab0a_20), but then the image version was updated to 20240825.1.0, which updated Visual Studio from 17.10 to 17.11 (https://github.com/actions/runner-images/commit/e01292aaa1de7f18023b7d6ace9130316b288b5a, https://github.com/actions/runner-images/pull/10490)

jdblischak commented 2 weeks ago

Last night's tiledb-feedstock win-64 build passed because it used image version 20240825.1.0 (Visual Studio 17.11) and installed vs2022_win-64 19.41. This was last night at 9:30 pm ET.

However, exactly 12 hours later this morning at 9:30 am ET, two migration PRs to tiledb-feedstock failed because they used image version 20240811.1.0 (Visual Studio 17.10) and installed vs2022_win-64 19.41 (https://github.com/conda-forge/tiledb-feedstock/pull/338, https://github.com/conda-forge/tiledb-feedstock/pull/339)

github-actions[bot] commented 2 weeks ago

Nightly feedstock build failure for tiledbfeedstock_CI at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=4&_a=summary

github-actions[bot] commented 2 weeks ago

Nightly feedstock build failure for TileDB-Py%20Feedstock%20Testing at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=5&_a=summary

teo-tsirpanis commented 2 weeks ago

Same Windows error apparently.

jdblischak commented 2 weeks ago

Same Windows error apparently.

Yes. It is super frustrating. As long as Azure keeps randomly flip-flopping between images that use either Visual Studio 17.10 or 17.11, there isn't much we can do. For more details see https://github.com/conda-forge/vc-feedstock/pull/82#issuecomment-2315619084

github-actions[bot] commented 2 weeks ago

Nightly feedstock build failure for tiledbfeedstock_CI at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=4&_a=summary

github-actions[bot] commented 2 weeks ago

Nightly feedstock build failure for TileDB-Py%20Feedstock%20Testing at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=5&_a=summary

jdblischak commented 2 weeks ago

The win-64 tiledb-feedstock build continues to fail because Azure is still using the outdated image version 20240811

jdblischak commented 2 weeks ago

I applied a workaround (https://github.com/conda-forge/admin-requests/pull/1062#issuecomment-2321509273) to unblock the latest migration PRs (https://github.com/conda-forge/tiledb-feedstock/issues/340). This should also fix the nightly build, that is, until Azure transitions to Visual Studio 17.11 (https://github.com/conda-forge/tiledb-feedstock/issues/341)