search

conda-forge / cmdstan-feedstock

A conda-smithy repository for cmdstan.
BSD 3-Clause "New" or "Revised" License
1 stars 6 forks source link

Try mambabuild #13

Closed WardBrian closed 1 year ago

WardBrian commented 3 years ago

Checklist

This is purely for testing Windows mambabuild

conda-forge-linter commented 3 years ago

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

WardBrian commented 3 years ago

@conda-forge-admin, please rerender

wolfv commented 3 years ago

Hmm, your isseu seems 100% related to tbb though :) How does tbb work on Windows?

g++ -E -x c++ ../tbb_2020.3/src/tbb/win64-gcc-tbb-export.def -O2 -DUSE_WINTHREAD -D_WIN32_WINNT=0x0502 -DMINGW_HAS_SECURE_API=1 -D__MSVCRT_VERSION__=0x0700 -msse -mthreads -m64 -mrtm    -D__TBB_BUILD=1 -Wall -Wno-parentheses -Wno-uninitialized -Wno-non-virtual-dtor -Wno-unknown-warning-option -Wno-deprecated-copy -Wno-missing-attributes -Wno-class-memaccess -Wno-sized-deallocation   -DTBB_SUPPRESS_DEPRECATED_MESSAGES=1 -I../tbb_2020.3/src -I../tbb_2020.3/src/rml/include -I../tbb_2020.3/include > tbb.def
g++  -o tbb.dll concurrent_hash_map.o concurrent_queue.o concurrent_vector.o dynamic_link.o itt_notify.o cache_aligned_allocator.o pipeline.o queuing_mutex.o queuing_rw_mutex.o reader_writer_lock.o spin_rw_mutex.o x86_rtm_rw_mutex.o spin_mutex.o critical_section.o mutex.o recursive_mutex.o condition_variable.o tbb_thread.o concurrent_monitor.o semaphore.o private_server.o rml_tbb.o tbb_misc.o tbb_misc_ex.o task.o task_group_context.o governor.o market.o arena.o scheduler.o observer_proxy.o tbb_statistics.o tbb_main.o concurrent_vector_v2.o concurrent_queue_v2.o spin_rw_mutex_v2.o task_v2.o    -lpsapi -shared -m64 -Wl,-L,"D:/bld/cmdstan_1634909878275/_h_env/Library/bin/cmdstan/stan/lib/stan_math/lib/tbb" -Wl,-rpath,"D:/bld/cmdstan_1634909878275/_h_env/Library/bin/cmdstan/stan/lib/stan_math/lib/tbb"   -Wl,--version-script,tbb.def
task.o: duplicate section `.rdata$_ZTIN3tbb4taskE[_ZTIN3tbb4taskE]' has different size
arena.o: duplicate section `.rdata$_ZTIN3tbb4taskE[_ZTIN3tbb4taskE]' has different size
scheduler.o: duplicate section `.rdata$_ZTIN3tbb4taskE[_ZTIN3tbb4taskE]' has different size
mingw32-make[1]: Leaving directory 'D:/bld/cmdstan_1634909878275/_h_env/Library/bin/cmdstan/stan/lib/stan_math/lib/tbb'

NOTE: Please add D:/bld/cmdstan_1634909878275/_h_env/Library/bin/cmdstan/stan/lib/stan_math/lib/tbb to your PATH variable.
You may call

mingw32-make install-tbb

to automatically update your user configuration.
WardBrian commented 3 years ago

@wolfv That same warning appears in our passing build logs here: https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=396249&view=logs&j=a70f640f-cc53-5cd3-6cdc-236a1aa90802&t=f5d15007-a01c-5ad8-c9ce-4d519d3b275f

It's a bit of a false positive in this case. We actually copy the necessary TBB files into PREFIX\Library\bin to avoid the path resolution issue. This is indeed a bit hacky - on MacOS and Linux we use a conda-vendored TBB, but that is not possible on Windows due to our reliance on mingw-gcc (see https://github.com/conda-forge/tbb-feedstock/issues/91)

wolfv commented 3 years ago

hm, no clue. I compared the two resolutions and they appear to be identical:

mamba:

2021-10-22T13:39:04.8572433Z ## Package Plan ##
2021-10-22T13:39:04.8573038Z 
2021-10-22T13:39:04.8574162Z   environment location: D:\bld\cmdstan_1634909878275\_h_env
2021-10-22T13:39:04.8575022Z 
2021-10-22T13:39:04.8575507Z 
2021-10-22T13:39:04.8576437Z The following NEW packages will be INSTALLED:
2021-10-22T13:39:04.8577083Z 
2021-10-22T13:39:04.8578065Z     m2w64-gcc-libgfortran:   5.3.0-6              conda-forge
2021-10-22T13:39:04.8579178Z     m2w64-gcc-libs:          5.3.0-7              conda-forge
2021-10-22T13:39:04.8580344Z     m2w64-gcc-libs-core:     5.3.0-7              conda-forge
2021-10-22T13:39:04.8581661Z     m2w64-gmp:               6.1.0-2              conda-forge
2021-10-22T13:39:04.8582756Z     m2w64-libwinpthread-git: 5.0.0.4634.697f757-2 conda-forge
2021-10-22T13:39:04.8583882Z     msys2-conda-epoch:       20160418-1           conda-forge

2021-10-22T13:39:06.9821345Z The following NEW packages will be INSTALLED:
2021-10-22T13:39:06.9822468Z 
2021-10-22T13:39:06.9823051Z     m2-bash:                        4.3.042-5             conda-forge
2021-10-22T13:39:06.9824185Z     m2-coreutils:                   8.25-102              conda-forge
2021-10-22T13:39:06.9824890Z     m2-filesystem:                  2016.04-4             conda-forge
2021-10-22T13:39:06.9825935Z     m2-gcc-libs:                    5.3.0-4               conda-forge
2021-10-22T13:39:06.9826623Z     m2-gmp:                         6.1.0-3               conda-forge
2021-10-22T13:39:06.9827639Z     m2-libiconv:                    1.14-3                conda-forge
2021-10-22T13:39:06.9828318Z     m2-libintl:                     0.19.7-4              conda-forge
2021-10-22T13:39:06.9829401Z     m2-msys2-runtime:               2.5.0.17080.65c939c-3 conda-forge
2021-10-22T13:39:06.9830045Z     m2-sed:                         4.2.2-3               conda-forge
2021-10-22T13:39:06.9831207Z     m2w64-binutils:                 2.25.1-5              conda-forge
2021-10-22T13:39:06.9831865Z     m2w64-bzip2:                    1.0.6-6               conda-forge
2021-10-22T13:39:06.9832934Z     m2w64-crt-git:                  5.0.0.4636.2595836-2  conda-forge
2021-10-22T13:39:06.9833620Z     m2w64-gcc:                      5.3.0-6               conda-forge
2021-10-22T13:39:06.9834650Z     m2w64-gcc-ada:                  5.3.0-6               conda-forge
2021-10-22T13:39:06.9835300Z     m2w64-gcc-fortran:              5.3.0-6               conda-forge
2021-10-22T13:39:06.9836392Z     m2w64-gcc-libgfortran:          5.3.0-6               conda-forge
2021-10-22T13:39:06.9837067Z     m2w64-gcc-libs:                 5.3.0-7               conda-forge
2021-10-22T13:39:06.9838395Z     m2w64-gcc-libs-core:            5.3.0-7               conda-forge
2021-10-22T13:39:06.9839112Z     m2w64-gcc-objc:                 5.3.0-6               conda-forge
2021-10-22T13:39:06.9840220Z     m2w64-gmp:                      6.1.0-2               conda-forge
2021-10-22T13:39:06.9840950Z     m2w64-headers-git:              5.0.0.4636.c0ad18a-2  conda-forge
2021-10-22T13:39:06.9841976Z     m2w64-isl:                      0.16.1-2              conda-forge
2021-10-22T13:39:06.9842648Z     m2w64-libiconv:                 1.14-6                conda-forge
2021-10-22T13:39:06.9843792Z     m2w64-libmangle-git:            5.0.0.4509.2e5a9a2-2  conda-forge
2021-10-22T13:39:06.9844443Z     m2w64-libwinpthread-git:        5.0.0.4634.697f757-2  conda-forge
2021-10-22T13:39:06.9845545Z     m2w64-make:                     4.1.2351.a80a8b8-2    conda-forge
2021-10-22T13:39:06.9846192Z     m2w64-mpc:                      1.0.3-3               conda-forge
2021-10-22T13:39:06.9847257Z     m2w64-mpfr:                     3.1.4-4               conda-forge
2021-10-22T13:39:06.9848105Z     m2w64-pkg-config:               0.29.1-2              conda-forge
2021-10-22T13:39:06.9872501Z     m2w64-toolchain:                5.3.0-7               conda-forge
2021-10-22T13:39:06.9873300Z     m2w64-toolchain_win-64:         2.4.0-0               conda-forge
2021-10-22T13:39:06.9874371Z     m2w64-tools-git:                5.0.0.4592.90b8472-2  conda-forge
2021-10-22T13:39:06.9875049Z     m2w64-windows-default-manifest: 6.4-3                 conda-forge
2021-10-22T13:39:06.9876074Z     m2w64-winpthreads-git:          5.0.0.4634.697f757-2  conda-forge
2021-10-22T13:39:06.9876778Z     m2w64-zlib:                     1.2.8-10              conda-forge
2021-10-22T13:39:06.9877858Z     msys2-conda-epoch:              20160418-1            conda-forge

conda:

2021-10-22T13:36:00.8984450Z ## Package Plan ##
2021-10-22T13:36:00.8985591Z 
2021-10-22T13:36:00.8987015Z   environment location: D:\bld\cmdstan_1634909641114\_h_env
2021-10-22T13:36:00.8987584Z 
2021-10-22T13:36:00.8987999Z 
2021-10-22T13:36:00.8988690Z The following NEW packages will be INSTALLED:
2021-10-22T13:36:00.8989416Z 
2021-10-22T13:36:00.8990139Z     m2w64-gcc-libgfortran:   5.3.0-6              conda-forge
2021-10-22T13:36:00.8991150Z     m2w64-gcc-libs:          5.3.0-7              conda-forge
2021-10-22T13:36:00.8992117Z     m2w64-gcc-libs-core:     5.3.0-7              conda-forge
2021-10-22T13:36:00.8993082Z     m2w64-gmp:               6.1.0-2              conda-forge
2021-10-22T13:36:00.8994039Z     m2w64-libwinpthread-git: 5.0.0.4634.697f757-2 conda-forge
2021-10-22T13:36:00.8994945Z     msys2-conda-epoch:       20160418-1           conda-forge
2021-10-22T13:36:00.8995339Z 
2021-10-22T13:36:00.9765364Z Preparing transaction: ...working... done
2021-10-22T13:36:01.7821214Z Verifying transaction: ...working... done
2021-10-22T13:36:01.9995171Z Executing transaction: ...working... done
2021-10-22T13:36:13.1311062Z Collecting package metadata (repodata.json): ...working... done
2021-10-22T13:36:31.1380663Z Solving environment: ...working... done
2021-10-22T13:36:42.9220228Z Collecting package metadata (repodata.json): ...working... done
2021-10-22T13:36:44.7862499Z Solving environment: ...working... done
2021-10-22T13:36:45.0425827Z 
2021-10-22T13:36:45.0431102Z ## Package Plan ##
2021-10-22T13:36:45.0431572Z 
2021-10-22T13:36:45.0432605Z   environment location: D:\bld\cmdstan_1634909641114\_build_env
2021-10-22T13:36:45.0433151Z 
2021-10-22T13:36:45.0433540Z 
2021-10-22T13:36:45.0434229Z The following NEW packages will be INSTALLED:
2021-10-22T13:36:45.0434797Z 
2021-10-22T13:36:45.0435540Z     m2-bash:                        4.3.042-5             conda-forge
2021-10-22T13:36:45.0437281Z     m2-coreutils:                   8.25-102              conda-forge
2021-10-22T13:36:45.0440092Z     m2-filesystem:                  2016.04-4             conda-forge
2021-10-22T13:36:45.0441534Z     m2-gcc-libs:                    5.3.0-4               conda-forge
2021-10-22T13:36:45.0442604Z     m2-gmp:                         6.1.0-3               conda-forge
2021-10-22T13:36:45.0443585Z     m2-libiconv:                    1.14-3                conda-forge
2021-10-22T13:36:45.0444601Z     m2-libintl:                     0.19.7-4              conda-forge
2021-10-22T13:36:45.0446371Z     m2-msys2-runtime:               2.5.0.17080.65c939c-3 conda-forge
2021-10-22T13:36:45.0447962Z     m2-sed:                         4.2.2-3               conda-forge
2021-10-22T13:36:45.0449328Z     m2w64-binutils:                 2.25.1-5              conda-forge
2021-10-22T13:36:45.0451639Z     m2w64-bzip2:                    1.0.6-6               conda-forge
2021-10-22T13:36:45.0452819Z     m2w64-crt-git:                  5.0.0.4636.2595836-2  conda-forge
2021-10-22T13:36:45.0453824Z     m2w64-gcc:                      5.3.0-6               conda-forge
2021-10-22T13:36:45.0454751Z     m2w64-gcc-ada:                  5.3.0-6               conda-forge
2021-10-22T13:36:45.0455669Z     m2w64-gcc-fortran:              5.3.0-6               conda-forge
2021-10-22T13:36:45.0456638Z     m2w64-gcc-libgfortran:          5.3.0-6               conda-forge
2021-10-22T13:36:45.0458790Z     m2w64-gcc-libs:                 5.3.0-7               conda-forge
2021-10-22T13:36:45.0459894Z     m2w64-gcc-libs-core:            5.3.0-7               conda-forge
2021-10-22T13:36:45.0460863Z     m2w64-gcc-objc:                 5.3.0-6               conda-forge
2021-10-22T13:36:45.0461786Z     m2w64-gmp:                      6.1.0-2               conda-forge
2021-10-22T13:36:45.0462686Z     m2w64-headers-git:              5.0.0.4636.c0ad18a-2  conda-forge
2021-10-22T13:36:45.0463671Z     m2w64-isl:                      0.16.1-2              conda-forge
2021-10-22T13:36:45.0464532Z     m2w64-libiconv:                 1.14-6                conda-forge
2021-10-22T13:36:45.0465465Z     m2w64-libmangle-git:            5.0.0.4509.2e5a9a2-2  conda-forge
2021-10-22T13:36:45.0466402Z     m2w64-libwinpthread-git:        5.0.0.4634.697f757-2  conda-forge
2021-10-22T13:36:45.0467294Z     m2w64-make:                     4.1.2351.a80a8b8-2    conda-forge
2021-10-22T13:36:45.0468261Z     m2w64-mpc:                      1.0.3-3               conda-forge
2021-10-22T13:36:45.0469142Z     m2w64-mpfr:                     3.1.4-4               conda-forge
2021-10-22T13:36:45.0470217Z     m2w64-pkg-config:               0.29.1-2              conda-forge
2021-10-22T13:36:45.0471101Z     m2w64-toolchain:                5.3.0-7               conda-forge
2021-10-22T13:36:45.0472044Z     m2w64-toolchain_win-64:         2.4.0-0               conda-forge
2021-10-22T13:36:45.0472938Z     m2w64-tools-git:                5.0.0.4592.90b8472-2  conda-forge
2021-10-22T13:36:45.0473867Z     m2w64-windows-default-manifest: 6.4-3                 conda-forge
2021-10-22T13:36:45.0474790Z     m2w64-winpthreads-git:          5.0.0.4634.697f757-2  conda-forge
2021-10-22T13:36:45.0475709Z     m2w64-zlib:                     1.2.8-10              conda-forge
2021-10-22T13:36:45.0476635Z     msys2-conda-epoch:              20160418-1            conda-forge
2021-10-22T13:36:45.0477126Z
WardBrian commented 3 years ago

I agree. The failure actually seems to come during packaging, not the build, but I have no idea how to start debugging that.

WardBrian commented 2 years ago

@conda-forge-admin, please rerender

conda-forge-linter commented 2 years ago

Hi! This is the friendly automated conda-forge-linting service.

I was trying to look for recipes to lint for you, but it appears we have a merge conflict. Please try to merge or rebase with the base branch to resolve this conflict.

Please ping the 'conda-forge/core' team (using the @ notation in a comment) if you believe this is a bug.

conda-forge-linter commented 2 years ago

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

WardBrian commented 2 years ago

@conda-forge-admin, please rerender

WardBrian commented 2 years ago

Just confirming @wolfv that this still fails, and it fails during packaging not compilation.

maresb commented 2 years ago

@WardBrian, I think you meant to ping @wolfv

WardBrian commented 2 years ago

@conda-forge-admin, please rerender

WardBrian commented 2 years ago

@conda-forge-admin, please rerender

wolfv commented 2 years ago

Unfortunately I'm confident that the issue still persists (if nothing has changed in cmdstan). I've tried to debug it last week but it segfaults in a weird way in LIEF as far as I can tell. My first idea was an Abi incompatibility between lief and mamba (both use different versions of pybind11) but reconciliation with same pybind11 hasn't helped. Will need to continue digging!

WardBrian commented 2 years ago

Nothing major has changed (to my knowledge) but I wanted to keep this relatively up to date just in case. I appreciate you looking into it!

wolfv commented 2 years ago

Thanks for keep pushing :) it's an unfortunate hard-to-debug bug. But at least it's easy to reproduce e.g. by compiling cmake or this one here. I need to write a script that triggers this without having to compile for 30 minutes, though.

conda-forge-linter commented 2 years ago

Hi! This is the friendly automated conda-forge-linting service.

I was trying to look for recipes to lint for you, but it appears we have a merge conflict. Please try to merge or rebase with the base branch to resolve this conflict.

Please ping the 'conda-forge/core' team (using the @ notation in a comment) if you believe this is a bug.

conda-forge-linter commented 2 years ago

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

WardBrian commented 2 years ago

@conda-forge-admin, please rerender

conda-forge-linter commented 1 year ago

Hi! This is the friendly automated conda-forge-linting service.

I wanted to let you know that I linted all conda-recipes in your PR (recipe) and found some lint.

Here's what I've got...

For recipe:

conda-forge-linter commented 1 year ago

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

WardBrian commented 1 year ago

@conda-forge-admin, please rerender

conda-forge-webservices[bot] commented 1 year ago

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

WardBrian commented 1 year ago

@conda-forge-admin, please rerender

WardBrian commented 1 year ago

@conda-forge-admin, please rerender

wolfv commented 1 year ago

Does this mean, for whatever reason, the segfault is not happening anymore?! :)

WardBrian commented 1 year ago

@wolfv - yes, it seems like it has resolved itself!

WardBrian commented 1 year ago

@wolfv - would you like me to close https://github.com/mamba-org/boa/issues/197 or should I leave it for the other packages that may have issues?

Also, could you remind me if a PR like this needs to bump the build number?