conda-forge / code-aster-feedstock

A conda-smithy repository for code-aster.
BSD 3-Clause "New" or "Revised" License
1 stars 5 forks source link

Ongoing work on compiling code aster v16.4.X #16

Open Krande opened 1 year ago

Krande commented 1 year ago

Hey @ldallolio,

I have done some work on making a conda package for code aster v16.4.2 (with help from Mathieu Courtois and by studying your work and the docker work by aether engineering). You can check the progress here https://gitlab.com/codeaster/src/-/issues/1, The source code for all of the compilation work currently done on v16.4.2 can be found here

At the moment I've compiled and tested the v16.4.2 package for python 3.9, 3.10 and 3.11 and I am currently in the process of debugging the 5-10 % failing tests (150-200 of 2182 tests are failing due to various errors).

Once all (or a sufficient number of) the tests pass, my plan is to create a PR here and at the same time add the dependencies that currently aren't on conda-forge (such as mgis which I have discussed with Thomas Helfer).

I would really appreciate any thoughts you might have on things I should consider before making the PR, or if you have feedback/help you can provide on the current state of compiling code please let me know?

Best Regards Kristoffer

ldallolio commented 1 year ago

Hi Kristoffer,

First of all, thank you for your contribution to the community! I am glad that someone is working for Code_Aster in the conda-forge environment :) I will try to look into your work as soon as possible and give you all the feedback that I can

As you already found out, I spent some time recently with the packaging of code_aster in conda-forge (I was asked by a research center to do so as they use it for they research project) and I could contribute some packages :

https://anaconda.org/conda-forge/code-aster (github https://github.com/conda-forge/code-aster-feedstock) https://anaconda.org/conda-forge/mfront https://anaconda.org/conda-forge/miss3d https://anaconda.org/conda-forge/libmed (as well as others like freefem, getfem, ...)

I have currently different targets :

I do not know yet your current knowledge in building "conda-forge packages" : I am more than willing to provide all the help that I can and save you as much time as possible :-) Conda-forge is in my opinion a step forward (both in complexity and final result) with respect to creating a simple "conda" package, we can talk about that if this interests you. If you are willing, I can add you among the package maintainers so that you can contribute directly to those projects.

First advice (you may already know that, in this case sorry for pointing out) : you should work on a fork on your personal repository https://conda-forge.org/docs/maintainer/updating_pkgs.html#forking-and-pull-requests . Second advice : use mamba, not conda. Faster, much faster. Also much better dependency error reportings ;) Once installed, they are almost identical to use (you can simply use "mamba" instead of "conda"). But you are free to do as you prefer, of course. Third advice : build locally before committing, this will save you time, I can show you how if it helps you.

I have an hard time moving to a newer version of the python interpreter (I really need to have a >=3.10 version because of dependencies constraints) : it works perfectly using my local python but it doesn't with the one provided with conda forge (as I wrote, there are additional constraints in the forge to assure mutual compatibility among packages). I did isolate the issue in some python parts of Code_Aster which I cannot find in open source (I think that Matthieu coded those parts, I should ask him one of these days). I suspect that something is wrong with the code I had to put here in the package because I could not find anywhere else (and it is required ad build and at runtime): https://github.com/ldallolio/code-aster-feedstock/tree/main/recipe/contrib/asrun Code_Aster segfaults when building the command catalog, similar to this one : https://code-aster.org/forum2/viewtopic.php?id=1708

Please let me know I can help you, I will do my best :)

All the best, Luca

Il giorno mar 22 ago 2023 alle ore 08:08 Kristoffer Andersen < @.***> ha scritto:

Hey @ldallolio https://github.com/ldallolio,

I have done some work on making a conda package for code aster v16.4.2 (with help from Mathieu Courtois). You can check the progress here https://gitlab.com/codeaster/src/-/issues/1, The source code for all of the compilation work currently done on v16.4.2 can be found here https://github.com/Krande/condapackaging/tree/code-aster/16.4.2/src/code_aster

At the moment I've compiled and tested the v16.4.2 package for python 3.9, 3.10 and 3.11 and I am currently in the process of debugging the 5-10 % failing tests (150-200 of 2182 tests are failing due to various errors).

Once all (or a sufficient number of) the tests pass, my plan is to create a PR here and at the same time add the dependencies that currently aren't on conda-forge (such as mgis https://github.com/thelfer/MFrontGenericInterfaceSupport which I have discussed with Thomas Helfer).

I would really appreciate any thoughts you might have on things I should consider before making the PR, or if you have feedback/help you can provide on the current state of compiling code please let me know?

Best Regards Kristoffer

— Reply to this email directly, view it on GitHub https://github.com/conda-forge/code-aster-feedstock/issues/16, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAPAUDPXKWLOYNUNLQEXFXTXWREENANCNFSM6AAAAAA3ZMAFE4 . You are receiving this because you were mentioned.Message ID: @.***>

Krande commented 1 year ago

@ldallolio Great to hear! It seems our targets are aligned perfectly (next up on my list is also mpi and windows support of code aster) :)

I have also converted completely to the mamba-bandwagon :) And I've found boa for running fast conda-build iterations locally help significantly to speed up workflows (even though it requires a boa convert meta.yaml>recipe.yaml step)!

I also struggled with compiling the element catalogue catalo for any interpreters > 3.9 for code aster <=15.8.0. It was Mathieu who recommended me to focus my efforts on v16.4.* (which is/will be the new stable version apparently).

I do have some experience with maintaining packages on conda-forge already, so hopefully I'll be able to contribute. So if you add me as a maintainer on code-aster when you have time I'll happily try to contribute as best I can! And maybe you can add me as a maintainer on the dependencies (such as libmed, mfront) as well?

Btw. I have noticed that the latest (or at least among the latest) libmed packages on conda-forge does not contain any reference to the MEDlibraryNumVersion in libmed.so causing the code-aster compilation to fail. I am not sure why though (it might be there is something wrong in the code-aster waf)?

Either way, I added a test like this on my libmed packaging to ensure that the MEDlibraryNumVersion is found in the compiled file:

https://github.com/Krande/condapackaging/blob/code-aster/16.4.2/src/code_aster/libmed/meta.yaml#L44-L49

test:
  imports:
    - med
  commands:
    - |
      if [[ -z $(nm -D $CONDA_PREFIX/lib/libmed.so | grep MEDlibraryNumVersion) ]]; then
        echo "MEDlibraryNumVersion is not present in libmed.so";
        exit 1;
      fi

I will add a todo-list in my PR here of all the modifications I had to do in compiling the dependencies so that we can link them with PR's on the different feedstocks.

Hopefully by the end of this week I will have solved most of the failing tests and I will make a PR!

By the way, the 16.4.2 variant passed the mfron01a test:

execute: rm -rf /mnt/c/work/ctest_single
execute: ctest -R mfron01a -j 14 -L sequential
Test project /mnt/c/work/ctest_single
    Start 2879: ASTER_16.4.2_mfron01a
1/1 Test #2879: ASTER_16.4.2_mfron01a ............   Passed   11.75 sec

100% tests passed, 0 tests failed out of 1

Label Time Summary:
ASTER_16.4.2 SMECA_INTEGR nodes=01 sequential verification    =  11.75 sec*proc (1 test)

Total Test time (real) =  12.44 sec
ldallolio commented 1 year ago

Hi Kristoffer,

I am happy to hear that you have previous experience in conda-forge, things will be quicker and simpler this way, glad to have you onboard! I just added you as a maintainer in some projects (code-aster, libmed, mfront, miss3d), please let me know if I can do more.

Thanks for pointing out boa, did not know it, seems great!

No idea about the cause of "missing MEDlibraryNumVersion", we definitely need more tests there. It is great to know that the issue was fixed in later versions, this really helps.

I will try to have a look at things from here, please let me know if I can help in any other way, I will be happy to!

All the best, Luca

Il giorno mar 22 ago 2023 alle ore 12:18 Kristoffer Andersen < @.***> ha scritto:

@ldallolio https://github.com/ldallolio Great to hear! It seems our targets are aligned perfectly (next up on my list is also mpi and windows support of code aster) :)

I have also converted completely to the mamba-bandwagon :) And I've found boa for running fast conda-build iterations locally help significantly to speed up workflows (even though it requires a boa convert meta.yaml>recipe.yaml step)!

I also struggled with compiling the element catalogue catalo for any interpreters > 3.9 for code aster <15.8.0. It was Mathieu who recommended me to focus my efforts on v16.4.* (which is/will be the new stable version apparently).

I do have some experience with maintaining packages on conda-forge already, so hopefully I'll be able to contribute. So if you add me as a maintainer on code-aster when you have time I'll happily try to contribute as best I can! And maybe you can add me as a maintainer on the dependencies (such as libmed, mfront) as well?

Btw. I have noticed that the latest (or at least among the latest) libmed packages on conda-forge does not contain any reference to the MEDlibraryNumVersion in libmed.so causing the code-aster compilation to fail. I am not sure why though (it might be there is something wrong in the code-aster waf)?

Either way, I added a test like this on my libmed packaging to ensure that the MEDlibraryNumVersion is found in the compiled file:

https://github.com/Krande/condapackaging/blob/code-aster/16.4.2/src/code_aster/libmed/meta.yaml#L44-L49

test: imports:

  • med commands:
  • | if [[ -z $(nm -D $CONDA_PREFIX/lib/libmed.so | grep MEDlibraryNumVersion) ]]; then echo "MEDlibraryNumVersion is not present in libmed.so"; exit 1; fi

I will add a todo-list in my PR here of all the modifications I had to do in compiling the dependencies so that we can link them with PR's on the different feedstocks?

By the way, the 16.4.2 variant passed the mfron01a test:

[image: image] https://user-images.githubusercontent.com/9642232/262317407-6eb4a82a-048e-45ac-a82b-8d48e3d8868f.png

— Reply to this email directly, view it on GitHub https://github.com/conda-forge/code-aster-feedstock/issues/16#issuecomment-1687908495, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAPAUDJ26JR5XXMKOU7FLODXWSBORANCNFSM6AAAAAA3ZMAFE4 . You are receiving this because you were mentioned.Message ID: @.***>

Krande commented 11 months ago

Hey @ldallolio!

Here's an update. I have now also compiled an MPI-build in addition to the sequential build for linux.

Mathieu Courtois will attempt to compile Code Aster in a more updated Singularity container based on ubuntu 22.04 to help with debugging the remaining failed tests (see https://gitlab.com/codeaster/src/-/issues/1#note_1553448646). This will hopefully solve much of the remaining issues.

For now the test results are

Version python mpi Numpy hdf5 Failed Tests Total Tests
16.4.2 3.11 no 1.23 1.10.6 8 2182
16.4.2 3.11 no 1.25.2 1.10.6 11 2182
16.4.2 3.11 yes 1.23 1.10.6 256 2252
16.4.2 3.11 yes 1.25.2 1.10.6 259 2252

I am still waiting on conda-forge to accept my pull request for mgis (https://github.com/conda-forge/staged-recipes/pull/23818#issuecomment-1701187829). Once that gets accepted I will start to roll out PR's for the existing dependencies.

I haven't started looking at the work to be done for the windows compilation yet. But I think we'll be able to solve that in time as well :)

Best Regards Kristoffer

ldallolio commented 11 months ago

Hi Kristoffer,

Sorry about the late answer (I had some work going on at our old house in Italy, wood and wall repairs among other things), now I am back to France and I would like to speed up the process!

Thanks for your great news, a MPI version would be really helpful for a lot of people, I think. Most of the tests seem to pass, which is wonderful.

I see that your mgis request is still open, hope it will be accepted as soon as possible! Do you wish that we start working on a code_aster 16.4 upgrade ? I could start to integrate from your repository if you wish

Thank you once again, all the best

Luca

Il giorno sab 16 set 2023 alle ore 15:17 Kristoffer Andersen < @.***> ha scritto:

Hey @ldallolio https://github.com/ldallolio!

Here's an update. I have now also compiled an MPI-build in addition to the sequential build for linux.

Mathieu Courtois will attempt to compile Code Aster in a more updated Singularity container based on ubuntu 22.04 to help with debugging the remaining failed tests (see https://gitlab.com/codeaster/src/-/issues/1#note_1553448646). This will hopefully solve much of the remaining issues.

For now the test results are Version python mpi Numpy hdf5 Failed Tests Total Tests 16.4.2 3.11 no 1.23 1.10.6 8 2182 16.4.2 3.11 no 1.25.2 1.10.6 11 2182 16.4.2 3.11 yes 1.23 1.10.6 256 2252 16.4.2 3.11 yes 1.25.2 1.10.6 259 2252

I am still waiting on conda-forge to accept my pull request for mgis (conda-forge/staged-recipes#23818 (comment) https://github.com/conda-forge/staged-recipes/pull/23818#issuecomment-1701187829). Once that gets accepted I will start to roll out PR's for the remaining tests.

I haven't started looking at the work to be done for the windows compilation yet. But I think we'll be able to solve that in time as well :)

Best Regards Kristoffer

— Reply to this email directly, view it on GitHub https://github.com/conda-forge/code-aster-feedstock/issues/16#issuecomment-1722228580, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAPAUDOERHLSUDJ6RBQ2HVTX2WRH7ANCNFSM6AAAAAA3ZMAFE4 . You are receiving this because you were mentioned.Message ID: @.***>

Krande commented 11 months ago

No worries :) Yes, it would be really great if you can start the work on integrating what I've done so far into the official conda-forge feedstocks!

I have most certainly made mistakes and done strange things along the way in the different meta yaml's and build scripts. So feel free to ping me in the PR's whenever you come across anything you find that seems out of place :)

We should also decide whether or not we want to create a dedicated conda-forge package for medcoupling or if we want to embed it with the Code Aster package. Maybe it would be good to have a dedicated package? If so we should also start the process of adding that package to conda-forge as well. Besides medcoupling I think most of the other dependencies already exist.

I'll try to ping the conda-forge admins again to see if they can expedite the PR review for mgis

Oh, and by the way! I have made minor progress on the windows variant of Code_Aster. But I had to use a newer Mingw compiler version than currently is available on conda-forge to successfully compile HDF5. From what I can tell (and the feedback I got here there is an ongoing process to update the windows fortran compilers.

I got as far as getting working compilations of HDF5 and libmed. Medcoupling failed for some reason. But I digress. Let's fix the linux variants first! :)

Best Regards Kristoffer

ldallolio commented 11 months ago

Il giorno dom 8 ott 2023 alle ore 16:11 Kristoffer Andersen < @.***> ha scritto:

No worries :) Yes, it would be really great if you can start the work on integrating what I've done so far into the official conda-forge feedstocks!

I will do my best :)

I have most certainly made mistakes and done strange things along the way in the different meta yaml's and build scripts. So feel free to ping me in the PR's whenever you come across anything you find that seems out of place :)

I will certainly share any question with you, thanks for all you have done once again!

We should also decide whether or not we want to create a dedicated conda-forge package for medcoupling or if we want to embed it with the Code Aster package. Maybe it would be good to have a dedicated package? If so we should also start the process of adding that package to conda-forge as well. Besides medcoupling I think most of the other dependencies already exist.

I would go for a separate package : I have seen many projects using medcoupling only. Also, they have different licensing terms I think

I'll try to ping the conda-forge admins again to see if they can expedite the PR review for mgis

If I can do anything to help just ask but it also took some months to me to get the projets in conda_forge, thinks speed up after the project is in

Oh, and by the way! I have made minor progress on the windows variant of Code_Aster. But I had to use a newer Mingw compiler version than currently is available on conda-forge to successfully compile HDF5. From what I can tell (and the feedback I got here https://matrix.to/#/!SOyumkgPRWoXfQYIFH:matrix.org/$169590581923890qByPT:matrix.org?via=matrix.org&via=gitter.im&via=cadair.com there is an ongoing process to update the windows fortran compilers.

That's also great news! Those compilers are pretty old indeed.

I got as far as getting working compilations of HDF5 and libmed. Medcoupling failed for some reason. But I digress. Let's fix the linux variants first! :)

I agree, first things first :)

Best Regards Kristoffer

All the best, Luca

— Reply to this email directly, view it on GitHub https://github.com/conda-forge/code-aster-feedstock/issues/16#issuecomment-1752038169, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAPAUDOVZN3OA3QYEA6QXMTX6KYCNAVCNFSM6AAAAAA3ZMAFE6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONJSGAZTQMJWHE . You are receiving this because you were mentioned.Message ID: @.***>

ldallolio commented 10 months ago

Hi Kristoffer, A quick word to let you know that I integrated most of your work in a new version of libmed. I will go on with the other packages as soon as possible, Thank you once again, all the best, Luca

Krande commented 10 months ago

That's great!

Let me know if you run into any issues on the way and I'll answer as best I can,

Best Regards Kristoffer

Krande commented 10 months ago

Hey @ldallolio, just letting you know that I'll create a PR for a medcoupling to conda-forge now. Let me know if you have already started the process and I'll focus my efforts elsewhere.

ldallolio commented 10 months ago

Hi Kristoffer,

I am still working on merging your contributions to conda forge for code_aster so a medcoupling packager would be very welcome indeed! Thanks in advance, if I can help don't hesitate to ask!

Have a nice day, Luca

Il giorno lun 30 ott 2023 alle ore 09:58 Kristoffer Andersen < @.***> ha scritto:

Hey @ldallolio https://github.com/ldallolio, just letting you know that I'll create a PR for a medcoupling to conda-forge now. Let me know if you have already started the process and I'll focus my efforts elsewhere.

— Reply to this email directly, view it on GitHub https://github.com/conda-forge/code-aster-feedstock/issues/16#issuecomment-1784750039, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAPAUDJCUWQ3EEYKN3AX6WLYB5T3VAVCNFSM6AAAAAA3ZMAFE6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOBUG42TAMBTHE . You are receiving this because you were mentioned.Message ID: @.***>

Krande commented 10 months ago

Btw. The mgis package is now merged!

Do you want to be added as a maintainer?

ldallolio commented 10 months ago

Kudos !!!! (^__^)/

Il giorno lun 30 ott 2023 alle ore 10:24 Kristoffer Andersen < @.***> ha scritto:

Btw. The mgis https://github.com/conda-forge/mgis-feedstock package is now merged!

Do you want to be added as a maintainer?

— Reply to this email directly, view it on GitHub https://github.com/conda-forge/code-aster-feedstock/issues/16#issuecomment-1784792232, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAPAUDPRC5FNMKRDIJTV2A3YB5W4PAVCNFSM6AAAAAA3ZMAFE6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOBUG44TEMRTGI . You are receiving this because you were mentioned.Message ID: @.***>

Krande commented 10 months ago

Medcoupling PR up and running https://github.com/conda-forge/staged-recipes/pull/24378

ldallolio commented 9 months ago

Hi Kristoffer,

I could progress a bit towards a new code_aster 16 package these last few days, I prepared a not-yet-working-but-almost-there pull-request :

https://github.com/conda-forge/code-aster-feedstock/pull/22

I did something slightly different from your "condapackaging" solution to make it acceptable for conda-forge : metis and mumps are built and statically linked with code_aster Also, code_aster and medcoupling are installed into $PREFIX in a way that should work without activating environments. The package builds locally (using "./build-locally.py --debug") and tests are ok in the build environment.

I could also build a local package with "mamba build -c conda-forge ./recipe" and install it locally using "mamba create -n testca16 -c conda-forge -c local code-aster=16.4.9" and most tests are passing (excluding umat, which is not a big deal)

The main issue seems to be that "./build-locally", without --debug, fails (both locally and in the pull request checks https://dev.azure.com/conda-forge/84710dde-1620-425b-80d0-4cf5baca359d/_apis/build/builds/822354/logs/20 ). The issue is during IMPR_RESU, because of med and/or medcoupling not working properly :

"La bibliothèque "MED" n'est pas installée dans cette version de Code_Aster."

I think that the issue is some missing med and/or medcoupling runtime dependencies. Maybe you have already seen this ? I will keep looking into it next week but I wanted to share this. The only libraries that I cannot see at test time are astor and pybind11-stubgen (maybe they are needed by medcoupling at runtime ?)

I will try adding those and see what happens :)

Also, I tried linking statically med and hdf5 but failed, because code_aster needs medcoupling now and medcoupling does not link with a static med (even if recompiled with -fpic), I left some commented instructions in the build.sh if you want to have a look

All the best,

Luca

Il giorno lun 30 ott 2023 alle ore 12:19 Kristoffer Andersen < @.***> ha scritto:

Medcoupling PR up and running conda-forge/staged-recipes#24378 https://github.com/conda-forge/staged-recipes/pull/24378

— Reply to this email directly, view it on GitHub https://github.com/conda-forge/code-aster-feedstock/issues/16#issuecomment-1784976817, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAPAUDJO2DWJOHHBW55DUPTYB6ELHAVCNFSM6AAAAAA3ZMAFE6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOBUHE3TMOBRG4 . You are receiving this because you were mentioned.Message ID: @.***>

ldallolio commented 9 months ago

Hi Kristoffer,

I merged the pull request, since it will pass some tests at least with python 3.8 and 3.9 https://anaconda.org/conda-forge/code-aster/files Still not working (when MED tries to write files in IMPR_RESU) on python 3.10 and 3.11 I think I will also try to merge the libmed PR so that we have a new hdf, maybe it will help...

If you have any idea about this it would be very welcome

All the best, Luca

Il giorno dom 12 nov 2023 alle ore 16:56 Luca Dall'Olio < @.***> ha scritto:

Hi Kristoffer,

I could progress a bit towards a new code_aster 16 package these last few days, I prepared a not-yet-working-but-almost-there pull-request :

https://github.com/conda-forge/code-aster-feedstock/pull/22

I did something slightly different from your "condapackaging" solution to make it acceptable for conda-forge : metis and mumps are built and statically linked with code_aster Also, code_aster and medcoupling are installed into $PREFIX in a way that should work without activating environments. The package builds locally (using "./build-locally.py --debug") and tests are ok in the build environment.

I could also build a local package with "mamba build -c conda-forge ./recipe" and install it locally using "mamba create -n testca16 -c conda-forge -c local code-aster=16.4.9" and most tests are passing (excluding umat, which is not a big deal)

The main issue seems to be that "./build-locally", without --debug, fails (both locally and in the pull request checks https://dev.azure.com/conda-forge/84710dde-1620-425b-80d0-4cf5baca359d/_apis/build/builds/822354/logs/20 ). The issue is during IMPR_RESU, because of med and/or medcoupling not working properly :

"La bibliothèque "MED" n'est pas installée dans cette version de Code_Aster."

I think that the issue is some missing med and/or medcoupling runtime dependencies. Maybe you have already seen this ? I will keep looking into it next week but I wanted to share this. The only libraries that I cannot see at test time are astor and pybind11-stubgen (maybe they are needed by medcoupling at runtime ?)

I will try adding those and see what happens :)

Also, I tried linking statically med and hdf5 but failed, because code_aster needs medcoupling now and medcoupling does not link with a static med (even if recompiled with -fpic), I left some commented instructions in the build.sh if you want to have a look

All the best,

Luca

Il giorno lun 30 ott 2023 alle ore 12:19 Kristoffer Andersen < @.***> ha scritto:

Medcoupling PR up and running conda-forge/staged-recipes#24378 https://github.com/conda-forge/staged-recipes/pull/24378

— Reply to this email directly, view it on GitHub https://github.com/conda-forge/code-aster-feedstock/issues/16#issuecomment-1784976817, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAPAUDJO2DWJOHHBW55DUPTYB6ELHAVCNFSM6AAAAAA3ZMAFE6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOBUHE3TMOBRG4 . You are receiving this because you were mentioned.Message ID: @.***>

Krande commented 9 months ago

Great work! Feel free to merge the libmed PR!

I'll be back from vacation on Friday. Will take a look at the state of things then :)

Best Regards Kristoffer

Krande commented 5 months ago

Hey @ldallolio,

I thought I might share that I have gotten further on compiling Code Aster for windows.

There's still much to do, but now that most of the dependencies are compiled for windows on conda-forge I started the process of updating the WAF build for code aster to work with Windows MSVC.

You can follow the progress here: https://gitlab.com/krande/src/-/commits/win-support

I have tried to list the different steps required for a quick and easy local debugging of the waf compilation here: https://gitlab.com/krande/src/-/blob/win-support/conda/conda_win_compilation.md

Let me know if you have any thoughts on the matter :)

Krande commented 5 months ago

Another update ,@ldallolio.

I've managed to update the WAF build to pass the compilation all the individual c, c++ and fortran files and it has started on the linking phase of each library.

I am seeing several linking issues that I need to resolve. Everything seems to be related to the linker unable to find fortran subroutines.

For the linking phase and creation of bibc.dll I am seeing the following issues:

aster_core_module.c.1.o : error LNK2019: unresolved external symbol utgtme referenced in function asterc_get_mem_stat
aster_core_module.c.1.o : error LNK2019: unresolved external symbol utptme referenced in function asterc_set_mem_stat
aster_core_module.c.1.o : error LNK2019: unresolved external symbol asmpi_warn referenced in function aster_mpi_warn
aster_core_module.c.1.o : error LNK2019: unresolved external symbol _raiseException referenced in function aster_mpi_warn
aster_module.c.1.o : error LNK2001: unresolved external symbol _raiseException
aster_module.c.1.o : error LNK2019: unresolved external symbol fclose2 referenced in function aster_fclose
iodr.c.1.o : error LNK2001: unresolved external symbol fclose2
aster_module.c.1.o : error LNK2019: unresolved external symbol ulopen referenced in function aster_ulopen
aster_module.c.1.o : error LNK2019: unresolved external symbol affich referenced in function aster_affich
aster_module.c.1.o : error LNK2019: unresolved external symbol impers referenced in function aster_impers
aster_module.c.1.o : error LNK2019: unresolved external symbol mdnoma referenced in function aster_mdnoma
aster_module.c.1.o : error LNK2019: unresolved external symbol mdnoch referenced in function aster_mdnoch
aster_module.c.1.o : error LNK2019: unresolved external symbol rcvale_wrap referenced in function aster_rcvale
aster_module.c.1.o : error LNK2019: unresolved external symbol gmardm referenced in function aster_gmardm
aster_module.c.1.o : error LNK2019: unresolved external symbol postkutil referenced in function aster_postkutil
aster_module.c.1.o : error LNK2019: unresolved external symbol jemarq referenced in function aster_getvectjev
aster_module.c.1.o : error LNK2019: unresolved external symbol getcon referenced in function aster_getvectjev
aster_module.c.1.o : error LNK2019: unresolved external symbol jedema referenced in function aster_getvectjev
aster_module.c.1.o : error LNK2019: unresolved external symbol jedetr referenced in function aster_getvectjev
aster_module.c.1.o : error LNK2019: unresolved external symbol putcon referenced in function aster_putvectjev
aster_module.c.1.o : error LNK2019: unresolved external symbol tailsd referenced in function aster_getcolljev
aster_module.c.1.o : error LNK2019: unresolved external symbol rsacch referenced in function aster_GetResu
aster_module.c.1.o : error LNK2019: unresolved external symbol rsacpa referenced in function aster_GetResu
aster_module.c.1.o : error LNK2019: unresolved external symbol jelst3 referenced in function jeveux_getobjects
aster_module.c.1.o : error LNK2019: unresolved external symbol jelira referenced in function jeveux_getattr
aster_module.c.1.o : error LNK2019: unresolved external symbol jeexin referenced in function jeveux_exists
aster_module.c.1.o : error LNK2019: unresolved external symbol gcncon referenced in function aster_gcncon
aster_mpi.c.1.o : error LNK2019: unresolved external symbol abortf referenced in function asabrt
hancpu.c.1.o : error LNK2019: unresolved external symbol sigcpu referenced in function hancpu
hanfpe.c.1.o : error LNK2019: unresolved external symbol utmfpe referenced in function hanfpe
inisig.c.1.o : error LNK2019: unresolved external symbol sigsegv referenced in function inisig
inisig.c.1.o : error LNK2019: unresolved external symbol utmess_cwrap referenced in function stpusr1
iodr.c.1.o : error LNK2019: unresolved external symbol utmess_core referenced in function opendr
C:\work\code\krande-code-aster-src\build\debug\bibc\bibc.dll : fatal error LNK1120: 31 unresolved externals

Do you have any experience with this?

ldallolio commented 5 months ago

Hi @Krande I will try to look into this in detail asap, what you are missing are some Code_Aster internal functions (like those in bibfor/utilitai). I think that you might have a "name-mangling" issue, so the function name is compiled but the linker cannot find it because it is exposed under a slightly different name with respect to what it thinks, or it might just miss the object file, or again it might not have libraries in the right order. Under windows I think that you have other issues (.lib vs .dll vs .dll.lib) This won't help you much, sorry, I know, I usually solve those things by inspecting compiled files (using nm or dll inspectors), I will look into my notes to see if I can help! All the best,

Krande commented 5 months ago

@ldallolio Yeah, name mangling is what I am currently suspecting as well.

I have tried to force lowercase names for the fortran compiler using /names:lowercase, and when inspecting the symbols using dumpbin /symbols build\debug\bibfor\utilitai\utgtme.F90.1.o I see that the utgtme becomes lowercase (as expected). But it changed little, so I am suspecting there is more to it!

I am also investigating if it might be about build order. As of now it always tries to link and build bibc.dll first. So I'll see if I can force building bibfor.dll before building bibc.dll and see if it's possible to add bibfor.dll as a compile dep.

Any extra pair of eyes is more than welcome. I've been stuck on this for a few days now and haven't really progressed much :(

ldallolio commented 5 months ago

Hi @Krande I just followed your instructions (Installing Intel Fortran etc.) but I ran into an issue, maybe you know how to get over it : it does not seem to find numpy headers during waf configure (see part of my config.log below).

Weird thing is : it seems to be looking in the right place on my disk : `'C:\Users\lucad\mambaforge\envs\codeaster-deps\Lib\site-packages\numpy\core\include'

The include file is there too C:\Users\lucad\mambaforge\envs\codeaster-deps\Lib\site-packages\numpy\core\include\numpy\arrayobject.h

but I think that clang is not adding that directory to the include search path ?

Did you have a similar issue in your build ? Thanks!

`

Checking for numpy include ['C:\Users\lucad\mambaforge\envs\codeaster-deps\python.exe', '-c', 'import numpy as np\n\nprint(repr("\n".join([np.get_include()])))'] out: Intel MKL WARNING: Support of Intel(R) Advanced Vector Extensions (Intel(R) AVX) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library will use Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) instructions instead.

'C:\Users\lucad\mambaforge\envs\codeaster-deps\Lib\site-packages\numpy\core\include'

Intel MKL WARNING: Support of Intel(R) Advanced Vector Extensions (Intel(R) AVX) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library will use Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) instructions instead.

'C:\Users\lucad\mambaforge\envs\codeaster-deps\Lib\site-packages\numpy\core\include'

==>

include

include <numpy/arrayobject.h>

int main(int argc, char **argv) { (void)argc; (void)argv; return 0; }

<== [1/2] Compiling C:\Users\lucad\source\repos\krande\src\build\conf_check_8c2d582c718d2d03cf53e4c77ce449f7\test.cpp

['clang-cl.exe', '/nologo', '/nologo', '/I..\..\..\..\..\..\..\..\mambaforge\envs\codeaster-deps\include', '/I..\..\..\..\..\..\..\..\mambaforge\envs\codeaster-deps\libs', '/I..\..\..\..\..\..\..\..\..\..\Program Files (x86)\Intel\oneAPI\compiler\latest\include', '/DH5_BUILT_AS_DYNAMIC_LIB', '/DASTER_HAVE_OPENMP=1', '/DPYTHONDIR="C:\Users\lucad\mambaforge\envs\codeaster-deps\Lib\site-packages"', '/DPYTHONARCHDIR="C:\Users\lucad\mambaforge\envs\codeaster-deps\Library\Lib\site-packages"', '..\..\test.cpp', '/FC', '/c', '/FoC:\Users\lucad\source\repos\krande\src\build\conf_check_8c2d582c718d2d03cf53e4c77ce449f7\testbuild\release\test.cpp.1.o'] err: ....\test.cpp(2,10): fatal error: 'numpy/arrayobject.h' file not found

2 | #include <numpy/arrayobject.h>

  |          ^~~~~~~~~~~~~~~~~~~~~

1 error generated.

from c:\Users\lucad\source\repos\krande\src: Test does not build: Traceback (most recent call last): File "C:\Users\lucad\mambaforge\envs\codeaster-deps\Scripts\waf3-2.0.27-2b32dbfafe4d560837b4f7c5925b7eb9\waflib\Configure.py", line 335, in run_build bld.compile() File "C:\Users\lucad\mambaforge\envs\codeaster-deps\Scripts\waf3-2.0.27-2b32dbfafe4d560837b4f7c5925b7eb9\waflib\Build.py", line 176, in compile raise Errors.BuildError(self.producer.error) waflib.Errors.BuildError: Build failed -> task in 'testprog' failed with exit status 1 (run with -v to display more information)

Krande commented 5 months ago

Hey @ldallolio,

I made a few commits just now to fix some issues in the Linux compilation. I believe I managed to get the windows build working again, so you could check if you are using the latest commit I made? (or I might be wrong and the windows configure step has regressed). See https://gitlab.com/krande/src/-/commit/b82346b495904137553333c408d2efeabf1f83dc for the last known working commit. And you can also doublecheck the github action I've set up for testing the windows builds here -> https://github.com/Krande/condapackaging/actions/workflows/ci-code-aster-src-win.yml)

Additionally, could you do mamba list in the codeaster-deps env and paste the contents here? Then we can compare the package versions with what I have.

It also seems you are building a release version. I have only tested a debug version. So There might be some bugs in msvc win config that currently only works for the debug build.

ldallolio commented 5 months ago

Hi @Krande , I did update to the latest commit (same issue as before) and I also rolled back to the latest working commit you pointed me to (thanks!), same issue. I definitely think that this is a weird issue with my windows environement...

This is my mamba list result:

(base) C:\Users\lucad\source\repos\krande\src>mamba list

packages in environment at C:\Users\lucad\mambaforge:

#

Name Version Build Channel

beautifulsoup4 4.12.3 pyha770c72_0 conda-forge blinker 1.7.0 pyhd8ed1ab_0 conda-forge boltons 23.1.1 pyhd8ed1ab_0 conda-forge boolean.py 4.0 pyhd8ed1ab_0 conda-forge brotli-python 1.1.0 py310h00ffb61_1 conda-forge bzip2 1.0.8 hcfcfb64_5 conda-forge ca-certificates 2024.2.2 h56e8100_0 conda-forge certifi 2024.2.2 pyhd8ed1ab_0 conda-forge cffi 1.16.0 py310h8d17308_0 conda-forge chardet 5.2.0 py310h5588dad_1 conda-forge charset-normalizer 3.3.2 pyhd8ed1ab_0 conda-forge click 8.1.7 win_pyh7428d3b_0 conda-forge colorama 0.4.6 pyhd8ed1ab_0 conda-forge conda 23.3.1 py310h5588dad_0 conda-forge conda-build 3.25.0 py310h5588dad_0 conda-forge conda-forge-pinning 2024.03.29.12.36.21 hd8ed1ab_0 conda-forge conda-index 0.4.0 pyhd8ed1ab_0 conda-forge conda-package-handling 2.2.0 pyh38be061_0 conda-forge conda-package-streaming 0.9.0 pyhd8ed1ab_0 conda-forge conda-smithy 3.23.1 pyhd8ed1ab_0 conda-forge conda-verify 3.1.1 py310h5588dad_1006 conda-forge cryptography 42.0.5 py310h6e82f81_0 conda-forge deprecated 1.2.14 pyh1a96a4e_0 conda-forge filelock 3.13.3 pyhd8ed1ab_0 conda-forge fmt 9.1.0 h181d51b_0 conda-forge future 1.0.0 pyhd8ed1ab_0 conda-forge git 2.44.0 h57928b3_0 conda-forge gitdb 4.0.11 pyhd8ed1ab_0 conda-forge gitpython 3.1.42 pyhd8ed1ab_0 conda-forge glob2 0.7 py_0 conda-forge idna 3.6 pyhd8ed1ab_0 conda-forge isodate 0.6.1 pyhd8ed1ab_0 conda-forge jinja2 3.1.3 pyhd8ed1ab_0 conda-forge jsonpatch 1.33 pyhd8ed1ab_0 conda-forge jsonpointer 2.4 py310h5588dad_3 conda-forge krb5 1.20.1 heb0366b_0 conda-forge libarchive 3.6.2 h6f8411a_1 conda-forge libcurl 7.88.1 h68f0423_1 conda-forge libffi 3.4.2 h8ffe710_5 conda-forge libiconv 1.17 hcfcfb64_2 conda-forge liblief 0.14.1 he0c23c2_1 conda-forge libmamba 1.1.0 h8a7d157_3 conda-forge libmambapy 1.1.0 py310h3fe4c2e_3 conda-forge libsolv 0.7.28 h12be248_2 conda-forge libsqlite 3.45.2 hcfcfb64_0 conda-forge libssh2 1.11.0 h7dfc565_0 conda-forge libxml2 2.12.6 hc3477c8_1 conda-forge libzlib 1.2.13 hcfcfb64_5 conda-forge license-expression 30.1.1 pyhd8ed1ab_0 conda-forge lz4-c 1.9.4 hcfcfb64_0 conda-forge lzo 2.10 he774522_1000 conda-forge m2-msys2-runtime 2.5.0.17080.65c939c 3 conda-forge m2-patch 2.7.5 2 conda-forge m2w64-gcc-libgfortran 5.3.0 6 conda-forge m2w64-gcc-libs 5.3.0 7 conda-forge m2w64-gcc-libs-core 5.3.0 7 conda-forge m2w64-gmp 6.1.0 2 conda-forge m2w64-libwinpthread-git 5.0.0.4634.697f757 2 conda-forge mamba 1.1.0 py310hd9d798f_3 conda-forge markupsafe 2.1.5 py310h8d17308_0 conda-forge menuinst 1.4.19 py310h5588dad_1 conda-forge miniforge_console_shortcut 1.0 h57928b3_0 conda-forge more-itertools 10.2.0 pyhd8ed1ab_0 conda-forge msrest 0.6.21 pyh44b312d_0 conda-forge msys2-conda-epoch 20160418 1 conda-forge oauthlib 3.2.2 pyhd8ed1ab_0 conda-forge openssl 3.2.1 hcfcfb64_1 conda-forge packaging 24.0 pyhd8ed1ab_0 conda-forge pip 23.0 pyhd8ed1ab_0 conda-forge pkginfo 1.10.0 pyhd8ed1ab_0 conda-forge pluggy 1.4.0 pyhd8ed1ab_0 conda-forge psutil 5.9.8 py310h8d17308_0 conda-forge py-lief 0.14.1 py310h9e98ed7_1 conda-forge pybind11-abi 4 hd8ed1ab_3 conda-forge pycosat 0.6.6 py310h8d17308_0 conda-forge pycparser 2.22 pyhd8ed1ab_0 conda-forge pycryptodome 3.20.0 py310h8d17308_0 conda-forge pygithub 1.59.1 pyhd8ed1ab_0 conda-forge pyjwt 2.8.0 pyhd8ed1ab_1 conda-forge pynacl 1.5.0 py310h635b8f1_3 conda-forge pyopenssl 24.0.0 pyhd8ed1ab_0 conda-forge pysocks 1.7.1 pyh0701188_6 conda-forge python 3.10.9 h4de0772_0_cpython conda-forge python-libarchive-c 5.1 py310h5588dad_0 conda-forge python_abi 3.10 4_cp310 conda-forge pytz 2024.1 pyhd8ed1ab_0 conda-forge pyyaml 6.0.1 py310h8d17308_1 conda-forge reproc 14.2.4.post0 hcfcfb64_1 conda-forge reproc-cpp 14.2.4.post0 h63175ca_1 conda-forge requests 2.31.0 pyhd8ed1ab_0 conda-forge requests-oauthlib 2.0.0 pyhd8ed1ab_0 conda-forge ripgrep 14.1.0 h7f3b576_0 conda-forge ruamel.yaml 0.17.40 py310h8d17308_0 conda-forge ruamel.yaml.clib 0.2.8 py310h8d17308_0 conda-forge scrypt 0.8.24 py310h03188d8_0 conda-forge setuptools 65.6.3 pyhd8ed1ab_0 conda-forge six 1.16.0 pyh6c4a22f_0 conda-forge smmap 5.0.0 pyhd8ed1ab_0 conda-forge soupsieve 2.5 pyhd8ed1ab_1 conda-forge tk 8.6.13 h5226925_1 conda-forge tomli 2.0.1 pyhd8ed1ab_0 conda-forge toolz 0.12.1 pyhd8ed1ab_0 conda-forge tqdm 4.66.2 pyhd8ed1ab_0 conda-forge typing_extensions 4.10.0 pyha770c72_0 conda-forge tzdata 2024a h0c530f3_0 conda-forge ucrt 10.0.22621.0 h57928b3_0 conda-forge urllib3 2.2.1 pyhd8ed1ab_0 conda-forge vc 14.3 hcf57466_18 conda-forge vc14_runtime 14.38.33130 h82b7239_18 conda-forge vs2015_runtime 14.38.33130 hcb4865c_18 conda-forge vsts-python-api 0.1.25 pyhd8ed1ab_1 conda-forge wheel 0.43.0 pyhd8ed1ab_1 conda-forge win_inet_pton 1.1.0 pyhd8ed1ab_6 conda-forge wrapt 1.16.0 py310h8d17308_0 conda-forge xz 5.2.6 h8d14728_0 conda-forge yaml 0.2.5 h8ffe710_2 conda-forge yaml-cpp 0.7.0 h63175ca_3 conda-forge zstandard 0.22.0 py310h0009e47_0 conda-forge zstd 1.5.5 h12be248_0 conda-forge

I looked into the gitlab actions, there seem to be some errors here and there but it might be something irrelevant and different. My build seems to fail at "waf configure" time so I am unsure about what I should do to request for a debug version (I thought it would have been chosen later), do you have any hint ?

Another difference is that I use Visual Studio 2019 but here it seems to me that it is configure that is failing to find numpy headers (maybe a missing INCLUDE somewhere?)

Thanks for looking into this, all the best

Krande commented 5 months ago

That looks like your base mamba env. Can you do mamba activate codeaster-deps and then do mamba list?

(if you haven't already, the setup I created will need to activate the codeaster-deps env prior to running waf configure)

I'll continue working on windows support today. So hopefully i will be able to help solve your issue!

ldallolio commented 5 months ago

Hi @Krande ,

I just did it :-) here is the result on my pc :

(base) C:\Users\lucad>mamba activate codeaster-deps

(codeaster-deps) C:\Users\lucad>mamba list packages in environment at C:\Users\lucad\mambaforge\envs\codeaster-deps:

Name Version Build Channel bzip2 1.0.8 hcfcfb64_5 conda-forge ca-certificates 2024.2.2 h56e8100_0 conda-forge clang 18.1.2 default_hb53fc94_1 conda-forge clang-18 18.1.2 default_h3a3e6c3_1 conda-forge clangxx 18.1.2 default_h3a3e6c3_1 conda-forge hdf5 1.14.3 nompi_h73e8ff5_100 conda-forge intel-openmp 2024.0.0 h57928b3_49841 conda-forge krb5 1.21.2 heb0366b_0 conda-forge libaec 1.1.3 h63175ca_0 conda-forge libblas 3.9.0 21_win64_mkl conda-forge libboost 1.82.0 h65993cd_6 conda-forge libboost-python 1.84.0 py312h4f1204c_2 conda-forge libcblas 3.9.0 21_win64_mkl conda-forge libcurl 8.7.1 hd5e4a3a_0 conda-forge libexpat 2.6.2 h63175ca_0 conda-forge libffi 3.4.2 h8ffe710_5 conda-forge libflang 5.0.0 h6538335_20180525 conda-forge libhwloc 2.9.3 default_haede6df_1009 conda-forge libiconv 1.17 hcfcfb64_2 conda-forge liblapack 3.9.0 21_win64_mkl conda-forge libmed 4.1.1 py312h66fc33d_11 conda-forge libscotch 7.0.4 h534da82_1 conda-forge libsqlite 3.45.2 hcfcfb64_0 conda-forge libssh2 1.11.0 h7dfc565_0 conda-forge libxml2 2.12.6 hc3477c8_1 conda-forge libzlib 1.2.13 hcfcfb64_5 conda-forge lld 18.1.2 h40628bb_0 conda-forge llvm-meta 5.0.0 0 conda-forge medcoupling 9.10.0 py312hdf98862_1 conda-forge metis 5.1.0 h63175ca_1007 conda-forge mfront 4.2.0 py312hfaab970_2 conda-forge mgis 2.2 py312h20b4ba9_3 conda-forge mkl 2024.0.0 h66d3029_49657 conda-forge mkl-devel 2024.0.0 h57928b3_49657 conda-forge mkl-include 2024.0.0 h66d3029_49657 conda-forge mumps-seq 5.6.2 h1f49738_4 conda-forge numpy 1.26.4 py312h8753938_0 conda-forge openmp 5.0.0 vc14_1 conda-forge openssl 3.2.1 hcfcfb64_1 conda-forge pip 24.0 pyhd8ed1ab_0 conda-forge pthreads-win32 2.9.1 hfa6e2cd_3 conda-forge python 3.12.0 h2628c8c_0_cpython conda-forge python-dotenv 1.0.1 pyhd8ed1ab_0 conda-forge python_abi 3.12 4_cp312 conda-forge pyyaml 6.0.1 py312he70551f_1 conda-forge scotch 7.0.4 h0e60cbc_1 conda-forge setuptools 69.2.0 pyhd8ed1ab_0 conda-forge tbb 2021.11.0 h91493d7_1 conda-forge tk 8.6.13 h5226925_1 conda-forge tzdata 2024a h0c530f3_0 conda-forge ucrt 10.0.22621.0 h57928b3_0 conda-forge vc 14.3 hcf57466_18 conda-forge vc14_runtime 14.38.33130 h82b7239_18 conda-forge vs2015_runtime 14.38.33130 hcb4865c_18 conda-forge vswhere 3.1.4 h57928b3_0 conda-forge waf 2.0.27 py312he70551f_0 conda-forge wheel 0.43.0 pyhd8ed1ab_1 conda-forge xz 5.2.6 h8d14728_0 conda-forge yaml 0.2.5 h8ffe710_2 conda-forge zlib 1.2.13 hcfcfb64_5 conda-forge zstd 1.5.5 h12be248_0 conda-forge

Thanks for looking into this if you can!

Krande commented 5 months ago

Okay, I see only minor differences in our python env's. Main difference is that I moved to python 3.11 (as we don't compile Code Aster for python 3.12 on linux yet). But I don't think that's the issue you're seeing.

When comparing your original error message and the includes you are using, it seems you are missing an include path.

From my numpy include check

'clang-cl.exe',
 '/nologo',
 '/nologo',
 '/I..\\..\\..\\..\\..\\..\\miniforge3\\envs\\codeaster-deps\\Lib\\site-packages\\numpy\\core\\include',
 '/I..\\..\\..\\..\\..\\..\\miniforge3\\envs\\codeaster-deps\\include',
 '/I..\\..\\..\\..\\..\\..\\miniforge3\\envs\\codeaster-deps\\libs',

From yours

'clang-cl.exe',
 '/nologo',
 '/nologo',
 '/I..\..\..\..\..\..\..\..\mambaforge\envs\codeaster-deps\include',
 '/I..\..\..\..\..\..\..\..\mambaforge\envs\codeaster-deps\libs',

Btw. It should be safe to move to the latest commit https://gitlab.com/krande/src/-/commit/55cb1551bf7c04caa9818233050ed668ece1364e

Hopefully, moving to the latest commit fixes it (and doing a mamba env remove -n codeaster-deps and re-install the codeaster-deps). If not, you will find the code for resolving the include paths here waftools/python_cfg.py. I've added a log statement there "numpy_includes=[..]". That should list out the include paths it finds.

If I where to speculate as to why the include path is not added, it might be that the Intel MKL deprecation warning is causing an issue with the numpy include path export. However, it's strange that I don't have that same deprecation warning when using the same numpy version (although on a different python version).

Btw. How do you run waf configure? Are you using the "conda_build.bat"?

Krande commented 5 months ago

Btw. I've made a little progress. Or rather, I've landed on the conclusion that trying to force the compilation of bibfor.dll first is not solving the issue. I ended up missing symbols from bibc :)

It seems more likely that it's a name mangling issue and/or a dependency resolving issue (either in the current WAF setup or the way I'm using the linker LINK.exe).

I am going over the 30 unresolved symbols in the bibc.dll linking and it seems they all share the same definition setup.

Take the following example missing symbol utgtme from bibc/supervis/aster_core_module.c.

aster_core_module.c

utgtme

CALL_UTGTME( &nbpar, nompar, valres, &codret );

From bibc/include/aster_fort_utils.h

#define CALL_UTGTME( a, b, c, d ) CALLPSPP( UTGTME, utgtme, a, b, c, d )
extern void DEFPSPP( UTGTME, utgtme, ASTERINTEGER *, char *, STRING_SIZE, ASTERDOUBLE *,
                     ASTERINTEGER * );

Here is the definition of DEFPSPP and CALLPSPP

From bibc/include/definition.h

#ifdef ASTER_STRLEN_AT_END
...
#define DEFPSPP(UN,LN,a,b,lb,c,d)               STDCALL(UN,LN)(a,b,c,d,lb)
#define CALLPSPP(UN,LN,a,b,c,d)                 F_FUNC(UN,LN)(a,b,c,d,strlen(b))
...
/* Appels et signatures avec strlen juste après le pointeur de chaine */
#else
...
#define DEFPSPP(UN,LN,a,b,lb,c,d)               STDCALL(UN,LN)(a,b,lb,c,d)
#define CALLPSPP(UN,LN,a,b,c,d)                 F_FUNC(UN,LN)(a,b,strlen(b),c,d)
...
#endif

Common References (bibc/include/definition.h)

#ifndef ASTER_NO_UNDERSCORE
#define F_FUNC( UN, LN ) LN##_
#else
#define F_FUNC( UN, LN ) LN
#endif
/* STDCALL for "old" windows version */
#ifdef ASTER_ADD_STDCALL
#define STDCALL( UN, LN ) __stdcall F_FUNC( UN, LN )
#else
#define STDCALL( UN, LN ) F_FUNC( UN, LN )
#endif

It seems on linux, gcc handles this "behind the scenes". Ie. I can't see that all the *.o containing the fortran external symbols are passed to the linking of bibc.so. So I guess, that might be a key difference when going from GNU to MSVC?

Maybe the clue is to utilize the .mod files somehow. But I'm not quite seeing how to do that at the moment

Krande commented 4 months ago

@ldallolio Hey. Good news. I believe I know how to solve the windows build.

It seems to be a circular dependency issue. Ie. bibfor, bibc and bibcxx rely on symbols in eachother.

After reading this it appears circular dependencies like this is fine on UNIX since they rely on "linking resolution at load time". However MSVC needs the symbols to be resolved at link time. So after a bit more digging, I found this microsoft article which might offer a solution by creating .lib and .exp files directly from the list of object files (in our case the .o files) per module (ie. bibfor, bibc and bibcxx) which in turn can be added to the inputs for the creation of each dll.

So I need to create a new task which performs the ".lib+.exp generation step" that runs right before the first linking of the dll's. The inputs for the new task are the inputs (.o files) that the dll's currently depend on.

I tested this manually and it seems like it now manages to resolve the missing symbols. Now all I need to do is to make a conditional switch in the Code Aster WAF build for windows that adds this extra .lib generation step right before the linking phase.

I'll have to figure out a reasonable order for the dependencies, but I'm thinking something like this

  1. Compile source files for bibfor, bibc and bibcxx
  2. Generate .lib files for bibfor and bibcxx (these will need to have different names than bibfor.lib. Something like bibfor_gen.lib)
  3. Create bibc.dll (and bibc.lib) by including output from step 2 (ie. the bibfor_gen.lib and bibcxx_gen.lib files)
  4. Create bibfor.dll by including bibc.lib and bibcxx_gen,lib (from step 2).
  5. Create bibcxx.dll by including bibc.lib and bibfor.lib

Obviously then it's a matter of enabling fortran on all the dependencies (which I've begun slightly in parallel here) :)

AND of course it's the case that currently I have been compiling Code Aster using the intel fortran compiler ifx.exe which isn't available on conda-forge. So I'll try to see if I can get flang to compile Code Aster also (which would require a few of the dependencies (such as scotch and mumps) to be compiled using a newer version of flang (currently they're locked to flang=5. While the latest flang is 18.1.3).

ldallolio commented 4 months ago

Hi @Krande These are great news! I would have really, really liked to help you somewhere but frankly I would have never found something like this (I almost never use Windows), I was thinking about it this morning. This is a big step forward understanding the issue, in unix you can have "grouped dependencies" at link time with -Wl,--start-group and -Wl,--end-group (I sometimes use it when static linking). I wonder if static linking could also be a way to go, but I prefer your building order. Other that static linking, if nothing else seems to work, we could try cross compiling... Once again, thanks for your hard work! Have a nice day,

ldallolio commented 4 months ago

Hi @Krande , I updated (git pull puis mamba env update -f environment.yml) and rebuilt (conda_build.bat), I still fail at configure. I suspect that now the issue is somewhere in my Visual Studio installation (the exact error is unresolved external symbol main referenced in function "int __cdecl __scrt_common_main_seh(void)"

I cannot help so much in this phase, if I can do anything useful please let me know, I will do my best!

Have a nice day,

Krande commented 4 months ago

Hey @ldallolio,

I am assuming you have installed the default c++ development tools using the visual studio installer? image

For reference I am using VS2022 build tools and I have installed the individual MSVC c++ version components v14.38.

image

However, I don't recognize the error you're seeing. So I doubt that its due to a different MSVC c++ version. I believe that is the supported version by vswhere (which is a conda package for activating the necessary VS env variables). However, the conda_build.bat calls the vsvars.bat directly which should set these variables for us either way.

Can you share the full cmd terminal log from running the bat file? And maybe also how you call the batch file? (if you prefer you can send the logs to me directly if you don't want to share publicly).

Btw. I believe I have a functional WAF build which generates the necessary .lib files and correctly rearranges the build order.

Right now I have only a single unresolved reference abort_ in the bibfor_gen.lib (which I believe should be quite possible to fix).

I have also managed to compile HDF5 and LIBMED locally using the intel Fortran compiler on windows. Which means that all dependencies with fortran symbols on windows are possible to compile with MSVC abi compatibility. We are also getting help on fixing LLVM Flang to support HDF5 in https://github.com/HDFGroup/hdf5/issues/4419 which hopefully will let us compile all of these soon on conda-forge as well!

Krande commented 4 months ago

Hey @ldallolio, I believe I have fixed most of the issues with the Code Aster build itself.

The issue I'm trying to figure out right now, is why we're not exporting fortran symbols for libmed (I managed to compile libmed successfully with fortran enabled. However, the symbols aren't exported :( )

I have tried to use dumpbin /symbols medfwrap.lib, but there aren't any symbols in there (the file is only 176kB). Similarly for the created dll, when I do dumpbin /exports medfwrap.dll I cannot find any references to the fortran symbols I'm looking for :(

I have tried different cmake flags and changes at https://github.com/Krande/med-copy/tree/conda-dev

Let me know if you have any ideas?

Best Regards Kristoffer

ldallolio commented 4 months ago

Hi @krande,

It is indeed surprising... The only thing I could find was to add a /verbose to the link command, I suppose you already tried that one... Are you using the intel fortran compiler ? Or flang ?

Maybe adding ENABLE_EXPORTS dans le Cmakefile ? As you know, I know almost nothing on windows builds... :-( https://cmake.org/cmake/help/v3.5/prop_tgt/ENABLE_EXPORTS.html

Another silly idea : we could create a minimal example (an hello world fortran library), build it directly and/or with cmake and send it to the corresponding forum ? I can work on that if you think that it could help :)

All the best, Luca

Il giorno mar 23 apr 2024 alle ore 17:20 Kristoffer Andersen < @.***> ha scritto:

Hey @ldallolio https://github.com/ldallolio, I believe I have fixed most of the issues with the Code Aster build itself.

The issue I'm trying to figure out right now, is why we're not exporting fortran symbols for libmed (I managed to compile libmed successfully with fortran enabled. However, the symbols aren't exported :( )

I have tried to use dumpbin /symbols medfwrap.lib, but there aren't any symbols in there (the file is only 176kB). Similarly for the created dll, when I do dumpbin /exports medfwrap.dll I cannot find any references to the fortran symbols I'm looking for :(

I have tried different cmake flags and changes at https://github.com/Krande/med-copy/tree/conda-dev

Let me know if you have any ideas?

Best Regards Kristoffer

— Reply to this email directly, view it on GitHub https://github.com/conda-forge/code-aster-feedstock/issues/16#issuecomment-2072674370, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAPAUDNPANCBDU4ZD2O254DY6Z33ZAVCNFSM6AAAAAA3ZMAFE6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANZSGY3TIMZXGA . You are receiving this because you were mentioned.Message ID: @.***>

Krande commented 4 months ago

Are you using the intel fortran compiler ? Or flang ?

I am using the intel fortran compiler. (flang failed unfortunately).

Maybe adding ENABLE_EXPORTS dans le Cmakefile ? As you know, I know almost nothing on windows builds... :-( https://cmake.org/cmake/help/v3.5/prop_tgt/ENABLE_EXPORTS.html

Thanks for the tip! I've tried something similar WINDOWS_EXPORT_ALL_SYMBOLS=ON already. But I'll try that suggestion also!

    SET_TARGET_PROPERTIES(medfwrap PROPERTIES
      SOVERSION 11
      VERSION   11.1.1
      WINDOWS_EXPORT_ALL_SYMBOLS ON
      POSITION_INDEPENDENT_CODE ON
    )

Another silly idea : we could create a minimal example (an hello world fortran library), build it directly and/or with cmake and send it to the corresponding forum ?

Not a silly idea at all! That's likely our path of least resistance wrt solving it. I have made a minimal example for the circular dependency problem using waf at https://github.com/Krande/mixedLang. That helped a lot when I was debugging my Code Aster build.

I'll start tomorrow on adding a minimal cmake example with the missing fortran symbols in that repository.

I can work on that if you think that it could help :)

Sure, I'll take all the help I can get :) But maybe you would like to check out the minimal WAF build example to see if it makes it easier to debug the MSVC issue you're having with the Code Aster build?

Krande commented 4 months ago

Hey @ldallolio! Good news. I managed to track down the issue in libmed that resulted in no symbols being exported. It's a CMAKE bug, and it is described here: https://gitlab.kitware.com/cmake/cmake/-/issues/25598.

I made a fix in https://github.com/Krande/med-copy/commit/36bf272ed74ae4a7433e897fdf82dfebb981e294 where I explicitly define the symbols in a new file src/medfwrap_symbols.def.in. Then I manually pass these in using a target_link_options(medfwrap PRIVATE "/DEF:${DLLDEF}") for medfwrap.

I created locally a libmed conda package with the symbols using Intel Fortran compiler and https://github.com/Krande/libmed-feedstock/tree/win-fortran/recipe.

However, I still need to align the symbol names convention. In Code Aster using Intel Fortran I am using /names:lowercase /assume:underscore which I havent used in libmed (yet). I ran into some minor snags when trying to apply these now. But I'll try to fix that tomorrow!

Krande commented 4 months ago

Hey @ldallolio, just FYI: https://gitlab.com/codeaster/src/-/issues/1#note_1896122953

I think I have managed to solve the circular dependency issue and I am now building the dll's using WAF.

Now I've reached a pybind11 issue, which seems to be caused by pybind11 not being initialized properly. I am currently investigating this.

Best Regards Kristoffer

Krande commented 4 months ago

Good news @ldallolio! I believe we're finally seeing progress on Code Aster for MSVC Windows :)

Minor update: First attempt on MSVC -> 5% tests passed, 2082 tests failed out of 2203

(codeaster-deps) C:\Work\code\code-aster-src>run_ctest --resutest=temp/ctest -L submit -L sequential -LE need_data --timefactor=5.0 --only-failed-results -j 6
(codeaster-deps) C:\Work\code\code-aster-src>echo OFF
Active code page: 65001
only the results files of testcases in failure will be kept!
execute: rd /s /q C:\Work\code\code-aster-src\temp/ctest
Invalid switch - "ctest".
execute: ctest -j 6 -L sequential.*submit -LE need_data
Test project C:/Work/code/code-aster-src/temp/ctest
          Start  102: ASTER_17.0.99_comp008b
          Start  105: ASTER_17.0.99_comp008e
          Start  115: ASTER_17.0.99_comp008o
          Start  128: ASTER_17.0.99_comp010i
          Start  176: ASTER_17.0.99_cont005e
          Start  268: ASTER_17.0.99_forma01c
   1/2203 Test  #105: ASTER_17.0.99_comp008e ...........***Failed    2.51 sec
          Start  339: ASTER_17.0.99_hpla100k
   2/2203 Test  #268: ASTER_17.0.99_forma01c ...........***Failed    2.51 sec
          Start  361: ASTER_17.0.99_hplv106a
   3/2203 Test  #115: ASTER_17.0.99_comp008o ...........   Passed    2.58 sec
          Start  362: ASTER_17.0.99_hplv106b
   4/2203 Test  #102: ASTER_17.0.99_comp008b ...........   Passed    2.61 sec
          Start  363: ASTER_17.0.99_hplv106c
   5/2203 Test  #128: ASTER_17.0.99_comp010i ...........***Failed    2.98 sec
          Start  364: ASTER_17.0.99_hplv108a
   6/2203 Test  #176: ASTER_17.0.99_cont005e ...........***Failed    3.93 sec
          Start  365: ASTER_17.0.99_hplv108b
   7/2203 Test  #339: ASTER_17.0.99_hpla100k ...........***Failed    2.12 sec
          Start  366: ASTER_17.0.99_hplv108c
   8/2203 Test  #363: ASTER_17.0.99_hplv106c ...........***Failed    2.27 sec

Using

from code_aster import CA

CA.close()

Gives the following console output

C:\Work\miniforge3\envs\codeaster-deps\python.exe C:\Work\code\code-aster-src\conda\test_install.py --simple 
Ouverture en écriture du fichier ./vola.1

<INFO> Démarrage de l'exécution.

                       -- CODE_ASTER -- VERSION : DÉVELOPPEMENT (unstable) --                       
                               Version 17.0.99 modifiée le 08/05/2024                               
                                    révision n/a - branche 'n/a'                                    
                                   Copyright EDF R&D 1991 - 2024                                    

                              Exécution du : Wed May  8 12:49:36 2024                               
                                        Nom de la machine :                                         
                                            MyWinMachine                                            
                                        Architecture : 64bit                                        
                                     Type de processeur : AMD64                                     
                                      Système d'exploitation :                                      
                                     Windows-10-10.0.22631-SP0                                      
                                 Langue des messages : nb (cp1252)                                  
                                     Version de Python : 3.11.0                                     
                                     Version de NumPy : 1.23.5                                      
                                     Parallélisme MPI : inactif                                     
                                    Parallélisme OpenMP : actif                                     
                              Nombre de processus OpenMP utilisés : 1                               
                               Version de la librairie HDF5 : 1.14.4                                
                                Version de la librairie MED : 4.1.1                                 
                               Version de la librairie MFront : 4.2.0                               
                                  Librairie MUMPS : non disponible                                  
                                  Librairie PETSc : non disponible                                  
                               Version de la librairie SCOTCH : 7.0.4                               

restarting from a previous execution...
Valeur initiale du temps CPU maximum =   86400 secondes
  Valeur du temps CPU maximum passé aux
commandes =   77760 secondes
  Réserve CPU prévue = 8640 secondes

Ouverture en lecture du fichier ./glob.1

Ajustement de la taille maximale des bases à 2048.00 Go.

Nom de la base                          :  GLOBALE
     Créée avec la version                   : 
17.00.99
     Nombre d'enregistrements utilisés       :  152
     Nombre d'enregistrements maximum  
     :  2684354
     Nombre d'enregistrements par fichier    :  15728
     Longueur d'enregistrement
(octets)      :  819200
     Nombre d'identificateurs utilisés       :  126
     Taille maximum du
répertoire            :  2000
     Pourcentage d'utilisation du répertoire :  6 %

Ouverture en lecture du fichier ./glob.1

Ouverture en écriture du fichier ./vola.1

Fin de lecture (durée  0.000000  s.) 

                      Mémoire limite pour l'allocation dynamique : 4608.00 Mo                       
                              ajouté à l'initialisation du processus :                              
                                             249.94 Mo                                              
                               Limite cible du processus : 4857.94 Mo                               
                         Taille limite des fichiers d'échange : 2048.00 Go                          
TMP_DIR                  <class 'pathlib.WindowsPath'>
CONDA_PREFIX             <class 'pathlib.WindowsPath'>
ASTER_DIR                <class 'pathlib.WindowsPath'>
BIN_DIR                  <class 'pathlib.WindowsPath'>
# Mémoire (Mo) :   250.00 /   250.00 /   167.58 /   167.58 (VmPeak / VmSize / Optimum / Minimum)
# Fin commande #0001   user+syst:        0.08s (syst:        0.05s, elaps:        0.00s)
# ----------------------------------------------------------------------------------------------

.. _stg1_txt88
# ----------------------------------------------------------------------------------------------
# Commande #0002 de
C:\Work\miniforge3\envs\codeaster-deps\Library\lib\aster\code_aster\CodeCommands\fin.py, ligne
88
FIN(INFO_RESU='NON',
    RETASSAGE='NON')

Saving objects...
TMP_DIR                  <class 'pathlib.WindowsPath'>
CONDA_PREFIX             <class 'pathlib.WindowsPath'>
ASTER_DIR                <class 'pathlib.WindowsPath'>
BIN_DIR                  <class 'pathlib.WindowsPath'>

 ╔════════════════════════════════════════════════════════════════════════════════════════════════╗
 ║ <I> <CATAMESS_89>                                                                              ║
 ║                                                                                                ║
 ║ Liste des alarmes émises lors de l'exécution du calcul.                                        ║
 ║                                                                                                ║
 ║     Les alarmes que vous avez                                                                  ║
 ║ choisies d'ignorer sont précédées de (*).                                                      ║
 ║     Nombre d'occurrences pour chacune des alarmes :                                            ║
 ║                                                                                                ║
 ║           aucune alarme                                                                        ║
 ╚════════════════════════════════════════════════════════════════════════════════════════════════╝

 ---------------------------------------------------------------------------
 Concepts de la base: G
    Nom        Type                Taille (Mo)         Nombre      Nombre de
                                                     d'objets       segments
    TOTAL                                30.49             45            106

    &FOZERO                               0.00              2              2
    &&_NUM_C                              0.00              1              1
    &CATA                                30.49             42            103
 (*) sous-objets non accessible directement
 ---------------------------------------------------------------------------
Nom de la base                          :  GLOBALE
     Nombre d'enregistrements utilisés       : 
152
     Nombre d'enregistrements maximum        :  2684354
     Nombre d'enregistrements par
fichier    :  15728
     Longueur d'enregistrement (octets)      :  819200
     Nombre total d'accès
en lecture         :  133
     Volume des accès en lecture             :        103.91 Mo.

Nombre total d'accès en écriture        :  223
     Volume des accès en écriture            :       
174.22 Mo.
     Nombre d'identificateurs utilisés       :  126
     Taille maximum du répertoire    
       :  2000
     Pourcentage d'utilisation du répertoire :  6 %

Nom de la base                          :  VOLATILE
     Nombre d'enregistrements utilisés       : 
107
     Nombre d'enregistrements maximum        :  2684354
     Nombre d'enregistrements par
fichier    :  15728
     Longueur d'enregistrement (octets)      :  819200
     Nombre total d'accès
en lecture         :  0
     Volume des accès en lecture             :          0.00 Mo.
     Nombre
total d'accès en écriture        :  291
     Volume des accès en écriture            :        227.34
Mo.
     Nombre d'identificateurs utilisés       :  25
     Taille maximum du répertoire           
:  2000
     Pourcentage d'utilisation du répertoire :  1 %

<I> <FIN> ARRET NORMAL DANS "FIN" PAR APPEL A "JEFINI".

 <I> <FIN> MEMOIRE JEVEUX MINIMALE REQUISE POUR L'EXECUTION :                     171.31 Mo
 <I> <FIN> MEMOIRE JEVEUX OPTIMALE REQUISE POUR L'EXECUTION :                     176.30 Mo
 <I> <FIN> MAXIMUM DE MEMOIRE UTILISEE PAR LE PROCESSUS LORS DE L'EXECUTION :     259.79 Mo

 <I>       FERMETURE DES BASES EFFECTUEE

   STATISTIQUES CONCERNANT L'ALLOCATION DYNAMIQUE :
     TAILLE CUMULEE MAXIMUM            :                   176  Mo.
     TAILLE CUMULEE LIBEREE            :                    22  Mo.
     NOMBRE TOTAL D'ALLOCATIONS        :                   163
     NOMBRE TOTAL DE LIBERATIONS       :                   163
     APPELS AU MECANISME DE LIBERATION :                     0
     TAILLE MEMOIRE CUMULEE RECUPEREE  :                     0  Mo.
     VOLUME DES LECTURES               :                     0  Mo.
     VOLUME DES ECRITURES              :                     0  Mo.

   MEMOIRE JEVEUX MINIMALE REQUISE POUR L'EXECUTION :     171.31 Mo
     - IMPOSE DE NOMBREUX ACCES DISQUE
     - RALENTIT LA VITESSE D'EXECUTION
   MEMOIRE JEVEUX OPTIMALE REQUISE POUR L'EXECUTION :     176.30 Mo
     - LIMITE LES ACCES DISQUE
     - AMELIORE LA VITESSE D'EXECUTION
   MAXIMUM DE MEMOIRE UTILISEE PAR LE PROCESSUS     :     259.79 Mo
     - COMPREND LA MEMOIRE CONSOMMEE PAR  JEVEUX, 
       LE SUPERVISEUR PYTHON, LES LIBRAIRIES EXTERNES

 <I>       FIN D'EXECUTION LE : ME-08-MAI -2024 12:49:36
Signature of pickled file   : db2b79f16030d5f871eae12c2d8f92044854d1aa8ac23098e1530240f1c86612
Signature of info file      : 98586c4668893eb1a6a6cb880f4654029130feb0a04dd7d22af9f652be382c1d
Signature of Jeveux database: eee03bedc5ad4f78cc1aed30b0eaa9dd283abc612a539e68dffe44ce6117462b

 ********************************************************************************
 * COMMAND                  :       USER :     SYSTEM :   USER+SYS :    ELAPSED *
 ********************************************************************************
 * DEBUT                    :       0.08 :       0.05 :       0.12 :       0.00 *
 * FIN                      :       0.06 :       0.09 :       0.16 :       0.00 *
 *  . check syntax          :       0.00 :       0.00 :       0.00 :       0.00 *
 *  . fortran               :       0.14 :       0.14 :       0.28 :       0.00 *
 ********************************************************************************
 * TOTAL_JOB                :       0.14 :       0.14 :       0.28 :       0.00 *
 ********************************************************************************

# Mémoire (Mo) :   259.79 /    83.00 /   176.30 /   171.31 (VmPeak / VmSize / Optimum / Minimum)
# Fin commande #0002   user+syst:        0.06s (syst:        0.09s, elaps:        0.00s)
# ----------------------------------------------------------------------------------------------

Process finished with exit code 0
ldallolio commented 3 months ago

Hi @Krande,

Thanks for this great result and for sharing this news, I am really impressed!! In my opinion, tests might be randomly failing on windows because of "too strict expected result precision check", I have seen that in the past. Tests results are fixed in "non regression mode" under linux, where they tend to stay almost the same but under windows results might be slightly different. My advice is to have a look at the failure in a couple of cases : if it is simply a matter of a result change and not a big one, it means that everything is probably ok (and that you cannot do anything about it).

Once again, thanks for this important result, it will help a lot of people!!

All the best,

Il giorno mer 8 mag 2024 alle ore 12:55 Kristoffer Andersen < @.***> ha scritto:

Good news @ldallolio https://github.com/ldallolio! I believe we're finally seeing progress on Code Aster for MSVC Windows :)

(codeaster-deps) C:\Work\code\code-aster-src>run_ctest --resutest=temp/ctest -L submit -L sequential -LE need_data --timefactor=5.0 --only-failed-results -j 6 (codeaster-deps) C:\Work\code\code-aster-src>echo OFF Active code page: 65001 only the results files of testcases in failure will be kept! execute: rd /s /q C:\Work\code\code-aster-src\temp/ctest Invalid switch - "ctest". execute: ctest -j 6 -L sequential.*submit -LE need_data Test project C:/Work/code/code-aster-src/temp/ctest Start 102: ASTER_17.0.99_comp008b Start 105: ASTER_17.0.99_comp008e Start 115: ASTER_17.0.99_comp008o Start 128: ASTER_17.0.99_comp010i Start 176: ASTER_17.0.99_cont005e Start 268: ASTER_17.0.99_forma01c 1/2203 Test #105: ASTER_17.0.99_comp008e ...........Failed 2.51 sec Start 339: ASTER_17.0.99_hpla100k 2/2203 Test #268: ASTER_17.0.99_forma01c ...........Failed 2.51 sec Start 361: ASTER_17.0.99_hplv106a 3/2203 Test #115: ASTER_17.0.99_comp008o ........... Passed 2.58 sec Start 362: ASTER_17.0.99_hplv106b 4/2203 Test #102: ASTER_17.0.99_comp008b ........... Passed 2.61 sec Start 363: ASTER_17.0.99_hplv106c 5/2203 Test #128: ASTER_17.0.99_comp010i ...........Failed 2.98 sec Start 364: ASTER_17.0.99_hplv108a 6/2203 Test #176: ASTER_17.0.99_cont005e ...........Failed 3.93 sec Start 365: ASTER_17.0.99_hplv108b 7/2203 Test #339: ASTER_17.0.99_hpla100k ...........Failed 2.12 sec Start 366: ASTER_17.0.99_hplv108c 8/2203 Test #363: ASTER_17.0.99_hplv106c ...........Failed 2.27 sec

Using

from code_aster import CA

CA.close()

Gives the following console output

C:\Work\miniforge3\envs\codeaster-deps\python.exe C:\Work\code\code-aster-src\conda\test_install.py --simple Ouverture en écriture du fichier ./vola.1

Démarrage de l'exécution. -- CODE_ASTER -- VERSION : DÉVELOPPEMENT (unstable) -- Version 17.0.99 modifiée le 08/05/2024 révision n/a - branche 'n/a' Copyright EDF R&D 1991 - 2024 Exécution du : Wed May 8 12:49:36 2024 Nom de la machine : MyWinMachine Architecture : 64bit Type de processeur : AMD64 Système d'exploitation : Windows-10-10.0.22631-SP0 Langue des messages : nb (cp1252) Version de Python : 3.11.0 Version de NumPy : 1.23.5 Parallélisme MPI : inactif Parallélisme OpenMP : actif Nombre de processus OpenMP utilisés : 1 Version de la librairie HDF5 : 1.14.4 Version de la librairie MED : 4.1.1 Version de la librairie MFront : 4.2.0 Librairie MUMPS : non disponible Librairie PETSc : non disponible Version de la librairie SCOTCH : 7.0.4 restarting from a previous execution... Valeur initiale du temps CPU maximum = 86400 secondes Valeur du temps CPU maximum passé aux commandes = 77760 secondes Réserve CPU prévue = 8640 secondes Ouverture en lecture du fichier ./glob.1 Ajustement de la taille maximale des bases à 2048.00 Go. Nom de la base : GLOBALE Créée avec la version : 17.00.99 Nombre d'enregistrements utilisés : 152 Nombre d'enregistrements maximum : 2684354 Nombre d'enregistrements par fichier : 15728 Longueur d'enregistrement (octets) : 819200 Nombre d'identificateurs utilisés : 126 Taille maximum du répertoire : 2000 Pourcentage d'utilisation du répertoire : 6 % Ouverture en lecture du fichier ./glob.1 Ouverture en écriture du fichier ./vola.1 Fin de lecture (durée 0.000000 s.) Mémoire limite pour l'allocation dynamique : 4608.00 Mo ajouté à l'initialisation du processus : 249.94 Mo Limite cible du processus : 4857.94 Mo Taille limite des fichiers d'échange : 2048.00 Go TMP_DIR CONDA_PREFIX ASTER_DIR BIN_DIR # Mémoire (Mo) : 250.00 / 250.00 / 167.58 / 167.58 (VmPeak / VmSize / Optimum / Minimum) # Fin commande #0001 user+syst: 0.08s (syst: 0.05s, elaps: 0.00s) # ---------------------------------------------------------------------------------------------- .. _stg1_txt88 # ---------------------------------------------------------------------------------------------- # Commande #0002 de C:\Work\miniforge3\envs\codeaster-deps\Library\lib\aster\code_aster\CodeCommands\fin.py, ligne 88 FIN(INFO_RESU='NON', RETASSAGE='NON') Saving objects... TMP_DIR CONDA_PREFIX ASTER_DIR BIN_DIR ╔════════════════════════════════════════════════════════════════════════════════════════════════╗ ║ ║ ║ ║ ║ Liste des alarmes émises lors de l'exécution du calcul. ║ ║ ║ ║ Les alarmes que vous avez ║ ║ choisies d'ignorer sont précédées de (*). ║ ║ Nombre d'occurrences pour chacune des alarmes : ║ ║ ║ ║ aucune alarme ║ ╚════════════════════════════════════════════════════════════════════════════════════════════════╝ --------------------------------------------------------------------------- Concepts de la base: G Nom Type Taille (Mo) Nombre Nombre de d'objets segments TOTAL 30.49 45 106 &FOZERO 0.00 2 2 &&_NUM_C 0.00 1 1 &CATA 30.49 42 103 (*) sous-objets non accessible directement --------------------------------------------------------------------------- Nom de la base : GLOBALE Nombre d'enregistrements utilisés : 152 Nombre d'enregistrements maximum : 2684354 Nombre d'enregistrements par fichier : 15728 Longueur d'enregistrement (octets) : 819200 Nombre total d'accès en lecture : 133 Volume des accès en lecture : 103.91 Mo. Nombre total d'accès en écriture : 223 Volume des accès en écriture : 174.22 Mo. Nombre d'identificateurs utilisés : 126 Taille maximum du répertoire : 2000 Pourcentage d'utilisation du répertoire : 6 % Nom de la base : VOLATILE Nombre d'enregistrements utilisés : 107 Nombre d'enregistrements maximum : 2684354 Nombre d'enregistrements par fichier : 15728 Longueur d'enregistrement (octets) : 819200 Nombre total d'accès en lecture : 0 Volume des accès en lecture : 0.00 Mo. Nombre total d'accès en écriture : 291 Volume des accès en écriture : 227.34 Mo. Nombre d'identificateurs utilisés : 25 Taille maximum du répertoire : 2000 Pourcentage d'utilisation du répertoire : 1 % ARRET NORMAL DANS "FIN" PAR APPEL A "JEFINI". MEMOIRE JEVEUX MINIMALE REQUISE POUR L'EXECUTION : 171.31 Mo MEMOIRE JEVEUX OPTIMALE REQUISE POUR L'EXECUTION : 176.30 Mo MAXIMUM DE MEMOIRE UTILISEE PAR LE PROCESSUS LORS DE L'EXECUTION : 259.79 Mo FERMETURE DES BASES EFFECTUEE STATISTIQUES CONCERNANT L'ALLOCATION DYNAMIQUE : TAILLE CUMULEE MAXIMUM : 176 Mo. TAILLE CUMULEE LIBEREE : 22 Mo. NOMBRE TOTAL D'ALLOCATIONS : 163 NOMBRE TOTAL DE LIBERATIONS : 163 APPELS AU MECANISME DE LIBERATION : 0 TAILLE MEMOIRE CUMULEE RECUPEREE : 0 Mo. VOLUME DES LECTURES : 0 Mo. VOLUME DES ECRITURES : 0 Mo. MEMOIRE JEVEUX MINIMALE REQUISE POUR L'EXECUTION : 171.31 Mo - IMPOSE DE NOMBREUX ACCES DISQUE - RALENTIT LA VITESSE D'EXECUTION MEMOIRE JEVEUX OPTIMALE REQUISE POUR L'EXECUTION : 176.30 Mo - LIMITE LES ACCES DISQUE - AMELIORE LA VITESSE D'EXECUTION MAXIMUM DE MEMOIRE UTILISEE PAR LE PROCESSUS : 259.79 Mo - COMPREND LA MEMOIRE CONSOMMEE PAR JEVEUX, LE SUPERVISEUR PYTHON, LES LIBRAIRIES EXTERNES FIN D'EXECUTION LE : ME-08-MAI -2024 12:49:36 Signature of pickled file : db2b79f16030d5f871eae12c2d8f92044854d1aa8ac23098e1530240f1c86612 Signature of info file : 98586c4668893eb1a6a6cb880f4654029130feb0a04dd7d22af9f652be382c1d Signature of Jeveux database: eee03bedc5ad4f78cc1aed30b0eaa9dd283abc612a539e68dffe44ce6117462b ******************************************************************************** * COMMAND : USER : SYSTEM : USER+SYS : ELAPSED * ******************************************************************************** * DEBUT : 0.08 : 0.05 : 0.12 : 0.00 * * FIN : 0.06 : 0.09 : 0.16 : 0.00 * * . check syntax : 0.00 : 0.00 : 0.00 : 0.00 * * . fortran : 0.14 : 0.14 : 0.28 : 0.00 * ******************************************************************************** * TOTAL_JOB : 0.14 : 0.14 : 0.28 : 0.00 * ******************************************************************************** # Mémoire (Mo) : 259.79 / 83.00 / 176.30 / 171.31 (VmPeak / VmSize / Optimum / Minimum) # Fin commande #0002 user+syst: 0.06s (syst: 0.09s, elaps: 0.00s) # ---------------------------------------------------------------------------------------------- Process finished with exit code 0 — Reply to this email directly, view it on GitHub , or unsubscribe . You are receiving this because you were mentioned.Message ID: ***@***.***>
Krande commented 3 months ago

Thanks for the tip!

Once my debugging is done, there are still some remaining tasks we'll need to solve if we want to distribute code aster for Windows on conda forge;

  • Fix LLVM Flang support for HDF5 on windows (or add a cf-package of Intel OneAPI Fortran for windows)
  • Current build-setup on windows uses symlinking of bibcxx.dll -> libaster.pyd and bibc.dll -> (aster.pyd,aster_core.pyd,aster_fonctions.pyd & med_aster.pyd). This requires administrator privileges or windows development mode. So not sure if this is OK for conda-forge

Oh, and I had to use the PYBIND11_NO_ASSERT_GIL_HELD_INCREF_DECREF flag in order to successfully compile on windows. I don't think that's something we should need. So I'll likely have to investigate this a bit further

Krande commented 2 months ago

Hey, I've solved most of the core issues; I no longer need to run with PYBIND11_NO_ASSERT_GIL_HELD_INCREF_DECREF and I've fixed the symlinking bit..

The number of passing tests were at 82% earlier today, and I believe I know how to fix most of the remaining non-passing tests.

Btw. One of the bugs that took me some time to hunt down was finally found in libmed. I fixed it, and I can now finally write my results to RMED on MSVC (I suspect the issue might be related to the issue we saw in medpro ref. https://github.com/ldallolio/medpro/pull/1#issuecomment-1997454244)

Anyways, with the libmed issue fixed, I can finally visualize the results using Paraview. So here's a figure of an eigenfrequency analysis using the MSVC compiled version :)

image

ldallolio commented 2 months ago

Hi Krande,

You have a great result there! I am pretty sure that we will see a lot of downloads for this version as soon as it gets out :-) Nice picture!

Il giorno mer 3 lug 2024 alle ore 18:03 Kristoffer Andersen < @.***> ha scritto:

Hey, I've solved most of the core issues now. The number of passing tests were at 82% earlier today, and I believe I know how to fix most of the remaining non-passing tests.

Btw. One of the bugs that took me some time to hunt down was in libmed. I finally fixed it, and I can now finally write to RMED on MSVC (I suspect the issue might be related to the issue we saw in medpro ref. ldallolio/medpro#1 (comment) https://github.com/ldallolio/medpro/pull/1#issuecomment-1997454244)

Anyways, with the libmed issue fixed, I can finally visualize the results using Paraview. So here's a figure of an eigenfrequency analysis using the MSVC compiled version :)

image.png (view on web) https://github.com/conda-forge/code-aster-feedstock/assets/9642232/7f8d7951-fb00-468a-8fd9-26767e43f95f

— Reply to this email directly, view it on GitHub https://github.com/conda-forge/code-aster-feedstock/issues/16#issuecomment-2206692535, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAPAUDIEIPECFZ7WNWOCVYLZKQOD7AVCNFSM6AAAAAA3ZMAFE6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMBWGY4TENJTGU . You are receiving this because you were mentioned.Message ID: @.***>