Build for python 3.8 or 3.9

[caiofcm]

Hello, I had tried to build for python version 3.8 both using the docker files and in the host machine. However I am facing issues.

Could you please provide the steps for building? Can be with conda, system python in host or in docker. With the conda approach would be preferable.

Any guidance would be appreciated. Thank you.

[v1kko]

I have created a conda recipe which can build daetools for linux. : https://github.com/v1kko/staged-recipes/tree/daetools/recipes/daetools

[v1kko]

Windows and OSX are not supported because the Superlu (and idas) solver dependencies I believe are not easily installeable on those platforms via conda

[v1kko]

see also : https://github.com/conda-forge/staged-recipes/pull/14695

[caiofcm]

Hi @v1kko thank you sharing.

But I am still unable to install from the staged-recipes.

• The simple install command conda install daetools has to wait the conda forge repo to merge your PR, right?
• Meanwhile, how we can install daetools from your repository?

Thank you!

[v1kko]

There are 2 ways to compile daetools from this repository:

1. • Clone https://github.com:v1kko/staged-recipes.git, check out daetools branch
   • use the local build pipeline (see https://conda-forge.org/docs/maintainer/adding_pkgs.html#staging-test-locally) or just run CONFIG=linux64 ./.scripts/run_docker_build.sh in the staged-recipes directory
   • The resulting package should be in the build-artifacts folder if everything succeeds
2. • Clone this repository
   • use the docker pipelines as specified in .github/workflows:

     
     docker build -f ./dockerfiles/daetools-dependencies.docker -t daetools-dependencies .
     docker build -f ./dockerfiles/daetools-libs.docker -t daetools-libs .
     #name: Build py38
     docker build -f ./dockerfiles/daetools_python_bindings.docker \
       --build-arg PY_VERSION=3.8 -t daetools_py38 .
     #name: Extract Artifacts
     docker run -v `pwd`/daetools-package:/out daetools_py38 rsync -a \
       /daetools/trunk/daetools-package/  /out/

Rename the /out/ to the directory where you want the package if you want (it should be an absolute path because of docker)

[caiofcm]

Hello @v1kko . Thank you again for the instructions. I tried the two approaches, but had issue on both.

With the staged-recipes approaches it ended with the error:

Traceback (most recent call last):
  File "/home/conda/staged-recipes-copy/.ci_support/build_all.py", line 175, in <module>
    build_all(os.path.join(root_dir, "recipes"), args.arch)
  File "/home/conda/staged-recipes-copy/.ci_support/build_all.py", line 100, in build_all
    build_folders(recipes_dir, folders, arch, channel_urls)
  File "/home/conda/staged-recipes-copy/.ci_support/build_all.py", line 156, in build_folders
    conda_build.api.build([recipe], config=get_config(arch, channel_urls))
  File "/opt/conda/lib/python3.8/site-packages/conda_build/api.py", line 186, in build
    return build_tree(
  File "/opt/conda/lib/python3.8/site-packages/conda_build/build.py", line 3068, in build_tree
    packages_from_this = build(metadata, stats,
  File "/opt/conda/lib/python3.8/site-packages/conda_build/build.py", line 2351, in build
    newly_built_packages = bundlers[pkg_type](output_d, m, env, stats)
  File "/opt/conda/lib/python3.8/site-packages/conda_build/build.py", line 1644, in bundle_conda
    files = post_process_files(metadata, initial_files)
  File "/opt/conda/lib/python3.8/site-packages/conda_build/build.py", line 1490, in post_process_files
    post_build(m, new_files, build_python=python)
  File "/opt/conda/lib/python3.8/site-packages/conda_build/post.py", line 1317, in post_build
    check_overlinking(m, files, host_prefix)
  File "/opt/conda/lib/python3.8/site-packages/conda_build/post.py", line 1223, in check_overlinking
    return check_overlinking_impl(m.get_value('package/name'),
  File "/opt/conda/lib/python3.8/site-packages/conda_build/post.py", line 1206, in check_overlinking_impl
    raise OverLinkingError(overlinking_errors)
conda_build.exceptions.OverLinkingError: overlinking check failed 
['  ERROR (daetools,lib/python3.9/site-packages/daetools/solibs/Linux_x86_64/lib/libcdaeSuperLU_LASolver.so): lib/python3.8/site-packages/daetools/solibs/Linux_x86_64/lib/libcdaeConfig.so not found in any packages', '  ERROR (daetools,lib/python3.9/site-packages/daetools/solibs/Linux_x86_64/lib/libcdaeDataReporting.so): lib/python3.8/site-packages/daetools/solibs/Linux_x86_64/lib/libcdaeUnits.so not found in any packages', '  ERROR (daetools,lib/python3.9/site-packages/daetools/solibs/Linux_x86_64/lib/libcdaeDataReporting.so): lib/python3.8/site-packages/daetools/solibs/Linux_x86_64/lib/libcdaeConfig.so not found in any packages', '  ERROR (daetools,lib/python3.9/site-packages/daetools/solibs/Linux_x86_64/lib/libcdaeActivity.so): lib/python3.8/site-packages/daetools/solibs/Linux_x86_64/lib/libcdaeCore.so not found in any packages', '  ERROR (daetools,lib/python3.9/site-packages/daetools/solibs/Linux_x86_64/lib/libcdaeActivity.so): lib/python3.8/site-packages/daetools/solibs/Linux_x86_64/lib/libcdaeDataReporting.so not found in any packages', '  ERROR (daetools,lib/python3.9/site-packages/daetools/solibs/Linux_x86_64/lib/libcdaeActivity.so): lib/python3.8/site-packages/daetools/solibs/Linux_x86_64/lib/libcdaeIDAS_DAESolver.so not found in any packages', '  ERROR (daetools,lib/python3.9/site-packages/daetools/solibs/Linux_x86_64/lib/libcdaeActivity.so): lib/python3.8/site-packages/daetools/solibs/Linux_x86_64/lib/libcdaeUnits.so not found in any packages', '  ERROR (daetools,lib/python3.9/site-packages/daetools/solibs/Linux_x86_64/lib/libcdaeActivity.so): lib/python3.8/site-packages/daetools/solibs/Linux_x86_64/lib/libcdaeConfig.so not found in any packages', '  ERROR (daetools,lib/python3.9/site-packages/daetools/solibs/Linux_x86_64/lib/libcdaeIDAS_DAESolver.so): lib/python3.8/site-packages/daetools/solibs/Linux_x86_64/lib/libcdaeUnits.so not found in any packages', '

With the docker approach I had the following errors:

ln: failed to create symbolic link './OpenCS': File exists

this I could fix just inserting a -f in the respective RUN command:

RUN cd daetools/trunk/OpenCS/ && mkdir -p build/include && cd build/include && ln -s -f ../../OpenCS

But when I run the line:

docker build -f ./dockerfiles/daetools_python_bindings.docker \
          --build-arg PY_VERSION=3.8 -t daetools_py38 .

I get the error consering CoolProp:

[compile.sh](http://compile.sh/): 189: [[: not found
Project MESSAGE: Compiling on Linux_x86_64
ls: cannot access '/CoolProp': No such file or directory
WARNING: ${SP_DIR}/CoolProp/.path is not defined: install target not created

/bin/sh: 1: -c: not found
make: * [Makefile:243: release/cool_prop.o] Error 127

I appreciate any help on this regard. Thank you,

[v1kko]

I see that I broke the docker building process while trying to get the conda process to work.

I created a separate branch (just removing the changes for conda) called docker_pipeline. If you pull that and run the docker pipeline it should work

git clone --branch docker_pipeline https://github.com/v1kko/daetools.git
cd daetools
docker build -f ./dockerfiles/daetools-dependencies.docker -t daetools-dependencies .
docker build -f ./dockerfiles/daetools-libs.docker -t daetools-libs .
#name: Build py38
docker build -f ./dockerfiles/daetools_python_bindings.docker --build-arg PY_VERSION=3.8 -t daetools_py38 .
#name: Extract Artifacts
docker run -v `pwd`/daetools-package:/out daetools_py38 rsync -a  /daetools/trunk/daetools-package/  /out/

[caiofcm]

Thank you for the further instruction @v1kko .

Could you also give me guidance on how to fix the staged-recipes approach ?

Thank you!

[v1kko]

The staged-recipes approach works, but only seems to work when compiling for a single python version locally (apparently some information is leaked between the builds, this might be conda-forge local-build specific)

when the skip flags in recipes/daetools/meta.yml are changed to:

 skip: True  # [not linux]
 skip: True  # [not py==39]

it works for me for py39, and of course if you want py38 it is possible to change the py==39 part`

[caiofcm]

Thank you again @v1kko.

Sorry keeping bothering you with this, but I could not find any guide online for installing a package using the staged-recipes locally.

The skip modification worked for me and a build_artifacts folder was created.

From this point, how can I install daetools from build_artifacts such that I can run a tutorial using the activated conda environment?

Bests

[v1kko]

I have published the packages so they can be used from the internet https://conda.lipsum.eu

Daetools can be installed in a new conda environment as follows: conda create -n daetools -c https://conda.lipsum.eu -c conda-forge daetools python=3.9

Or if you want to use the local build_artifacts you can replace https://conda.lipsum.eu with file://<path_to_build_artifacts_folder>

[caiofcm]

Thank you.

I ran the command:

conda create -n daetools -c https://conda.lipsum.eu -c conda-forge daetools python=3.9

Then I activated the new environment:

conda activate <path>/temp

But when trying to run the tutorial case I had the error:

  File "/home/caio/Downloads/temp/lib/python3.9/site-packages/daetools/dae_simulator/simulator.py", line 218, in run
    raise RuntimeError("Cannot connect daeTCPIPDataReporter")

Cannot connect daeTCPIPDataReporter

or:

python tutorial1.py 
/home/caio/Downloads/temp/bin/python: No module named daetools.dae_plotter
Cannot connect TCPIPDataReporter: Connection refused

Thank you!

[v1kko]

This can be fixed by manually running daeplotter before running the tutorial

daeplotter &
python tutorial1.py

[caiofcm]

Thank you!