An example of how to freeze conan and how make garbage my local conan cache

[thelfer]

Dear conan developers,

First of all, thank you for your wonderfull work.

I am a new conan user. I am trying to package my code.

For some unknown reason, the following conanfile.py freezes conan, uses 100% of my CPU and make garbage my local conan cache, only if you uncomment the comments. Otherwise, this file works as expected. The commented is are meant to define an option named python_version and use it to set the -DPython_ADDITIONAL_VERSION in cmake.

I am working on debian jessie, python 2.7 and use pip to install conan.

This is my workflow, once uncommented:

rm -fr ~/.conan/
conan remote add bincrafters https://api.bintray.com/conan/bincrafters/public-conan
conan install Boost.Python/1.65.1@bincrafters/stable
conan export thelfer/TFEL  
conan install TFEL/3.1-dev@thelfer/TFEL --build TFEL -o python_version=2.7

With the commented version, everything works well. With the uncommented version, conan freezes in the build step and takes 100% of my CPU. If I kill it, my .conan directory seems garbage and nothing seems working anymore.

Here is the conanfile.py file:

import os
from conans import ConanFile, CMake

class TFELConan(ConanFile):
    name = "TFEL"
    author = "Thomas Helfer (tfel-contact@cea.fr)"
    version = "3.1-dev"
    license = "GPL/CECILL-A"
    url = "http://tfel.sourceforge.net"
    description = """TFEL is a collaborative development of CEA and EDF.
    MFront is a code generator which translates a set of closely related 
    domain specific languages into plain C++ on top of the  TFEL library.
    Those languages cover three kinds of material knowledge:
    - material properties (for instance the Young modulus, 
      the thermal conductivity, etc.)
    - mechanical behaviours. Numerical performances of generated
      mechanical behaviours were given a particular attention.
      Various benchmarks show that MFront implementations are 
      competitive with native implementations available in the Cast3M,
      Code-Aster, Abaqus Standard and Cyrano3 solvers.
    - simple point-wise models, such as material swelling used in 
      fuel performance codes."""
    settings = "os", "compiler", "build_type", "arch"
    options = {"shared": [True, False]} #, "python_version" : "ANY"}
    requires = "Boost.Python/1.65.1@bincrafters/stable"
    default_options = "shared=True" #, "python_version=2.7"
    generators = "cmake"

    def source(self):
        self.run("git clone https://github.com/thelfer/tfel.git")

    def build(self):
        cmake = CMake(self)
        args  = ['-DCMAKE_INSTALL_PREFIX="%s"' % self.package_folder,
                 '-Denable-python-bindings=ON',
                 '-Dlocal-castem-header=ON',
                 '-Denable-aster=ON',
                 '-Denable-europlexus=ON',
                 '-Denable-abaqus=ON',
                 '-Denable-calculix=ON',
                 '-DTFEL_SVN_REVISION=0']
        # if 'python_version' in self.options :
        #     args.append('-DPython_ADDITIONAL_VERSION='+ \
        #                 str(self.options['"python_version']))
        self.run('cmake %s/tfel %s %s'
              % (self.conanfile_directory,
                 cmake.command_line,
                 ' '.join( args )))
        self.run("cmake --build . --target install %s" % cmake.build_config)

    def package(self):
        pass

    def package_info(self):
        self.cpp_info.libs = ["tfel"]
        self.env_info.PATH.append(os.path.join(self.package_folder, "bin"))
        if not self.settings.os == "Windows":
            self.env_info.LD_LIBRARY_PATH.append(os.path.join(self.package_folder, "lib"))
            self.env_info.PYTHONPATH.append(os.path.join(self.package_folder, "lib/python2.7/site-packages/"))

I hope that you have enough information to investigate the issue. I am willing to help, so do not hesitate to contact me.

Regards,

Thomas Helfer

[memsharded]

I am afraid we have another problem, not directly related, with the upgrade to conan 0.28 of bincrafters boost package:

TFEL/3.1-dev@user/testing: Folder: C:\Users\memsharded\.conan\data\TFEL\3.1-dev\user\testing\export
ERROR: Unable to load conanfile in C:\Users\memsharded\.conan\data\Boost.Python\1.65.1\bincrafters\stable\export\conanfile.py
  File "C:\Users\memsharded\.conan\data\Boost.Python\1.65.1\bincrafters\stable\export\conanfile.py", line 1, in <module>
    from conans import ConanFile, tools, os
ImportError: cannot import name os

cc / @solvingj

[memsharded]

Are you working in conan 0.28? Why didn't you get such an error? While we try to fix this, could you please try to reduce to a simpler scenario that also hangs? Maybe without the Boost dependency still hangs? Also have you tried in other OS and it doesn't hang? Or you just tried in debian? Can you try to spot a specific command that is locking in the build() method? what is the output until it hangs? Many thanks for your help!

[thelfer]

Thanks for this quick reply.

How can I known conan version ? (I did pip install conan two days ago, I though I had the latest version)

It seems that the dependency to bincrafters is not mandatory for the problem to occur.

Best, Thomas

[thelfer]

The output, up to the moment I kill the process

$ conan install TFEL/3.1-dev@thelfer/TFEL --build TFEL -o python_version=2.7
TFEL/3.1-dev@thelfer/TFEL: Installing package
Requirements
WARN: Remotes registry file missing, creating default one in /home/th202608/.conan/registry.txt
    TFEL/3.1-dev@thelfer/TFEL from local
Packages
    TFEL/3.1-dev@thelfer/TFEL:c578773582977d03bc280486023a0fd13c0dd324

TFEL/3.1-dev@thelfer/TFEL: WARN: Forced build from source
TFEL/3.1-dev@thelfer/TFEL: Building your package in /home/th202608/.conan/data/TFEL/3.1-dev/thelfer/TFEL/build/c578773582977d03bc280486023a0fd13c0dd324
TFEL/3.1-dev@thelfer/TFEL: Configuring sources in /home/th202608/.conan/data/TFEL/3.1-dev/thelfer/TFEL/source
Clonage dans 'tfel'...
remote: Counting objects: 50643, done.
remote: Compressing objects: 100% (66/66), done.
remote: Total 50643 (delta 9), reused 19 (delta 5), pack-reused 50572
Réception d'objets: 100% (50643/50643), 83.42 MiB | 950.00 KiB/s, fait.
Résolution des deltas: 100% (43107/43107), fait.
Vérification de la connectivité... fait.
TFEL/3.1-dev@thelfer/TFEL: Copying sources to build folder
TFEL/3.1-dev@thelfer/TFEL: Generator cmake created conanbuildinfo.cmake
TFEL/3.1-dev@thelfer/TFEL: Calling build()

[thelfer]

Seems like I have conan 0.28 installed:

cat ~/.local/bin/conan_build_info 
#!/usr/bin/python
# EASY-INSTALL-ENTRY-SCRIPT: 'conan==0.28.0','console_scripts','conan_build_info'
__requires__ = 'conan==0.28.0'
import sys
from pkg_resources import load_entry_point

if __name__ == '__main__':
    sys.exit(
        load_entry_point('conan==0.28.0', 'console_scripts', 'conan_build_info')()
    )

[memsharded]

quick way: conan --version

You can check the http://docs.conan.io/en/latest/changelog.html for release dates. Will have a look to the issue when possible, thanks for your feedback.

[thelfer]

Indeed it is a simple as that,

$ conan --version
Conan version 0.28.0

BTW, it's a shame the --version option does not appear when doing conan --help.

[memsharded]

Sorry, I hadn't read well your report. I have been able to reproduce it now. Checking.

[thelfer]

This simply means that I was not clear enough. Do you now have enough information to investigate the trouble ? Anyway, thank you for your reactivity

[thelfer]

Also sorry for not having given a more smaller example, I know that cloning TFEL can be quite long.

[memsharded]

Yes, no worries, it was good! I have spotted the issue, it is in the check if 'python_version' in self.options. You don't need it, "python_version" will be there if you defined it. You can access its value with self.options.python_version.

Try to do without it. I will check if I can improve the value in self.options issue, so it doesn't hang there.

[memsharded]

Also, one tip: Use git clone repo-url --depth=1 for much faster clones

[thelfer]

I am not sure to understand what I shall do. If I remove the test, how could the following line work if python_version is not defined ?

args.append('-DPython_ADDITIONAL_VERSION='+ \ str(self.options['"python_version']))

Thank you again for your help.

[memsharded]

If you define the options it will always be there, it will always exist. If you want to check that it has no value, because nobody provided a valid version (only possible if ANY), you can check:

if self.options.python_version:
     args.append("-DPython_ADDITIONAL_VERSION=%s" % self.options.python_version)

Also you might find convenient the newer syntax of the CMake helper:

cmake = CMake(self)
if self.options.python_version:
     cmake.definitions["Python_ADDITIONAL_VERSION"] = self.options.python_version
# it will be automatically added to cmake.command_line

check latest docs: http://docs.conan.io/en/latest/reference/build_helpers/cmake.html#cmake-reference

[memsharded]

I have submitted a fix, so at least the operation option in self.options doesnt freeze. It shouldn't be necessary in your case (in fact in your case will not work, because the in operation returns if the thing exists, and in your case, it does exist, but with a None value): https://github.com/conan-io/conan/pull/1976

[thelfer]

Ok. So it all boils down to my misunderstanding of how self.options works. I think that we can close the issue. Thanks again for your help. Regards, Thomas

[thelfer]

BTW: just one last thing: when it hangs, does my .conan directory go garbage as I though. In this case, isn't there anything to do ?

[memsharded]

What do you mean garbage? The hang shouldn't affect other packages, for example. In theory, you should be fine just removing the package, like:

$ conan remove Pkg/version@user/channel -f

Also, you can go to the local cache in your <userhome>/.conan/data and remove manually the folder. Also, it is mostly a cache, you should be fine completely removing the <userhome>/.conan folder too (it will require re-downloading packages)

[thelfer]

If I remember correctly, the conan export thelfer/TFEL would also hang if the problem is triggered. I did not test it again though.

[memsharded]

It shouldn't because it doesn't execute the offending line. But if it happens again, please re-open the issue and report :)

[thelfer]

ok. Probably got confused. Thank again.