Fix CMake Build

[ryanmrichard]

Description

Using CMake the configuration step gives a warning:

CMake Warning (dev) in CMakeLists.txt: No project() command is present. The top-level CMakeLists.txt file must contain a literal, direct call to the project() command. Add a line of code such as

project(ProjectName)

near the top of the file, but after cmake_minimum_required().

CMake is pretending there is a "project(Project)" command on the first line. This warning is for project developers. Use -Wno-dev to suppress it.

Other than that it configures successfully.

The build step, however, fails with:

Traceback (most recent call last): File "/home/ryan/CMakePP/CMakeDoc/build/setup.py", line 4, in from setuptools import setup ModuleNotFoundError: No module named 'setuptools' Error in sys.excepthook: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 63, in apport_excepthook from apport.fileutils import likely_packaged, get_recent_crashes File "/usr/lib/python3/dist-packages/apport/init.py", line 5, in from apport.report import Report File "/usr/lib/python3/dist-packages/apport/report.py", line 30, in import apport.fileutils File "/usr/lib/python3/dist-packages/apport/fileutils.py", line 23, in from apport.packaging_impl import impl as packaging File "/usr/lib/python3/dist-packages/apport/packaging_impl.py", line 23, in import apt File "/usr/lib/python3/dist-packages/apt/init.py", line 23, in import apt_pkg ImportError: /home/ryan/spack/opt/spack/linux-ubuntu19.10-broadwell/gcc-9.2.1/gcc-8.3.0-y3pfafsmcq7k35yfrt5lgpgqn2ssbiru/lib64/libstdc++.so.6: version `GLIBCXX_3.4.26' not found (required by /usr/lib/x86_64-linux-gnu/libapt-pkg.so.5.90)

Original exception was: Traceback (most recent call last): File "/home/ryan/CMakePP/CMakeDoc/build/setup.py", line 4, in from setuptools import setup ModuleNotFoundError: No module named 'setuptools' make[2]: [CMakeFiles/target.dir/build.make:61: build/timestamp] Error 1 make[1]: [CMakeFiles/Makefile2:76: CMakeFiles/target.dir/all] Error 2 make: *** [Makefile:130: all] Error 2

The point of the configure step is to find all dependencies, so the configure step needs adjusted to ensure that all required Python modules are available (can be done by running the already found Python command and seeing if it can load the module).

Once I install the missing dependencies the build step works; however, the install step does not honor CMAKE_INSTALL_PREFIX (which is the CMake variable dictating where to install the project).

Tasks

• [ ] Add project command to CMakeLists.txt
• [ ] Look for required Python modules during configure
• [ ] Honor CMAKE_INSTALL_PREFIX

[AutonomicPerfectionist]

For the installation prefix, there are three options:

The first and easiest is to use the built-in --prefix option to setup.py, but this requires the prefix location to include an entire python installation. Ex. prefix of ~/.python will require a subdirectory of ~/.python/lib/python3.7/site-packages that the package will actually be placed in.

The second option is to place the package in the prefix, and then also write a .pth file to the system include path, this will tell the interpreter where to find the package but will still require adding files outside of the prefix.

The third option is to create a virtual environment at the prefix, this will keep everything contained but also adds a layer of complexity.

Which would be preferred?

[ryanmrichard]

Your solutions seem heavy. Admittedly I have not worried about packaging a Python project to this extreme before. Ultimately the goal is to get the installed product to work as if it is a single executable. Looking around the internet I'm wondering if pyinstaller is the way to go. We then stick the resulting single file in ${CMAKE_INSTALL_PREFIX}/bin (on *nix OSes) and assume the user has added ${CMAKE_INSTALL_PREFIX}/bin to their path.

[AutonomicPerfectionist]

PyInstaller could work, there are a couple things to discuss though.

First, should we install PyInstaller automatically via a virtual environment in the CMake build directory, or should it just error out if it can't find it? Keeping in mind that we also need someway to fetch dependencies for the actual program, since PyInstaller does not do that automatically.

Second, a single-file executable will self-extract the python files to a temp directory. On Unix-like servers, /tmp will likely be mounted with the noexec bit enabled, which will cause this method to fail entirely. The extraction directory can be changed at build-time, but how should that be communicated to the user?

[ryanmrichard]

I'd treat PyInstaller like the other Python modules we require: just raise an error if it can't be found (we could also throw in a line saying "Python Module X can be installed by running 'pip install x'" to be user friendly).

I didn't realize that it actually has to "install" all those files (I didn't delve very far into the documentation). For sake of argument, let's say the user wants to install CMakeDoc to /home/programs, the single-file then goes to /home/programs/bin. On *nix OSes, what if we make a directory /home/programs/bin/.cmakedoc and use that as the temp directory for unpacking?

[ryanmrichard]

Worth noting that the directory will have to be made at install time since creating it may require root privileges.

[AutonomicPerfectionist]

To avoid having to meddle with the user installing packages to the system path, I've had CMake create a virtual environment in the build directory to install PyInstaller and other dependencies to automatically. This way the user only needs to preinstall setuptools and venv, both of which are almost always included in a standard installation anyways, and CMake will handle the rest. The build directory can still be safely deleted after CMake copies the resultant executable out in the install phase, so nothing external is ever modified. CMake will error out if setuptools or venv are not installed, and will error out if any of the other packages fail to install in the virtualenv.

If you'd like everything to be installed manually, I can easily remove the virtualenv commands and it will still function just fine.

The extraction directory is also set to ${CMAKE_INSTALL_PREFIX}/bin/.cmakedoc, and the required directories are created at install time.