cms-gem-daq-project / gem-plotting-tools

Repository for GEM commissioning plotting tools
GNU General Public License v3.0
1 stars 26 forks source link

Bug Report: make rpm fails to build the package on lxplus #107

Closed lmoureaux closed 6 years ago

lmoureaux commented 6 years ago

Brief summary of issue

Even though lxplus isn't a supported development environment, some of the tools cannot be run on the P5 machines (eg because they require cluster access). Developing and testing on different machines would be painful.

make rpm crashes on lxplus.

Types of issue

Expected Behavior

make rpm works on lxplus.

Current Behavior

As recommended, the following wasn't done in a virtualenv:

$ cd GEM/gem-plotting-tools/
$ LANG=C make && LANG=C make rpm
PythonModules=["gempython.gemplotting", "gempython.gemplotting.utils", "gempython.gemplotting.fitting", "gempython.gemplotting.macros", "gempython.gemplotting.mapping" ]
Using BUILD_HOME=/afs/cern.ch/user/l/lmoureau/GEM
OS Detected: slc6
mkdir -p pkg/gempython/gemplotting/utils
PythonModules=["gempython.gemplotting", "gempython.gemplotting.utils", "gempython.gemplotting.fitting", "gempython.gemplotting.macros", "gempython.gemplotting.mapping" ]
Using BUILD_HOME=/afs/cern.ch/user/l/lmoureau/GEM
OS Detected: slc6
mkdir -p pkg/gempython/gemplotting/utils
mkdir -p pkg/gempython/scripts
cp -rf README.md LICENSE CHANGELOG.md MANIFEST.in requirements.txt pkg/gempython/gemplotting
cp: cannot stat `CHANGELOG.md': No such file or directory
make: [preprpm] Error 1 (ignored)
cp -rf README.md LICENSE CHANGELOG.md MANIFEST.in requirements.txt pkg
cp: cannot stat `CHANGELOG.md': No such file or directory
make: [preprpm] Error 1 (ignored)
Running _setup_update target
PackagePath ackagePath
mkdir -p /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/rpm/RPMBUILD
if [ -e /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/setup.py ]; then \
                echo Found /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/setup.py; \
                cp /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/setup.py /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/rpm/setup.py; \
        elif [ -e /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/pkg/setup.py ]; then \
                echo Found /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/pkg/setup.py; \
                cp /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/pkg/setup.py /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/rpm/setup.py; \
        elif [ -e /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/setup/setup.py ]; then \
                echo Found /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/setup/setup.py; \
                cp /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/setup/setup.py /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/rpm/setup.py; \
        elif [ -e /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/setup/build/setup.py ]; then \
                echo Found /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/setup/build/setup.py; \
                cp /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/setup/build/setup.py /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/rpm/setup.py; \
        elif [ -e /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/setup/config/setupTemplate.py ]; then \
                echo Found /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/setup/config/setupTemplate.py; \
                cp /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/setup/config/setupTemplate.py /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/rpm/setup.py; \
        elif [ -e /afs/cern.ch/user/l/lmoureau/GEM/config/build/setupTemplate.py ]; then \
                echo Found /afs/cern.ch/user/l/lmoureau/GEM/config/build/setupTemplate.pyz; \
                cp /afs/cern.ch/user/l/lmoureau/GEM/config/build/setupTemplate.py /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/rpm/setup.py; \
        else \
                echo Unable to find any setupTemplate.py; \
                exit 1; \
        fi
Found /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/pkg/setup.py
sed -i 's#__author__##'                /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/rpm/setup.py
sed -i 's#__project__#gem-plotting-tools#'                /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/rpm/setup.py
sed -i 's#__summary__#None#'                      /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/rpm/setup.py
sed -i 's#__gitrev__#a2464de#'                  /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/rpm/setup.py
sed -i 's#__builddate__#01062018#'           /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/rpm/setup.py
sed -i 's#__package__#gemplotting#'                /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/rpm/setup.py
sed -i 's#__packagedir__#/afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools#'         /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/rpm/setup.py
sed -i 's#__packagename__#gempython_gemplotting#'        /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/rpm/setup.py
sed -i 's#__longpackage__#gemplotting#'        /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/rpm/setup.py
sed -i 's#__pythonmodules__#["gempython.gemplotting", "gempython.gemplotting.utils", "gempython.gemplotting.fitting", "gempython.gemplotting.macros", "gempython.gemplotting.mapping" ]#'    /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/rpm/setup.py
sed -i 's#__version__#1.0.0#'   /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/rpm/setup.py
sed -i 's#__release__#slc6#'            /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/rpm/setup.py
sed -i 's#__prefix__##'          /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/rpm/setup.py
sed -i 's#__os__#slc6#'                 /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/rpm/setup.py
sed -i 's#__platform__#Linux-2.6.32-696.23.1.el6.x86_64-x86_64-with-redhat-6.9-Carbon#'     /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/rpm/setup.py
sed -i 's#__description__#None#'                  /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/rpm/setup.py
if [ -e /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/setup.cfg ]; then \
                echo Found /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/setup.cfg; \
                cp /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/setup.cfg /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/rpm/setup.cfg; \
        elif [ -e /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/pkg/setup.cfg ]; then \
                echo Found /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/pkg/setup.cfg; \
                cp /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/pkg/setup.cfg /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/rpm/setup.cfg; \
        elif [ -e /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/setup/setup.cfg ]; then \
                echo Found /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/setup/setup.cfg; \
                cp /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/setup/setup.cfg /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/rpm/setup.cfg; \
        elif [ -e /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/setup/build/setup.cfg ]; then \
                echo Found /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/setup/build/setup.cfg; \
                cp /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/setup/build/setup.cfg /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/rpm/setup.cfg; \
        elif [ -e /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/setup/config/setupTemplate.cfg ]; then \
                echo Found /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/setup/config/setupTemplate.cfg; \
                cp /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/setup/config/setupTemplate.cfg /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/rpm/setup.cfg; \
        elif [ -e /afs/cern.ch/user/l/lmoureau/GEM/config/build/setupTemplate.cfg ]; then \
                echo Found /afs/cern.ch/user/l/lmoureau/GEM/config/setupTemplate.cfg; \
                cp /afs/cern.ch/user/l/lmoureau/GEM/config/build/setupTemplate.cfg /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/rpm/setup.cfg; \
        else \
                echo Unable to find any setupTemplate.cfg; \
                exit 1; \
        fi
Found /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/pkg/setup.cfg
sed -i 's#__author__##'                /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/rpm/setup.cfg
sed -i 's#__project__#gem-plotting-tools#'                /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/rpm/setup.cfg
sed -i 's#__summary__#None#'                      /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/rpm/setup.cfg
sed -i 's#__gitrev__#a2464de#'                  /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/rpm/setup.cfg
sed -i 's#__builddate__#01062018#'           /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/rpm/setup.cfg
sed -i 's#__package__#gemplotting#'                /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/rpm/setup.cfg
sed -i 's#__packagedir__#/afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools#'         /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/rpm/setup.cfg
sed -i 's#__packagename__#gempython_gemplotting#'        /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/rpm/setup.cfg
sed -i 's#__longpackage__#gemplotting#'        /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/rpm/setup.cfg
sed -i 's#__pythonmodules__#["gempython.gemplotting", "gempython.gemplotting.utils", "gempython.gemplotting.fitting", "gempython.gemplotting.macros", "gempython.gemplotting.mapping" ]#'    /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/rpm/setup.cfg
sed -i 's#__version__#1.0.0#'   /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/rpm/setup.cfg
sed -i 's#__release__#slc6#'            /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/rpm/setup.cfg
sed -i 's#__prefix__##'          /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/rpm/setup.cfg
sed -i 's#__os__#slc6#'                 /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/rpm/setup.cfg
sed -i 's#__platform__#Linux-2.6.32-696.23.1.el6.x86_64-x86_64-with-redhat-6.9-Carbon#'     /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/rpm/setup.cfg
sed -i 's#__description__#None#'                  /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/rpm/setup.cfg
Running _rpmsetup target
cd pkg && \
        find . -iname 'setup.*' -prune -o -name "*" -exec install -D \{\} /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/rpm/\{\} \;
install: omitting directory `.'
install: omitting directory `./gempython'
install: omitting directory `./gempython/gemplotting'
install: omitting directory `./gempython/gemplotting/utils'
install: omitting directory `./gempython/gemplotting/macros'
install: omitting directory `./gempython/gemplotting/fitting'
install: omitting directory `./gempython/gemplotting/mapping'
install: omitting directory `./gempython/scripts'
Running _rpmbuild target
cd /afs/cern.ch/user/l/lmoureau/GEM/gem-plotting-tools/rpm && python setup.py bdist_rpm \
        --release slc6.python2.6 \
        --binary-only --force-arch=`uname -m`
Traceback (most recent call last):
  File "setup.py", line 2, in <module>
    from setuptools import setup,find_packages
  File "/afs/cern.ch/user/l/lmoureau/.local/lib/python2.6/site-packages/setuptools/__init__.py", line 13, in <module>
    from setuptools.extension import Extension
  File "/afs/cern.ch/user/l/lmoureau/.local/lib/python2.6/site-packages/setuptools/extension.py", line 9, in <module>
    from .monkey import get_unpatched
  File "/afs/cern.ch/user/l/lmoureau/.local/lib/python2.6/site-packages/setuptools/monkey.py", line 10, in <module>
    from importlib import import_module
ImportError: No module named importlib
make: *** [_rpmbuild] Error 1

Steps to Reproduce (for bugs)

See the above section.

Context (for feature requests)

Developing for #105. See also the Summary above.

Your Environment

bdorney commented 6 years ago

Before we accept this bug can you try on one of the 904 daq machines? We updated most of them to cc7 so you may need to ask @jsturdy to place your account back on the machines.

jsturdy commented 6 years ago

If you must try to develop on lxplus, your option at this time is to pip install --user the missing packages

lmoureaux commented 6 years ago

Tried to make rpm in various environments, on branch develop:

Environment Python Installed Packages Result
Vanilla lxplus 2.6 importlib 'setuptools<39.2.0' 'pip<10.1.0' :-1: (log
importlib 'setuptools<38' 'pip<10.1.0' :+1:
lxplus with² LCG 93 2.7 None :-1: (log)
cc7 (gem904qc8daq 2.7 None :-1: (log)
cc7 (gem904daq01) 2.7 importlib setuptools pip :+1:
slc6 (gem904daq03) 2.6 importlib 'setuptools<38' 'pip<10.1.0' :+1:

:+1: means that make rpm exited with exit code 0. I didn't check the produced packages.

There is a pattern:

Didn't try at P5.

Closing as it's more of a documentation issue, and hence is a duplicate of #96.

Footnotes

  1. Syntax error in setuptools; version as recommended here
  2. The environment used by Brian's script
  3. There's no system pip on gem904qc8daq.
  4. Or: the same minor version (?)