search

mjuric / conda-lsst

Building Conda packages for the LSST stack
4 stars 8 forks source link

build fails on sconsUtils #47

Closed danielsf closed 8 years ago

danielsf commented 8 years ago

Builds of sconsUtils are failing. I cannot determine why based on the resulting _build.log file. The specific command I tried was

conda lsst make-recipes build:b1923 sconsUtils --build

The resulting build log is

Removing old build environment
Removing old work directory
BUILD START: lsst-sconsutils-2016_01.0-0
Fetching package metadata: ........
Solving package specifications: ..........
The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    lsst-doxygen-1.8.5.1       |                4         3.4 MB
    lsst-scons-2.3.5           |                1         756 KB
    ------------------------------------------------------------
                                           Total:         4.1 MB

The following NEW packages will be INSTALLED:

    eups:                 1.5.9-7      
    lsst-doxygen:         1.8.5.1-4    
    lsst-product-configs: 0.0.1-0      
    lsst-scons:           2.3.5-1      
    nomkl:                1.0-0        
    openssl:              1.0.2f-0     
    pip:                  8.0.2-py27_0 
    python:               2.7.11-0     
    readline:             6.2-2        
    setuptools:           19.6.2-py27_0
    sqlite:               3.9.2-0      
    tk:                   8.5.18-0     
    wheel:                0.29.0-py27_0
    zlib:                 1.2.8-0      

Fetching packages ...
lsst-doxygen-1   0% |                              | ETA:  --:--:--   0.00  B/s
lsst-doxygen-1   1% |                               | ETA:  0:00:00  66.16 MB/s
lsst-doxygen-1   2% |                               | ETA:  0:00:00  81.65 MB/s
lsst-doxygen-1   4% |#                              | ETA:  0:00:00  90.63 MB/s
lsst-doxygen-1   5% |#                              | ETA:  0:00:00  93.85 MB/s
lsst-doxygen-1   6% |##                             | ETA:  0:00:00  94.41 MB/s
lsst-doxygen-1   8% |##                             | ETA:  0:00:00  96.50 MB/s
lsst-doxygen-1   9% |###                            | ETA:  0:00:00  98.36 MB/s
lsst-doxygen-1  11% |###                            | ETA:  0:00:00  99.96 MB/s
lsst-doxygen-1  12% |###                            | ETA:  0:00:00 101.27 MB/s
lsst-doxygen-1  13% |####                           | ETA:  0:00:00 102.27 MB/s
lsst-doxygen-1  15% |####                           | ETA:  0:00:00 102.73 MB/s
lsst-doxygen-1  16% |#####                          | ETA:  0:00:00 103.21 MB/s
lsst-doxygen-1  18% |#####                          | ETA:  0:00:00 103.01 MB/s
lsst-doxygen-1  19% |######                         | ETA:  0:00:00 104.52 MB/s
lsst-doxygen-1  20% |######                         | ETA:  0:00:00 107.08 MB/s
lsst-doxygen-1  22% |######                         | ETA:  0:00:00 109.61 MB/s
lsst-doxygen-1  23% |#######                        | ETA:  0:00:00 111.98 MB/s
lsst-doxygen-1  25% |#######                        | ETA:  0:00:00 114.19 MB/s
lsst-doxygen-1  26% |########                       | ETA:  0:00:00 116.27 MB/s
lsst-doxygen-1  27% |########                       | ETA:  0:00:00 118.18 MB/s
lsst-doxygen-1  29% |#########                      | ETA:  0:00:00 120.00 MB/s
lsst-doxygen-1  30% |#########                      | ETA:  0:00:00 121.68 MB/s
lsst-doxygen-1  32% |#########                      | ETA:  0:00:00 123.23 MB/s
lsst-doxygen-1  33% |##########                     | ETA:  0:00:00 124.71 MB/s
lsst-doxygen-1  34% |##########                     | ETA:  0:00:00 126.08 MB/s
lsst-doxygen-1  36% |###########                    | ETA:  0:00:00 127.40 MB/s
lsst-doxygen-1  37% |###########                    | ETA:  0:00:00 128.68 MB/s
lsst-doxygen-1  39% |############                   | ETA:  0:00:00 129.90 MB/s
lsst-doxygen-1  40% |############                   | ETA:  0:00:00 131.02 MB/s
lsst-doxygen-1  41% |############                   | ETA:  0:00:00 131.85 MB/s
lsst-doxygen-1  43% |#############                  | ETA:  0:00:00 132.90 MB/s
lsst-doxygen-1  44% |#############                  | ETA:  0:00:00 132.96 MB/s
lsst-doxygen-1  45% |##############                 | ETA:  0:00:00 130.35 MB/s
lsst-doxygen-1  47% |##############                 | ETA:  0:00:00 128.38 MB/s
lsst-doxygen-1  48% |###############                | ETA:  0:00:00 126.31 MB/s
lsst-doxygen-1  50% |###############                | ETA:  0:00:00 124.65 MB/s
lsst-doxygen-1  51% |###############                | ETA:  0:00:00 124.27 MB/s
lsst-doxygen-1  52% |################               | ETA:  0:00:00 124.93 MB/s
lsst-doxygen-1  54% |################               | ETA:  0:00:00 125.45 MB/s
lsst-doxygen-1  55% |#################              | ETA:  0:00:00 126.31 MB/s
lsst-doxygen-1  57% |#################              | ETA:  0:00:00 127.16 MB/s
lsst-doxygen-1  58% |##################             | ETA:  0:00:00 127.99 MB/s
lsst-doxygen-1  59% |##################             | ETA:  0:00:00 128.76 MB/s
lsst-doxygen-1  61% |###################            | ETA:  0:00:00 129.50 MB/s
lsst-doxygen-1  62% |###################            | ETA:  0:00:00 130.22 MB/s
lsst-doxygen-1  64% |###################            | ETA:  0:00:00 130.94 MB/s
lsst-doxygen-1  65% |####################           | ETA:  0:00:00 131.63 MB/s
lsst-doxygen-1  66% |####################           | ETA:  0:00:00 132.30 MB/s
lsst-doxygen-1  68% |#####################          | ETA:  0:00:00 132.94 MB/s
lsst-doxygen-1  69% |#####################          | ETA:  0:00:00 133.57 MB/s
lsst-doxygen-1  71% |######################         | ETA:  0:00:00 134.17 MB/s
lsst-doxygen-1  72% |######################         | ETA:  0:00:00 134.70 MB/s
lsst-doxygen-1  73% |######################         | ETA:  0:00:00 134.64 MB/s
lsst-doxygen-1  75% |#######################        | ETA:  0:00:00 135.07 MB/s
lsst-doxygen-1  76% |#######################        | ETA:  0:00:00 135.58 MB/s
lsst-doxygen-1  78% |########################       | ETA:  0:00:00 136.10 MB/s
lsst-doxygen-1  79% |########################       | ETA:  0:00:00 136.60 MB/s
lsst-doxygen-1  80% |#########################      | ETA:  0:00:00 137.10 MB/s
lsst-doxygen-1  82% |#########################      | ETA:  0:00:00 137.59 MB/s
lsst-doxygen-1  83% |#########################      | ETA:  0:00:00 138.04 MB/s
lsst-doxygen-1  85% |##########################     | ETA:  0:00:00 138.49 MB/s
lsst-doxygen-1  86% |##########################     | ETA:  0:00:00 138.90 MB/s
lsst-doxygen-1  87% |###########################    | ETA:  0:00:00 139.36 MB/s
lsst-doxygen-1  89% |###########################    | ETA:  0:00:00 139.80 MB/s
lsst-doxygen-1  90% |############################   | ETA:  0:00:00 140.22 MB/s
lsst-doxygen-1  91% |############################   | ETA:  0:00:00 140.63 MB/s
lsst-doxygen-1  93% |############################   | ETA:  0:00:00 141.02 MB/s
lsst-doxygen-1  94% |#############################  | ETA:  0:00:00 141.42 MB/s
lsst-doxygen-1  96% |#############################  | ETA:  0:00:00 141.81 MB/s
lsst-doxygen-1  97% |############################## | ETA:  0:00:00 142.19 MB/s
lsst-doxygen-1  98% |############################## | ETA:  0:00:00 142.43 MB/s
lsst-doxygen-1 100% |###############################| ETA:  0:00:00 142.48 MB/s
lsst-doxygen-1 100% |###############################| Time: 0:00:00 107.66 MB/s
lsst-scons-2.3   0% |                              | ETA:  --:--:--   0.00  B/s
lsst-scons-2.3   2% |                               | Time: 0:00:00  41.25 MB/s
lsst-scons-2.3   4% |#                              | Time: 0:00:00  50.09 MB/s
lsst-scons-2.3   6% |#                              | Time: 0:00:00  54.92 MB/s
lsst-scons-2.3   8% |##                             | Time: 0:00:00  57.99 MB/s
lsst-scons-2.3  10% |###                            | Time: 0:00:00  60.10 MB/s
lsst-scons-2.3  12% |###                            | Time: 0:00:00  61.86 MB/s
lsst-scons-2.3  14% |####                           | Time: 0:00:00  62.53 MB/s
lsst-scons-2.3  16% |#####                          | Time: 0:00:00  62.86 MB/s
lsst-scons-2.3  19% |#####                          | Time: 0:00:00  63.77 MB/s
lsst-scons-2.3  21% |######                         | Time: 0:00:00  64.50 MB/s
lsst-scons-2.3  23% |#######                        | Time: 0:00:00  65.18 MB/s
lsst-scons-2.3  25% |#######                        | Time: 0:00:00  65.87 MB/s
lsst-scons-2.3  27% |########                       | Time: 0:00:00  66.29 MB/s
lsst-scons-2.3  29% |#########                      | Time: 0:00:00  66.64 MB/s
lsst-scons-2.3  31% |#########                      | Time: 0:00:00  66.98 MB/s
lsst-scons-2.3  33% |##########                     | Time: 0:00:00  66.65 MB/s
lsst-scons-2.3  35% |###########                    | Time: 0:00:00  66.87 MB/s
lsst-scons-2.3  38% |###########                    | Time: 0:00:00  67.07 MB/s
lsst-scons-2.3  40% |############                   | Time: 0:00:00  67.31 MB/s
lsst-scons-2.3  42% |#############                  | Time: 0:00:00  67.67 MB/s
lsst-scons-2.3  44% |#############                  | Time: 0:00:00  68.10 MB/s
lsst-scons-2.3  46% |##############                 | Time: 0:00:00  68.41 MB/s
lsst-scons-2.3  48% |###############                | Time: 0:00:00  68.82 MB/s
lsst-scons-2.3  50% |###############                | Time: 0:00:00  69.12 MB/s
lsst-scons-2.3  52% |################               | Time: 0:00:00  69.46 MB/s
lsst-scons-2.3  55% |#################              | Time: 0:00:00  69.76 MB/s
lsst-scons-2.3  57% |#################              | Time: 0:00:00  70.10 MB/s
lsst-scons-2.3  59% |##################             | Time: 0:00:00  70.42 MB/s
lsst-scons-2.3  61% |###################            | Time: 0:00:00  69.08 MB/s
lsst-scons-2.3  63% |###################            | Time: 0:00:00  69.12 MB/s
lsst-scons-2.3  65% |####################           | Time: 0:00:00  69.29 MB/s
lsst-scons-2.3  67% |####################           | Time: 0:00:00  69.50 MB/s
lsst-scons-2.3  69% |#####################          | Time: 0:00:00  69.71 MB/s
lsst-scons-2.3  71% |######################         | Time: 0:00:00  69.72 MB/s
lsst-scons-2.3  74% |######################         | Time: 0:00:00  69.31 MB/s
lsst-scons-2.3  76% |#######################        | Time: 0:00:00  69.45 MB/s
lsst-scons-2.3  78% |########################       | Time: 0:00:00  69.72 MB/s
lsst-scons-2.3  80% |########################       | Time: 0:00:00  70.00 MB/s
lsst-scons-2.3  82% |#########################      | Time: 0:00:00  70.30 MB/s
lsst-scons-2.3  84% |##########################     | Time: 0:00:00  70.60 MB/s
lsst-scons-2.3  86% |##########################     | Time: 0:00:00  70.92 MB/s
lsst-scons-2.3  88% |###########################    | Time: 0:00:00  70.75 MB/s
lsst-scons-2.3  90% |############################   | Time: 0:00:00  70.75 MB/s
lsst-scons-2.3  93% |############################   | Time: 0:00:00  70.87 MB/s
lsst-scons-2.3  95% |#############################  | Time: 0:00:00  70.99 MB/s
lsst-scons-2.3  97% |############################## | Time: 0:00:00  71.13 MB/s
lsst-scons-2.3  99% |############################## | Time: 0:00:00  71.27 MB/s
lsst-scons-2.3 100% |###############################| Time: 0:00:00  70.56 MB/s
lsst-scons-2.3 100% |###############################| Time: 0:00:00  69.64 MB/s
lsst-scons-2.3 100% |###############################| Time: 0:00:00  67.69 MB/s
Extracting packages ...
[                    ]|                                                  |   0%
[lsst-doxygen        ]|                                                  |   0%
[lsst-scons          ]|#########################                         |  50%
[      COMPLETE      ]|##################################################| 100%
Linking packages ...
[      COMPLETE      ]|                                                  |   0%
[nomkl-1.0           ]|                                                  |   0%
[nomkl-1.0           ]|                                                  |   0%
[openssl-1.0.2f      ]|###                                               |   7%
[readline-6.2        ]|#######                                           |  14%
[sqlite-3.9.2        ]|##########                                        |  21%
[tk-8.5.18           ]|##############                                    |  28%
[zlib-1.2.8          ]|#################                                 |  35%
[python-2.7.11       ]|#####################                             |  42%
[eups-1.5.9          ]|#########################                         |  50%
[setuptools-19.6.2   ]|############################                      |  57%
[wheCloning into '/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/conda-bld/work'...
done.
Note: checking out '98762fb2d962014e5636e94dbe79afd08843681f'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at 98762fb... Merge branch 'tickets/DM-4921'
patching file python/lsst/sconsUtils/state.py
++ git rev-parse --show-toplevel
+ export SRC_DIR=/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/conda-bld/work
+ SRC_DIR=/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/conda-bld/work
+ cd /Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/conda-bld/work
+ [[ darwin14 == darwin* ]]
+ export MACOSX_DEPLOYMENT_TARGET=10.9
+ MACOSX_DEPLOYMENT_TARGET=10.9
+ export 'LDFLAGS=-arch x86_64 -headerpad_max_install_names'
+ LDFLAGS='-arch x86_64 -headerpad_max_install_names'
+ export ARCHFLAGS=-headerpad_max_install_names
+ ARCHFLAGS=-headerpad_max_install_names
+++ which c++
++ dirname /usr/bin/c++
+ [[ /usr/bin -ef /Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/bin ]]
+ export LIBRARY_PATH=:/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/lib
+ LIBRARY_PATH=:/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/lib
+ [[ -f SConstruct ]]
+ CXX=c++
++ mktemp -d -t XXXXXX
+ export CXXSTUBPATH=/tmp/XXXXXX.NDWUJ7qW
+ CXXSTUBPATH=/tmp/XXXXXX.NDWUJ7qW
+ mkdir -p /tmp/XXXXXX.NDWUJ7qW
+ export PATH=/tmp/XXXXXX.NDWUJ7qW:/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/bin:/Users/danielsf/physics/conda_lsst_test/conda-lsst/bin/:/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/bin/:/opt/local/bin:/opt/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/usr/texbin:/opt/X11/bin
+ PATH=/tmp/XXXXXX.NDWUJ7qW:/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/bin:/Users/danielsf/physics/conda_lsst_test/conda-lsst/bin/:/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/bin/:/opt/local/bin:/opt/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/usr/texbin:/opt/X11/bin
+ CXXSTUB=/tmp/XXXXXX.NDWUJ7qW/c++
++ which c++
+ CXXBIN=/usr/bin/c++
+ cat
+ chmod +x /tmp/XXXXXX.NDWUJ7qW/c++
+ echo '/tmp/XXXXXX.NDWUJ7qW/c++ trampoline created, to ensure $LIBRARY_PATH is preserved:'
/tmp/XXXXXX.NDWUJ7qW/c++ trampoline created, to ensure $LIBRARY_PATH is preserved:
+ cat /tmp/XXXXXX.NDWUJ7qW/c++
#!/bin/bash
export LIBRARY_PATH=":/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/lib"
exec "/usr/bin/c++" "$@"
++ basename ups/sconsUtils.table .table
+ PRODUCT=sconsUtils
+ EUPS_VERSION=2016_01.0
+ PREFIX=/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/opt/lsst/sconsUtils
+ source eups-setups.sh
++ export EUPS_SHELL=sh
++ EUPS_SHELL=sh
++ '[' '' '!=' '' ']'
++ export EUPS_DIR=/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/opt/eups
++ EUPS_DIR=/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/opt/eups
+++ python -E -S -c '
import sys
pp = []
for d in sys.argv[1].split(":"):
    if d and d not in pp:
        pp += [d]
if not sys.argv[2] in pp:
    pp = [sys.argv[2]] + pp
print ":".join(pp)' '' /Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/var/opt/eups
++ export EUPS_PATH=/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/var/opt/eups
++ EUPS_PATH=/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/var/opt/eups
++ export SETUP_EUPS=eups
++ SETUP_EUPS=eups
++ unset PROD_DIR_PREFIX
++ unset PRODUCTS
++ export PATH=/tmp/XXXXXX.NDWUJ7qW:/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/bin:/Users/danielsf/physics/conda_lsst_test/conda-lsst/bin/:/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/bin/:/opt/local/bin:/opt/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/usr/texbin:/opt/X11/bin:/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/opt/eups/bin
++ PATH=/tmp/XXXXXX.NDWUJ7qW:/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/bin:/Users/danielsf/physics/conda_lsst_test/conda-lsst/bin/:/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/bin/:/opt/local/bin:/opt/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/usr/texbin:/opt/X11/bin:/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/opt/eups/bin
++ '[' X '!=' X ']'
++ export PYTHONPATH=/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/opt/eups/python
++ PYTHONPATH=/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/opt/eups/python
++ '[' X '!=' X -a -f /Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/opt/eups/etc/bash_completion.d/eups ']'
+ eupspkg PREFIX=/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/opt/lsst/sconsUtils PRODUCT=sconsUtils VERSION=2016_01.0 FLAVOR=generic prep
+ eups list
anaconda              system        current
doxygen               1.8.5.lsst1   current conda b1923
flask                 system        current
libevent              system        current
lsst_sims             tag:danielsf 
matplotlib            system        current
numpy                 system        current
obs_lsstSim           tag:danielsf 
protobuf              system        current
python                system        current
scons                 2.3.5         conda b1923
scons                 system        current
sims_GalSimInterface  tag:danielsf 
sims_catUtils         tag:danielsf 
sims_catalogs_generation tag:danielsf 
sims_catalogs_measures tag:danielsf 
sims_coordUtils       tag:danielsf 
sims_data             tag:danielsf 
sims_maf              tag:danielsf 
sims_maps             tag:danielsf 
sims_operations       tag:danielsf 
sims_photUtils        tag:danielsf 
sims_sed_library      tag:danielsf 
sims_sed_library      tag:new_sed_library 
sims_utils            tag:clone2 
sims_utils            tag:danielsf 
sims_utils            tag:danielsf_clean 
swig                  system        current
throughputs           tag:danielsf 
twisted               system        current
+ setup -r .
++ /Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/opt/eups/bin/eups_setup -r .
+ eval export 'PATH=/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/conda-bld/work/bin:/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/opt/lsst/doxygen/bin:/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/opt/lsst/lsst-product-configs/bin:/tmp/XXXXXX.NDWUJ7qW:/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/bin:/Users/danielsf/physics/conda_lsst_test/conda-lsst/bin/:/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/bin/:/opt/local/bin:/opt/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/usr/texbin:/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/opt/eups/bin;' export 'SETUP_PYTHON='\''python' system -f DarwinX86 -Z '/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/var/opt/eups'\'';' export 'SETUP_SCONSUTILS='\''sconsUtils' LOCAL:/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/conda-bld/work -f DarwinX86 -Z '(none)'\'';' export 'SCONSUTILS_DIR=/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/conda-bld/work;' export 'SCONS_DIR=/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/opt/lsst/lsst-product-configs;' export 'SETUP_DOXYGEN='\''doxygen' 1.8.5.lsst1 -f DarwinX86 -Z '/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/var/opt/eups'\'';' export 'SETUP_SCONS='\''scons' system -f DarwinX86 -Z '/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/var/opt/eups'\'';' export 'PYTHONPATH=/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/conda-bld/work/python:/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/opt/eups/python;' export 'PYTHON_DIR=/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/opt/lsst/lsst-product-configs;' export DOXYGEN_DIR=/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/opt/lsst/doxygen
++ export PATH=/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/conda-bld/work/bin:/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/opt/lsst/doxygen/bin:/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/opt/lsst/lsst-product-configs/bin:/tmp/XXXXXX.NDWUJ7qW:/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/bin:/Users/danielsf/physics/conda_lsst_test/conda-lsst/bin/:/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/bin/:/opt/local/bin:/opt/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/usr/texbin:/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/opt/eups/bin
++ PATH=/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/conda-bld/work/bin:/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/opt/lsst/doxygen/bin:/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/opt/lsst/lsst-product-configs/bin:/tmp/XXXXXX.NDWUJ7qW:/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/bin:/Users/danielsf/physics/conda_lsst_test/conda-lsst/bin/:/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/bin/:/opt/local/bin:/opt/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/usr/texbin:/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/opt/eups/bin
++ export 'SETUP_PYTHON=python system -f DarwinX86 -Z /Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/var/opt/eups'
++ SETUP_PYTHON='python system -f DarwinX86 -Z /Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/var/opt/eups'
++ export 'SETUP_SCONSUTILS=sconsUtils LOCAL:/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/conda-bld/work -f DarwinX86 -Z (none)'
++ SETUP_SCONSUTILS='sconsUtils LOCAL:/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/conda-bld/work -f DarwinX86 -Z (none)'
++ export SCONSUTILS_DIR=/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/conda-bld/work
++ SCONSUTILS_DIR=/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/conda-bld/work
++ export SCONS_DIR=/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/opt/lsst/lsst-product-configs
++ SCONS_DIR=/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/opt/lsst/lsst-product-configs
++ export 'SETUP_DOXYGEN=doxygen 1.8.5.lsst1 -f DarwinX86 -Z /Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/var/opt/eups'
++ SETUP_DOXYGEN='doxygen 1.8.5.lsst1 -f DarwinX86 -Z /Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/var/opt/eups'
++ export 'SETUP_SCONS=scons system -f DarwinX86 -Z /Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/var/opt/eups'
++ SETUP_SCONS='scons system -f DarwinX86 -Z /Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/var/opt/eups'
++ export PYTHONPATH=/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/conda-bld/work/python:/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/opt/eups/python
++ PYTHONPATH=/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/conda-bld/work/python:/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/opt/eups/python
++ export PYTHON_DIR=/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/opt/lsst/lsst-product-configs
++ PYTHON_DIR=/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/opt/lsst/lsst-product-configs
++ export DOXYGEN_DIR=/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/opt/lsst/doxygen
++ DOXYGEN_DIR=/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/opt/lsst/doxygen
+ export
declare -x ARCH="64"
declare -x ARCHFLAGS="-headerpad_max_install_names"
declare -x CFLAGS="-arch x86_64"
declare -x CONDA_BUILD="1"
declare -x CONDA_DEFAULT_ENV="/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build"
declare -x CPU_COUNT="4"
declare -x CXXFLAGS="-arch x86_64"
declare -x CXXSTUBPATH="/tmp/XXXXXX.NDWUJ7qW"
declare -x DOXYGEN_DIR="/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/opt/lsst/doxygen"
declare -x EUPS_DIR="/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/opt/eups"
declare -x EUPS_PATH="/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/var/opt/eups"
declare -x EUPS_SHELL="sh"
declare -x GIT_BUILD_STR="0_g98762fb"
declare -x GIT_DESCRIBE_HASH="g98762fb"
declare -x GIT_DESCRIBE_NUMBER="0"
declare -x GIT_DESCRIBE_TAG="w.2016.06"
declare -x GIT_FULL_HASH="98762fb2d962014e5636e94dbe79afd08843681f"
declare -x HOME="/Users/danielsf"
declare -x LDFLAGS="-arch x86_64 -headerpad_max_install_names"
declare -x LIBRARY_PATH=":/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/lib"
declare -x MACOSX_DEPLOYMENT_TARGET="10.9"
declare -x OLDPWD="/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/conda-bld/work"
declare -x OSX_ARCH="x86_64"
declare -x PATH="/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/conda-bld/work/bin:/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/opt/lsst/doxygen/bin:/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/opt/lsst/lsst-product-configs/bin:/tmp/XXXXXX.NDWUJ7qW:/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/bin:/Users/danielsf/physics/conda_lsst_test/conda-lsst/bin/:/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/bin/:/opt/local/bin:/opt/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/usr/texbin:/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/opt/eups/bin"
declare -x PERL_VER="5.18.2"
declare -x PKG_BUILDNUM="0"
declare -x PKG_BUILD_STRING="0"
declare -x PKG_CONFIG_PATH="/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/lib/pkgconfig"
declare -x PKG_NAME="lsst-sconsutils"
declare -x PKG_VERSION="2016_01.0"
declare -x PREFIX="/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/opt/lsst/sconsUtils"
declare -x PWD="/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/conda-bld/work"
declare -x PY3K="0"
declare -x PYTHON="/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/bin/python"
declare -x PYTHONNOUSERSITE="1"
declare -x PYTHONPATH="/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/conda-bld/work/python:/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/opt/eups/python"
declare -x PYTHON_DIR="/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/opt/lsst/lsst-product-configs"
declare -x PY_VER="2.7"
declare -x R="/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/bin/R"
declare -x RECIPE_DIR="/Users/danielsf/physics/conda_lsst_test/conda-lsst/recipes/lsst-sconsutils"
declare -x SCONSUTILS_DIR="/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/conda-bld/work"
declare -x SCONS_DIR="/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/opt/lsst/lsst-product-configs"
declare -x SETUP_DOXYGEN="doxygen 1.8.5.lsst1 -f DarwinX86 -Z /Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/var/opt/eups"
declare -x SETUP_EUPS="eups"
declare -x SETUP_PYTHON="python system -f DarwinX86 -Z /Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/var/opt/eups"
declare -x SETUP_SCONS="scons system -f DarwinX86 -Z /Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/var/opt/eups"
declare -x SETUP_SCONSUTILS="sconsUtils LOCAL:/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/conda-bld/work -f DarwinX86 -Z (none)"
declare -x SHLVL="1"
declare -x SP_DIR="/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/lib/python2.7/site-packages"
declare -x SRC_DIR="/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/conda-bld/work"
declare -x STDLIB_DIR="/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/lib/python2.7"
declare -x SYS_PREFIX="/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda"
declare -x SYS_PYTHON="/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/bin/python"
+ echo '#!/bin/bash'
+ export
+ echo 'PRODUCT='\''sconsUtils'\'''
+ echo 'eupspkg PREFIX="$PREFIX" PRODUCT="$PRODUCT" VERSION="$EUPS_VERSION" FLAVOR=generic $@'
+ chmod +x _build.sh
+ eupspkg PREFIX=/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/opt/lsst/sconsUtils PRODUCT=sconsUtils VERSION=2016_01.0 FLAVOR=generic config
+ eupspkg PREFIX=/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/opt/lsst/sconsUtils PRODUCT=sconsUtils VERSION=2016_01.0 FLAVOR=generic build
/Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/envs/_build/opt/eups/lib/eupspkg.sh: line 761: scons: command not found
el-0.29.0        ]|################################                  |  64%
[lsst-doxygen-1.8.5.1]|###################################               |  71%
[lsst-product-configs-0.0.1]|##################################          |  78%
[pip-8.0.2           ]|##########################################        |  85%
[lsst-scons-2.3.5    ]|##############################################    |  92%
[      COMPLETE      ]|##################################################| 100%
Removing old work directory
checkout: u'98762fb2d962014e5636e94dbe79afd08843681f'
==> git log -n1 <==

commit 98762fb2d962014e5636e94dbe79afd08843681f
Merge: 9960fc2 ce6f111
Author: Tim Jenness <tjenness@lsst.org>
Date:   Fri Jan 29 10:54:33 2016 -0700

    Merge branch 'tickets/DM-4921'

==> git describe --tags --dirty <==

w.2016.06

==> git status <==

HEAD detached at 98762fb
nothing to commit, working directory clean

Applying patch: u'/Users/danielsf/physics/conda_lsst_test/conda-lsst/recipes/lsst-sconsutils/macosx-deployment-target.patch'
Package: lsst-sconsutils-2016_01.0-0
source tree in: /Users/danielsf/physics/conda_lsst_test/conda-lsst/miniconda/conda-bld/work
Command failed: /bin/bash -x -e /Users/danielsf/physics/conda_lsst_test/conda-lsst/recipes/lsst-sconsutils/build.sh
danielsf commented 8 years ago

@mareuter did not have this problem on OSX, though he did have it on Linux.

The difference between his OSX and mine is that he is running El Capitan and I am still running Yosemite. I am still too scared to update my principal computer to El Capitan. I will try this on my older laptop, running El Capitan, when I get home tonight.

mjuric commented 8 years ago

Looking at the scons list output, looks like scons is declared twice, once from the lsst-scons package, and once with the version system:

scons                 2.3.5         conda b1923
scons                 system        current

I wonder if this has something to do with stopping to use conda-bundled scons, and switching to our scons (has it been removed from legacy_configs)? @jhoblitt, any ideas?

jhoblitt commented 8 years ago

Uh, err, umm... the PR to legacy_configs https://github.com/mjuric/legacy_configs/pull/2 needs to be merged.

jhoblitt commented 8 years ago

I've merged it but we probably need to make a tag and update config.yaml

mjuric commented 8 years ago

Uh, err, umm... the PR to legacy_configs mjuric/legacy_configs#2 needs to be merged.

:p. oops.

I've merged it but we probably need to make a tag and update config.yaml

Probably (I know you're swamped -- maybe @danielsf can help?)

danielsf commented 8 years ago

Sure. I'll see what I can do.

danielsf commented 8 years ago

how do I force conda-lsst to update lsst-product-configs? It insists that lsst-product-configs is up-to-date, despite the change in legacy_configs

jhoblitt commented 8 years ago

It needs to be tagged. I'm out at lunch but if you can sit tight for a couple of hours I will be able to take care of it.

danielsf commented 8 years ago

Thanks.

For future reference, can you document what you are doing so that I can handle this in the future? I get the sense that legacy_configs.git needs a tag, but the only tag I currently see there is 1.0.0 and lsst-product-configs says it is on version 0.0.1, so I am confused.

Again, thanks.

jhoblitt commented 8 years ago

I will skim the README and see if anything is missing. Most of my work was from b1881 and we're on b1925 -- I'm suspicious that there won't have been breakages. I've started a build of b1925 to see how far it gets. I'm planning to cut a tag along the lines of 1.0.1925 if its in a working state.

mjuric commented 8 years ago

@danielsf @jhoblitt Josh brings up a good point -- conda-lsst is almost by definition a moving target, wrt. applying it to arbitrary bNNNN builds. Given it has things such as package overrides, patches, etc., it should always work on the latest ones, but it's not guaranteed to work on the older bNNNN builds; e.g., a change in a patch that's being applied will break the old builds. To ensure reproducible builds, you should really save the generated recipes. That's why conda-lsst doesn't do a build itself, but generates the recipes (and the convenience rebuild.sh file).

If you have to generate recipes for an older release, you'd probably want to check out a commit of conda-lsst that is close in time to it. I'm thinking it may be a good (informal) practice to start tagging conda-lsst with bNNNN tags, whenever we use it to build one, to know which commit worked?

A long(er) term solution is potentially to split up the contents of the etc/ directory (patches, package overrides, etc) from the recipe generator (the rest of the code). They're now less coupled than they used to be. I'm not sure if it's worth it, though (I change my mind on that three times a day...).

jhoblitt commented 8 years ago

I've made an update to legacy_configs and tagged it as 1.0.1925 -- but already suspect I should have used the minor version to reflect the build #. With this PR https://github.com/mjuric/conda-lsst/pull/48, I am able to build all of lsst_distrib and qserv_distrib but run into trouble with sims_utils tests failing with ConfigurationMissingWarning: Configuration defaults will be used due to OSError:Could not find unix home directory to search for astropy config dir on None.

danielsf commented 8 years ago

@jhoblitt What machine are you running this build on. We saw this failure on Jenkins once. It was (I think) literally because the environment variable $HOME was meaningless. I thought we had commented all of the offending tests out.

jhoblitt commented 8 years ago

@danielsf Centos 5. This is the container I've been doing all conda-lsst related work in:

docker run -ti lsstsqre/centos:5-conda-base bash

jhoblitt commented 8 years ago

@danielsf The astropy error appears to be coming from https://github.com/astropy/astropy/blob/master/astropy/config/paths.py#L47-L48 -- I'm wondering if scons is santizing away HOME. Can you look into this?

jhoblitt commented 8 years ago

@danielsf Actually, I wasn't being very cleaver and looking at the entire backtrace. (The warning message from astropy was a red herring)The problem is obvious:

tests/test_samplingFunctions.py

/home/conda/conda-lsst/miniconda/envs/_build/lib/python2.7/site-packages/astropy/config/configuration.py:687: ConfigurationMissingWarning: Configuration defaults will be used due to OSError:Could not find unix home directory to search for astropy config dir on None
  warn(ConfigurationMissingWarning(msg))
Traceback (most recent call last):
  File "tests/test_samplingFunctions.py", line 18, in <module>
    from lsst.sims.utils import ObservationMetaData
  File "/home/conda/conda-lsst/miniconda/conda-bld/work/python/lsst/sims/utils/__init__.py", line 12, in <module>
    from .healpyUtils import *
  File "/home/conda/conda-lsst/miniconda/conda-bld/work/python/lsst/sims/utils/healpyUtils.py", line 2, in <module>
    import healpy as hp
  File "/home/conda/conda-lsst/miniconda/envs/_build/opt/lsst/healpy/lib/python/healpy-1.8.1-py2.7-linux-x86_64.egg/healpy/__init__.py", line 47, in <module>
    from .sphtfunc import (anafast, map2alm,
  File "/home/conda/conda-lsst/miniconda/envs/_build/opt/lsst/healpy/lib/python/healpy-1.8.1-py2.7-linux-x86_64.egg/healpy/sphtfunc.py", line 22, in <module>
    import six
ImportError: No module named six
danielsf commented 8 years ago

Well, that's "good"

I started down this rabbit hole because I couldn't get healpy to import 'six' in the latest version of the sims stack. I suspect it just needs to be added to the list of packages to be conda installed. I will open a branch and see if I can fix this.

jhoblitt commented 8 years ago

That works but adding it to healpy is a better solution (what I've done). However, I've run into another problem I haven't tracked down yet.

jhoblitt commented 8 years ago

@danielsf I can't get past sims_photUtils, which I am speculating is trying to load data from an LFS repo but I haven't investigated.

Failed test output:
tests/testPhotometry.py

/home/conda/conda-lsst/miniconda/envs/_build/lib/python2.7/site-packages/astropy/config/configuration.py:687: ConfigurationMissingWarning: Configuration defaults will be used due to OSError:Could not find unix home directory to search for astropy config dir on None
  warn(ConfigurationMissingWarning(msg))
.E.
======================================================================
ERROR: testEBV (__main__.photometryUnitTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/testPhotometry.py", line 100, in testEBV
    ebvOutput = ebvObject.calculateEbv(equatorialCoordinates=equatorialCoordinates)
  File "/home/conda/conda-lsst/miniconda/conda-bld/work/python/lsst/sims/photUtils/EBV.py", line 261, in calculateEbv
    self.load_ebvMapNorth()
  File "/home/conda/conda-lsst/miniconda/conda-bld/work/python/lsst/sims/photUtils/EBV.py", line 214, in load_ebvMapNorth
    self.ebvMapNorth.readMapFits(os.path.join(self.ebvDataDir,self.ebvMapNorthName))
  File "/home/conda/conda-lsst/miniconda/conda-bld/work/python/lsst/sims/photUtils/EBV.py", line 27, in readMapFits
    hdulist = pyfits.open(fileName)
  File "/home/conda/conda-lsst/miniconda/envs/_build/opt/lsst/pyfits/lib/python/pyfits-3.4-py2.7-linux-x86_64.egg/pyfits/hdu/hdulist.py", line 124, in fitsopen
    return HDUList.fromfile(name, mode, memmap, save_backup, **kwargs)
  File "/home/conda/conda-lsst/miniconda/envs/_build/opt/lsst/pyfits/lib/python/pyfits-3.4-py2.7-linux-x86_64.egg/pyfits/hdu/hdulist.py", line 266, in fromfile
    save_backup=save_backup, **kwargs)
  File "/home/conda/conda-lsst/miniconda/envs/_build/opt/lsst/pyfits/lib/python/pyfits-3.4-py2.7-linux-x86_64.egg/pyfits/hdu/hdulist.py", line 823, in _readfrom
    hdu = _BaseHDU.readfrom(ffo, **kwargs)
  File "/home/conda/conda-lsst/miniconda/envs/_build/opt/lsst/pyfits/lib/python/pyfits-3.4-py2.7-linux-x86_64.egg/pyfits/hdu/base.py", line 370, in readfrom
    **kwargs)
  File "/home/conda/conda-lsst/miniconda/envs/_build/opt/lsst/pyfits/lib/python/pyfits-3.4-py2.7-linux-x86_64.egg/pyfits/hdu/base.py", line 430, in _readfrom_internal
    header = Header.fromfile(data, endcard=not ignore_missing_end)
  File "/home/conda/conda-lsst/miniconda/envs/_build/opt/lsst/pyfits/lib/python/pyfits-3.4-py2.7-linux-x86_64.egg/pyfits/header.py", line 423, in fromfile
    padding)[1]
  File "/home/conda/conda-lsst/miniconda/envs/_build/opt/lsst/pyfits/lib/python/pyfits-3.4-py2.7-linux-x86_64.egg/pyfits/header.py", line 492, in _from_blocks
    raise IOError('Header missing END card.')
IOError: Header missing END card.

----------------------------------------------------------------------
Ran 3 tests in 0.367s

FAILED (errors=1)
rhiannonlynne commented 8 years ago

Yes, the data that testEBV is trying to load is indeed from a gitlfs repo -- sims_maps -- and your error is entirely consistent with trying to use the pointers in the git repo (instead of the data from git LFS). I believe Scott found that you could just try building against a repo properly downloaded with git LFS.

On Thu, Feb 18, 2016, 4:07 PM Joshua Hoblitt notifications@github.com wrote:

@danielsf https://github.com/danielsf I can't get past sims_photUtils, which I am speculating is trying to load data from an LFS repo but I haven't investigated.

Failed test output: tests/testPhotometry.py

/home/conda/conda-lsst/miniconda/envs/_build/lib/python2.7/site-packages/astropy/config/configuration.py:687: ConfigurationMissingWarning: Configuration defaults will be used due to OSError:Could not find unix home directory to search for astropy config dir on None warn(ConfigurationMissingWarning(msg))

.E.

ERROR: testEBV (main.photometryUnitTest)

Traceback (most recent call last): File "tests/testPhotometry.py", line 100, in testEBV ebvOutput = ebvObject.calculateEbv(equatorialCoordinates=equatorialCoordinates) File "/home/conda/conda-lsst/miniconda/conda-bld/work/python/lsst/sims/photUtils/EBV.py", line 261, in calculateEbv self.load_ebvMapNorth() File "/home/conda/conda-lsst/miniconda/conda-bld/work/python/lsst/sims/photUtils/EBV.py", line 214, in load_ebvMapNorth self.ebvMapNorth.readMapFits(os.path.join(self.ebvDataDir,self.ebvMapNorthName)) File "/home/conda/conda-lsst/miniconda/conda-bld/work/python/lsst/sims/photUtils/EBV.py", line 27, in readMapFits hdulist = pyfits.open(fileName) File "/home/conda/conda-lsst/miniconda/envs/_build/opt/lsst/pyfits/lib/python/pyfits-3.4-py2.7-linux-x86_64.egg/pyfits/hdu/hdulist.py", line 124, in fitsopen return HDUList.fromfile(name, mode, memmap, save_backup, _kwargs) File "/home/conda/conda-lsst/miniconda/envs/_build/opt/lsst/pyfits/lib/python/pyfits-3.4-py2.7-linux-x86_64.egg/pyfits/hdu/hdulist.py", line 266, in fromfile save_backup=save_backup, _kwargs) File "/home/conda/conda-lsst/miniconda/envs/_build/opt/lsst/pyfits/lib/python/pyfits-3.4-py2.7-linux-x86_64.egg/pyfits/hdu/hdulist.py", line 823, in _readfrom hdu = _BaseHDU.readfrom(ffo, _kwargs) File "/home/conda/conda-lsst/miniconda/envs/_build/opt/lsst/pyfits/lib/python/pyfits-3.4-py2.7-linux-x86_64.egg/pyfits/hdu/base.py", line 370, in readfrom _kwargs) File "/home/conda/conda-lsst/miniconda/envs/_build/opt/lsst/pyfits/lib/python/pyfits-3.4-py2.7-linux-x86_64.egg/pyfits/hdu/base.py", line 430, in _readfrom_internal header = Header.fromfile(data, endcard=not ignore_missing_end) File "/home/conda/conda-lsst/miniconda/envs/_build/opt/lsst/pyfits/lib/python/pyfits-3.4-py2.7-linux-x86_64.egg/pyfits/header.py", line 423, in fromfile padding)[1] File "/home/conda/conda-lsst/miniconda/envs/_build/opt/lsst/pyfits/lib/python/pyfits-3.4-py2.7-linux-x86_64.egg/pyfits/header.py", line 492, in _from_blocks raise IOError('Header missing END card.') IOError: Header missing END card.

Ran 3 tests in 0.367s

FAILED (errors=1)

— Reply to this email directly or view it on GitHub https://github.com/mjuric/conda-lsst/issues/47#issuecomment-185987311.

jhoblitt commented 8 years ago

@rhiannonlynne I suspected as much. This means building conda packages for sims is essentially blocked until LFS support is added to conda-lsst. I don't believe that will be very difficult -- just need to find the time.

rhiannonlynne commented 8 years ago

@danielsf didn't you release sims2.2.2 in conda? And that has sims_maps as a dependency? Or am I just confused? (I haven't followed which version is actually available as closely as I might have..)

On Thu, Feb 18, 2016, 4:19 PM Joshua Hoblitt notifications@github.com wrote:

@rhiannonlynne https://github.com/rhiannonlynne I suspected as much. This means building conda packages for sims is essentially blocked until LFS support is added to conda-lsst. I don't believe that will be very difficult -- just need to find the time.

— Reply to this email directly or view it on GitHub https://github.com/mjuric/conda-lsst/issues/47#issuecomment-185989885.

danielsf commented 8 years ago

@rhiannonlynne I released sims_2.2.2 through eups. Other things (which, I guess, @jhoblitt has now fixed -- thanks!) prevented me from releasing it through conda. I was not sure whether conda could handle git lfs. I guess we know the answer now.

mjuric commented 8 years ago

EUPS cant handle git-lfs either, so I'm confused how it could work in EUPS and not in conda?

danielsf commented 8 years ago

I talked with KT about this.

As long as you publish your package with

EUPSPKG_SOURCE=git publich -b bNNNN -t tag package_name

eups distrib install will be able to handle LFS (assuming the user has LFS installed on their machine).

danielsf commented 8 years ago

"publich" = "publish"

jhoblitt commented 8 years ago

There has been some debate as how to handle eups distrib publishing of lfs packages. I think people are leaning towards some sort of annotation on the lfs repos to force them to be published as a git package rather than distributing multi-gigabyte packages.

mjuric commented 8 years ago

I think it's a mistake to package these as EUPS products at all; they're really datasets, not versioned pieces of code. IMHO, they should live in repositories (butlerized, ultimately, though currently backed by git(-lfs)). It'd be good to get @ktlim's angle on this.

@danielsf @rhiannonlynne -- is there a reason not to have sims_maps be something the users clone themselves (maybe with some convenience scripts or APIs that would make that easy, or even fully automated)? I guess git-lfs itself would serve just that purpose (if we reinterpret the sims_maps package as "the code that fetches the maps dataset, which happens to be using git-lfs right now").

mjuric commented 8 years ago

I think people are leaning towards some sort of annotation on the lfs repos to force them to be published as a git package rather than distributing multi-gigabyte packages.

You should be able to just say SOURCE=git in eupspkg.cfg.sh to force it (haven't tested it).

rhiannonlynne commented 8 years ago

@mjuric I don't think I have any strong opinion on whether or not users should clone sims_maps themselves, except to say that we should make it as easy as possible. The instructions for setting up git-lfs are a little cumbersome, being a multi-step process even for anonymous access (http://developer.lsst.io/en/latest/tools/git_lfs.html). Making users then have to install the LSST stack, and also download sims_maps separately seems awkward (and I guess we'd have to require people to install sims_maps first, so that the packages would install and test correctly?).

I'd love to see this whole thing get easier and ideally be invisible (almost invisible?) within the process of installing the LSST stack.

mjuric commented 8 years ago

Agreed on the desire for the ease of use. @jhoblitt has been working on a conda package for git-lfs; that (in combination with SOURCE=git) may make the whole thing fully transparent.

But in the long run delivering repositories as EUPS products doesn't seem right...

danielsf commented 8 years ago

I think we would like to support some degree of versioning on these packages. For example: sims_sed_library is the next candidate for upgrade to git-lfs. This package contains simulated SEDs for different classes of stars and galaxies. We would like to be able to change models as new models become available, but easily revert to previous models should we need to recreate old results.

ktlim commented 8 years ago

Support is going into the Data Butler to be able to handle versions of repositories (but versions exposed as git versions may be a bit of a challenge), if you can hold out a month or two.

jhoblitt commented 8 years ago

@danielsf This should be resolved as of the b1925 tag. However, I am battling new issues with b1935.

danielsf commented 8 years ago

Thanks. It's all moot for me since conda cannot handle LFS (my principal interest in conda-lsst is for producing builds of lsst_sims, which now relies on LFS repositories)

jhoblitt commented 8 years ago

FYI - I have a working git-lfs recipe https://github.com/mjuric/conda-lsst/pull/49 but haven't started on adding lfs support logic to conda-lsst.

danielsf commented 8 years ago

Thanks for doing all of this, @jhoblitt