search

databio / bulker

Manager for multi-container computing environments
https://bulker.io
BSD 2-Clause "Simplified" License
24 stars 2 forks source link

Under some circumstances, "No config found in env var: BULKERCFG" #37

Closed lwaldron closed 4 years ago

lwaldron commented 5 years ago

Trying to setup my office (OSX) computer, I experienced bulker ignoring my BULKERCFG environment variable. I found a solution that worked for reasons I don't understand, below. Here's what happened originally:

wallabe:~ lwaldron$ pip uninstall bulker
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Uninstalling bulker-0.3.0:
  Would remove:
    /Users/lwaldron/Library/Python/2.7/bin/bulker
    /Users/lwaldron/Library/Python/2.7/lib/python/site-packages/bulker-0.3.0-py2.7.egg-info
    /Users/lwaldron/Library/Python/2.7/lib/python/site-packages/bulker/*
Proceed (y/n)? y
  Successfully uninstalled bulker-0.3.0
wallabe:~ lwaldron$ ls Library/Python/2.7/lib/python/site-packages/
Jinja2-2.10.3.dist-info/       jinja2/                        oyaml.py                       yacman-0.6.4-py2.7.egg-info/
MarkupSafe-1.1.1.dist-info/    logmuse/                       oyaml.pyc                      yaml/
PyYAML-5.1.2-py2.7.egg-info/   logmuse-0.2.5-py2.7.egg-info/  ubiquerg/                      
attmap/                        markupsafe/                    ubiquerg-0.5.0-py2.7.egg-info/ 
attmap-0.12.11-py2.7.egg-info/ oyaml-0.9.dist-info/           yacman/                        
wallabe:~ lwaldron$ bulker load demo
Using default config. No config found in env var: BULKERCFG
Bulker config: /Users/lwaldron/Library/Python/2.7/lib/python/site-packages/bulker/templates/bulker_config.yaml
Got URL: http://hub.bulker.io/bulker/demo.yaml
That manifest has already been loaded. Overwrite? [y/N] y
Removing all executables in: /Users/lwaldron/bulker_crates/bulker/demo/default
Loading manifest: 'bulker/demo:default'. Activate with 'bulker activate bulker/demo:default'.
Commands available: cowsay, fortune
wallabe:~ lwaldron$ echo $BULKERCFG
/Users/lwaldron/bulker_config.yaml
wallabe:~ lwaldron$ cat `echo $BULKERCFG`
bulker:
  volumes: ['$HOME']
  envvars: ['DISPLAY']
  registry_url: http://hub.bulker.io/
  default_crate_folder: ${HOME}/bulker_crates
  singularity_image_folder: ${HOME}/simages
  container_engine: docker
  default_namespace: bulker
  executable_template: templates/docker_executable.jinja2
  shell_template: templates/docker_shell.jinja2
  build_template: templates/docker_build.jinja2
  crates:
    bulker:
      demo:
        default: ${HOME}/bulker_crates/bulker/demo/default
    waldronlab:
      bioconductor:
        default: ${HOME}/bulker_crates/waldronlab/bioconductor/default
      levi:
        default: ${HOME}/bulker_crates/waldronlab/levi/default
    databio:
      nsheff:
        default: ${HOME}/bulker_crates/databio/nsheff/default
  tool_args:
    bioconductor:
      bioconductor_full:
        default:
          docker_args: --volume=${HOME}/R/bioc-release:/usr/local/lib/R/host-site-library -e PASSWORD=rstudiopassword -p 8787:8787
        devel:
          docker_args: --volume=${HOME}/R/bioc-devel:/usr/local/lib/R/host-site-library -e PASSWORD=rstudiopassword -p 8788:8787
wallabe:~ lwaldron$ bulker activate demo
Using default config. No config found in env var: BULKERCFG
Bulker config: /Users/lwaldron/Library/Python/2.7/lib/python/site-packages/bulker/templates/bulker_config.yaml
Activating bulker crate: demo

bulker-3.2$ echo $BULKERCFG

bulker-3.2$

Then here's what I did that solved it. 

wallabe:~ lwaldron$ bulker init -c ~/bulker_config.yaml 
Guessing container engine is docker.
Exists. Overwrite? [y/N] y
/Users/lwaldron/Library/Python/2.7/lib/python/site-packages/yacman/yacman.py:144: UserWarning: Writing to a non-locked, existing file. Beware of collisions.
  warnings.warn("Writing to a non-locked, existing file. Beware of collisions.", UserWarning)
Wrote new configuration file: /Users/lwaldron/bulker_config.yaml
wallabe:~ lwaldron$ rm bulker_config.yaml 
wallabe:~ lwaldron$ pip uninstall bulker
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Uninstalling bulker-0.3.0:
  Would remove:
    /Users/lwaldron/Library/Python/2.7/bin/bulker
    /Users/lwaldron/Library/Python/2.7/lib/python/site-packages/bulker-0.3.0-py2.7.egg-info
    /Users/lwaldron/Library/Python/2.7/lib/python/site-packages/bulker/*
Proceed (y/n)? y
  Successfully uninstalled bulker-0.3.0
wallabe:~ lwaldron$ pip install --user bulker
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Collecting bulker
  Using cached https://files.pythonhosted.org/packages/98/36/4808bd1172e39c197cc092093fedb7cc9f1168685446c076c87e3f987364/bulker-0.3.0.tar.gz
Requirement already satisfied: yacman>=0.6.0 in ./Library/Python/2.7/lib/python/site-packages (from bulker) (0.6.4)
Requirement already satisfied: pyyaml>=5.1 in ./Library/Python/2.7/lib/python/site-packages (from bulker) (5.1.2)
Requirement already satisfied: logmuse>=0.2.0 in ./Library/Python/2.7/lib/python/site-packages (from bulker) (0.2.5)
Requirement already satisfied: jinja2 in ./Library/Python/2.7/lib/python/site-packages (from bulker) (2.10.3)
Requirement already satisfied: ubiquerg>=0.4.9 in ./Library/Python/2.7/lib/python/site-packages (from bulker) (0.5.0)
Requirement already satisfied: attmap>=0.12.9 in ./Library/Python/2.7/lib/python/site-packages (from yacman>=0.6.0->bulker) (0.12.11)
Requirement already satisfied: oyaml in ./Library/Python/2.7/lib/python/site-packages (from yacman>=0.6.0->bulker) (0.9)
Requirement already satisfied: MarkupSafe>=0.23 in ./Library/Python/2.7/lib/python/site-packages (from jinja2->bulker) (1.1.1)
Installing collected packages: bulker
    Running setup.py install for bulker ... done
Successfully installed bulker-0.3.0
wallabe:~ lwaldron$ bulker init -c ~/bulker_config.yaml 
Guessing container engine is docker.
Wrote new configuration file: /Users/lwaldron/bulker_config.yaml
wallabe:~ lwaldron$ export BULKERCFG=/Users/lwaldron/bulker_config.yaml
wallabe:~ lwaldron$ bulker load demo
Bulker config: /Users/lwaldron/bulker_config.yaml
Got URL: http://hub.bulker.io/bulker/demo.yaml
Loading manifest: 'bulker/demo:default'. Activate with 'bulker activate bulker/demo:default'.
Commands available: cowsay, fortune
wallabe:~ lwaldron$ bulker activate demo
Bulker config: /Users/lwaldron/bulker_config.yaml
Activating bulker crate: demo

bulker-3.2$ echo $BULKERCFG
/Users/lwaldron/bulker_config.yaml

Not sure this report will be helpful, but if you you know what is happening in the above situation, it would help to add some more advice to the message about BULKERCFG not pointing to a config.

lwaldron commented 5 years ago

Sorry for the convoluted report. It turns out the key here is using export BULKERCFG=/Users/lwaldron/bulker_config.yaml and not just BULKERCFG=/Users/lwaldron/bulker_config.yaml in the .bash_profile, .bashrc, .profile, or whatever. In the second example above you see export on the command-line in the second example where it worked. That's all there is to it, and now I am aware of the difference between using export or not. You could document the importance of using export, or ignore and close the issue.