search

bcbio / bcbio-nextgen

Validated, scalable, community developed variant calling, RNA-seq and small RNA analysis
https://bcbio-nextgen.readthedocs.io
MIT License
986 stars 354 forks source link

Mutect 1.1.7 installation error #2861

Closed Fazulur closed 5 years ago

Fazulur commented 5 years ago

Dear Bcbio team,

I tried installing Mutect 1.1.7 and gatk 3.4 as per below instructions under section "GATK and MuTect/MuTect" https://bcbio-nextgen.readthedocs.io/en/latest/contents/installation.html

Commands bcbio_nextgen.py upgrade --tools --toolplus gatk=$HOME/gatkv3.4/GenomeAnalysisTK.jar bcbio_nextgen.py upgrade --tools --toolplus mutect=/path/to/mutect/mutect-1.1.7.jar

And set BCBIO_JAVA_HOME=$HOME/java1.7.0/

I am getting the below error while installing Mutect 1.1.7

File "BCBIO/v1.1.5/bin/bcbio_nextgen.py", line 221, in     install.upgrade_bcbio(kwargs["args"])   File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/bcbio/install.py", line 94, in upgrade_bcbio     _install_toolplus(args)   File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/bcbio/install.py", line 520, in _install_toolplus     _install_gatk_jar(tool.name, tool.fname, toolplus_manifest, system_config, toolplus_dir)   File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/bcbio/install.py", line 537, in _install_gatk_jar     version = get_gatk_jar_version(name, fname)   File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/bcbio/install.py", line 528, in get_gatk_jar_version     return broad.get_mutect_version(fname)   File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/bcbio/broad/init.py", line 116, in get_mutect_version     if "SomaticIndelDetector" in stdout.read().strip(): TypeError: a bytes-like object is required, not 'str'

Could you please help me to resolve this issue.

Thanks In Advance Fazulur Rehaman

roryk commented 5 years ago

Hi Fazulur,

Sorry about that-- we have been hunting down these issues, they are a byproduct of switching over to python3. I pushed a fix for this one and for another one you're likely to run into in the same file. If you update the code to latest development by doing:

bcbio_nextgen.py upgrade -u development

you should pull in this fix. Let me know if this fixes the problem for you!

Fazulur commented 5 years ago

Dear Rory,

Thanks a lot for your quick response. I updated bcbio to latest development. After setting BCBIO_JAVA_HOME=$HOME/software/tools/java/1.7.0/, I ran mutect job again It is giving the same error.

Here is sample yaml file

details:

And error log:

[2019-06-24T06:24Z] hpcsdcn31: Testing minimum versions of installed programs [2019-06-24T06:24Z] hpcsdcn31: Unexpected error Traceback (most recent call last): File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/bcbio/distributed/ipythontasks.py", line 52, in _setup_logging yield config File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/bcbio/distributed/ipythontasks.py", line 541, in organize_samples return ipython.zip_args(apply(run_info.organize, args)) File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/bcbio/distributed/ipythontasks.py", line 80, in apply return object(args, **kwargs) File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/bcbio/pipeline/run_info.py", line 93, in organize out = _add_provenance(out, dirs, config, not is_cwl) File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/bcbio/pipeline/run_info.py", line 106, in _add_provenance versioncheck.testall(items) File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/bcbio/provenance/versioncheck.py", line 91, in testall out = fn(items) File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/bcbio/provenance/versioncheck.py", line 73, in java for line in output.split("\n"): TypeError: a bytes-like object is required, not 'str'

Please suggest me how can I proceed further.

Thanks In Advance Fazulur Rehaman

chapmanb commented 5 years ago

Fazulur; Sorry about the additional problems. I pushed a fix for this python 3 issue as well, if you can update from development and try again. Hopefully this gets it all working cleanly for you, and please let us know if not. Thanks for helping debug.

Fazulur commented 5 years ago

Dear Brad,

Thanks a lot for your quick response & suggestions.

I set BCBIO_JAVA_HOME=$HOME/software/tools/java/1.7.0/ and tried to run mutect. Still it is picking java 1.8 when i run mutect. Please help me in setting BCBIO_JAVA_HOME variable to path.

Traceback (most recent call last): File "BCBIO/v1.1.5/bin/bcbio_nextgen.py", line 238, in main(kwargs) File "BCBIO/v1.1.5/bin/bcbio_nextgen.py", line 46, in main run_main(kwargs) File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/bcbio/pipeline/main.py", line 53, in run_main fc_dir, run_info_yaml) File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/bcbio/pipeline/main.py", line 89, in _run_toplevel for xs in pipeline(config, run_info_yaml, parallel, dirs, samples): File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/bcbio/pipeline/main.py", line 126, in variant2pipeline [x[0]["description"] for x in samples]]]) File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/bcbio/distributed/ipython.py", line 137, in run for data in view.map_sync(fn, items, track=False): File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/ipyparallel/client/view.py", line 344, in map_sync return self.map(f,*sequences,kwargs) File "<BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/decorator.py:decorator-gen-139>", line 2, in map File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/ipyparallel/client/view.py", line 52, in sync_results ret = f(self, *args, *kwargs) File "<BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/decorator.py:decorator-gen-138>", line 2, in map File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/ipyparallel/client/view.py", line 37, in save_ids ret = f(self, args, kwargs) File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/ipyparallel/client/view.py", line 1114, in map return pf.map(sequences) File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/ipyparallel/client/remotefunction.py", line 299, in map return self(sequences, __ipp_mapping=True) File "<BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/decorator.py:decorator-gen-121>", line 2, in call File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/ipyparallel/client/remotefunction.py", line 80, in sync_view_results return f(self, *args, **kwargs) File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/ipyparallel/client/remotefunction.py", line 285, in call return r.get() File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/ipyparallel/client/asyncresult.py", line 169, in get raise self.exception() File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/ipyparallel/client/asyncresult.py", line 228, in _resolve_result results = error.collect_exceptions(results, self._fname) File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/ipyparallel/error.py", line 233, in collect_exceptions raise e File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/ipyparallel/error.py", line 231, in collect_exceptions raise CompositeError(msg, elist) ipyparallel.error.CompositeError: one or more exceptions from call to method: organize_samples [0:apply]: OSError: Program problems found. You can upgrade dependencies with: bcbio_nextgen.py upgrade -u skip --tooldir=/usr/local

java version 1.7 required for running MuTect and GATK < 3.6. It needs to be first on your PATH so running 'java -version' give the correct version. Found version 1.8.0_192 at BCBIO/v1.1.5/anaconda/bin/java

And also after upgrade to development version, I am getting peddy error when running gatk-variant pipeline.

[2019-06-25T11:11Z] local: Running peddy on work/gatk-haplotype/test.vcf.gz against work test..ped. [2019-06-25T11:11Z] local: Uncaught exception occurred Traceback (most recent call last): File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/bcbio/provenance/do.py", line 26, in run _do_run(cmd, checks, log_stdout, env=env) File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/bcbio/provenance/do.py", line 106, in _do_run raise subprocess.CalledProcessError(exitcode, error_msg) subprocess.CalledProcessError: Command 'set -o pipefail; BCBIO/v1.1.5/anaconda/bin/peddy -p 8 --plot --prefix test work/gatk-haplotype/test.ped 2> work/bcbiotx/tmple0m57jt/run-stderr.log ' returned non-zero exit status 1. [2019-06-25T11:11Z] local: Traceback (most recent call last): File "BCBIO/v1.1.5/anaconda/bin/peddy", line 11, in load_entry_point('peddy==0.4.3', 'console_scripts', 'peddy')() File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/click/core.py", line 764, in call return self.main(*args, **kwargs) File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/click/core.py", line 696, in main _verify_python3_env() File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/click/_unicodefun.py", line 124, in _verify_python3_env ' mitigation steps.' + extra RuntimeError: Click will abort further execution because Python 3 was configured to use ASCII as encoding for the environment. Consult https://click.palletsprojects.com/en/7.x/python3/ for mitigation steps.

This system lists a couple of UTF-8 supporting locales that you can pick from.

Could you please help me to resolve this issue. Thanks In Advance Fazulur Rehaman

roryk commented 5 years ago

Hi Fazulur,

I pushed a fix for this issue with peddy. Regarding the java issue, if you track down the java binary for java 7 and add it first in your PATH before running bcbio, it should find the correct java version.

Fazulur commented 5 years ago

Dear Rory,

Thanks a lot for your quick response.

When I run mutect on my login node. it is able to pickup BCBIO_JAVA_HOME. But when I run with "-s lsf -q normal", it is not exporting java 1.7. On execution node, still it is capturing java 1.8 from anaconda libraries. Eg: mutect working when I run bcbio locally bcbio_nextgen.py test.yaml -n 100

But with the below command it is giving same error bcbio_nextgen.py test.yaml -n 100 --timeout 20000 -s lsf -q normal -r P=test

And, while upgrading bcbio to latest development, I am getting the below error.

ERROR conda.core.link:_execute(637): An error occurred while installing package 'None'. AssertionError() Attempting to roll back. Rolling back transaction: done AssertionError() () Traceback (most recent call last):   File "BCBIO/v1.1.5/bin/bcbio_nextgen.py", line 221, in     install.upgrade_bcbio(kwargs["args"])   File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/bcbio/install.py", line 90, in upgrade_bcbio     upgrade_thirdparty_tools(args, REMOTES)   File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/bcbio/install.py", line 495, in upgrade_thirdparty_tools     cbl_conda.install_in(_get_conda_bin(), args.tooldir, package_yaml)   File "/gpfs/home/appsadmin/tmpbcbio-install/cloudbiolinux/cloudbio/package/conda.py", line 69, in install_in     subprocess.check_call("{conda_bin} remove -y {problems}".format(**locals()), shell=True)   File "BCBIO/v1.1.5/anaconda/lib/python3.6/subprocess.py", line 291, in check_call     raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command 'BCBIO/v1.1.5/anaconda/bin/conda remove -y hisat2' returned non-zero exit status 1.

Could you please help me to resolve issues.

Thanks In Advance Fazulur Rehaman

chapmanb commented 5 years ago

Fazulur; Thanks for the work debugging and the detailed feedback. For the java 7 PATH and BCBIO_JAVA_HOME error, this sounds like something with your lsf setup where either:

Hopefully this gives some ideas for helping debug why the environment is different on submitted jobs.

For the conda issue, please try running the failing command outside of bcbio:

BCBIO/v1.1.5/anaconda/bin/conda remove -y hisat2

This should hopefully give some useful error messages about why it fails. Hope this helps.

Fazulur commented 5 years ago

Dear Brad,

Thanks a lot for your quick response.

Here is the full error. It is complaining about missing python module.

Error processing line 1 of BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/protobuf-3.6.1-py3.6-nspkg.pth:   Traceback (most recent call last):     File "BCBIO/v1.1.5/anaconda/lib/python3.6/site.py", line 168, in addpackage       exec(line)     File "", line 1, in     File "", line 568, in module_from_spec   AttributeError: 'NoneType' object has no attribute 'loader' Remainder of file ignored Traceback (most recent call last):   File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/conda/gateways/connection/init.py", line 21, in     from requests import ConnectionError, HTTPError, Session ModuleNotFoundError: No module named 'requests' During handling of the above exception, another exception occurred: Traceback (most recent call last):   File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/conda/exceptions.py", line 1043, in call     return func(*args, kwargs)   File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/conda/cli/main.py", line 84, in _main     exit_code = do_call(args, p)   File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/conda/cli/conda_argparse.py", line 80, in do_call     module = import_module(relative_mod, name.rsplit('.', 1)[0])   File "BCBIO/v1.1.5/anaconda/lib/python3.6/importlib/init.py", line 126, in import_module     return _bootstrap._gcd_import(name[level:], package, level)   File "", line 994, in _gcd_import   File "", line 971, in _find_and_load   File "", line 955, in _find_and_load_unlocked   File "", line 665, in _load_unlocked   File "", line 678, in exec_module   File "", line 219, in _call_with_frames_removed   File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/conda/cli/main_remove.py", line 10, in     from .install import handle_txn   File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/conda/cli/install.py", line 19, in     from ..core.index import calculate_channel_urls, get_index   File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/conda/core/index.py", line 9, in     from .package_cache_data import PackageCacheData   File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/conda/core/package_cache_data.py", line 17, in     from .path_actions import CacheUrlAction, ExtractPackageAction   File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/conda/core/path_actions.py", line 30, in     from ..gateways.connection.download import download   File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/conda/gateways/connection/init.py", line 38, in     from pip._vendor.requests import ConnectionError, HTTPError, Session ModuleNotFoundError: No module named 'pip._vendor.requests' During handling of the above exception, another exception occurred: Traceback (most recent call last):   File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/conda/gateways/connection/init.py", line 21, in     from requests import ConnectionError, HTTPError, Session ModuleNotFoundError: No module named 'requests' During handling of the above exception, another exception occurred: Traceback (most recent call last):   File "BCBIO/v1.1.5/anaconda/bin/conda", line 13, in     sys.exit(main())   File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/conda/cli/main.py", line 150, in main     return conda_exception_handler(_main, *args, *kwargs)   File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/conda/exceptions.py", line 1335, in conda_exception_handler     return_value = exception_handler(func, args, kwargs)   File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/conda/exceptions.py", line 1046, in call     return self.handle_exception(exc_val, exc_tb)   File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/conda/exceptions.py", line 1090, in handle_exception     return self.handle_unexpected_exception(exc_val, exc_tb)   File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/conda/exceptions.py", line 1101, in handle_unexpected_exception     self.print_unexpected_error_report(error_report)   File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/conda/exceptions.py", line 1171, in print_unexpected_error_report     from .cli.main_info import get_env_vars_str, get_main_info_str   File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/conda/cli/main_info.py", line 19, in     from ..core.index import _supplement_index_with_system   File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/conda/core/index.py", line 9, in     from .package_cache_data import PackageCacheData   File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/conda/core/package_cache_data.py", line 17, in     from .path_actions import CacheUrlAction, ExtractPackageAction   File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/conda/core/path_actions.py", line 30, in     from ..gateways.connection.download import download   File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/conda/gateways/connection/init.py", line 38, in     from pip._vendor.requests import ConnectionError, HTTPError, Session ModuleNotFoundError: No module named 'pip._vendor.requests'

Please help me to resolve this issue.

Thanks In Advance Fazulur Rehaman

chapmanb commented 5 years ago

Fazulur; Sorry about the additional problems. I'm not sure what happened here or how best to resolve. The error messages indicate something is problematic in your install itself since it's apparently failing trying to import the requests module with an apparent loader error. My best idea would be to try uninstalling and reinstalling requests. It's not clear if conda itself is broken but something like:

bcbio_conda uninstall requests
bcbio_conda install -c conda-forge -y requests

could get you a clean install. Sorry this is so difficult to debug remotely but hope this helps.

Fazulur commented 5 years ago

Dear Brad,

We tried the above commands and still we are having the same issues.

And we stopped troubleshooting this and will use Mutect outside of bcbio, since it is creating issues.

After Reverted back the bcbio v1.1.5 version which was working before. It is giving below error while running gatk-variant pipeline.

File "BCBIO/v1.1.5/anaconda/lib/python3.6/distutils/version.py", line 337, in _cmp if self.version < other.version: TypeError: '<' not supported between instances of 'str' and 'int'

Then, we tried to upgrade bcbio to development version, it is giving new error. bcbio_nextgen.py upgrade -u development

Please suggest me how can I resolve this.

Thanks In Advance Fazulur Rehaman

roryk commented 5 years ago

Hi Fazulur,

These are bugs that we fixed in the development version-- is there a reason you can't use that?

Fazulur commented 5 years ago

Hi Rory, Thanks a lot for your quick response.

I am sorry, we are getting additional issues after upgrade to development version. "ModuleNotFoundError: No module named 'pip._vendor.requests'"

Brad suggested to try the below bcbio_conda uninstall requests bcbio_conda install -c conda-forge -y requests

After trying this, again we are getting the same "module errors". Thatswhy we thought of using the backuped verison (v1.1.5) which was working before without issues for time being. Now we are having "Typeerror" issue with this as well.

Please suggest me how can I proceed further.

Thanks In Advance Fazulur Rehaman

roryk commented 5 years ago

Thanks, something seems messed up your with your install, but it is hard to tell without seeing the whole error, if you could paste in the entire thing it would be helpful. I'm guessing that somewhere another python installation on your machine is leaking into your PATH which is causing these problems.

Fazulur commented 5 years ago

Hi Rory,

Here is the full error while upgrade to development version.

$bcbio_nextgen.py upgrade -u development

Upgrading bcbio   Collecting package metadata (current_repodata.json): ...working... done Solving environment: ...working...  The environment is inconsistent, please check the package plan carefully The following packages are causing the inconsistency:     - defaults/linux-64::numpy-base==1.15.4=py27h2f8d375_0 failed Collecting package metadata (repodata.json): ...working... done Solving environment: ...working...  The environment is inconsistent, please check the package plan carefully The following packages are causing the inconsistency:     - defaults/linux-64::numpy-base==1.15.4=py27h2f8d375_0 failed   UnsatisfiableError: The following specifications were found to be incompatible with each other:     - bioconda/linux-64::bioconductor-biovizbase==1.30.1=r351h14c3975_0 -> bioconductor-ensembldb[version='>=2.6.0,<2.7.0'] -> bioconductor-genomicfeatures[version='>=1.34.0,<1.35.0'] -> bioconductor-biomart[version='>=2.38.0,<2.39.0'] -> r-httr -> r-curl[version='>=0.9.1']   - bioconda/linux-64::bioconductor-biovizbase==1.30.1=r351h14c3975_0 -> bioconductor-ensembldb[version='>=2.6.0,<2.7.0'] -> bioconductor-genomicfeatures[version='>=1.34.0,<1.35.0'] -> bioconductor-biomart[version='>=2.38.0,<2.39.0'] -> r-httr -> r-r6   - bioconda/linux-64::bioconductor-titancna==1.20.1=r351h14c3975_0 -> bioconductor-variantannotation[version='>=1.28.0,<1.29.0'] -> bioconductor-genomicfeatures[version='>=1.34.0,<1.35.0'] -> bioconductor-biomart[version='>=2.38.0,<2.39.0'] -> r-httr -> r-curl[version='>=0.9.1']   - bioconda/linux-64::bioconductor-titancna==1.20.1=r351h14c3975_0 -> bioconductor-variantannotation[version='>=1.28.0,<1.29.0'] -> bioconductor-genomicfeatures[version='>=1.34.0,<1.35.0'] -> bioconductor-biomart[version='>=2.38.0,<2.39.0'] -> r-httr -> r-r6   - bioconda/linux-64::bioconductor-titancna==1.20.1=r351h14c3975_0 -> r-foreach[version='>=1.4.3']   - bioconda/linux-64::bioconductor-variantannotation==1.28.3=r351h14c3975_0 -> bioconductor-genomicfeatures[version='>=1.34.0,<1.35.0'] -> bioconductor-biomart[version='>=2.38.0,<2.39.0'] -> r-httr -> r-curl[version='>=0.9.1']   - bioconda/linux-64::bioconductor-variantannotation==1.28.3=r351h14c3975_0 -> bioconductor-genomicfeatures[version='>=1.34.0,<1.35.0'] -> bioconductor-biomart[version='>=2.38.0,<2.39.0'] -> r-httr -> r-r6   - bioconda/linux-64::gdk-pixbuf==2.32.2=1   - bioconda/linux-64::r-basejump==0.7.2=r351_1 -> bioconductor-annotationhub -> bioconductor-interactivedisplaybase[version='>=1.20.0,<1.21.0'] -> r-shiny -> r-httpuv[version='>=1.5.0'] -> r-promises -> r-r6   - bioconda/linux-64::r-basejump==0.7.2=r351_1 -> bioconductor-annotationhub -> r-curl   - bioconda/linux-64::r-bcbiobase==0.4.1=r351_1 -> r-basejump[version='>=0.7.2'] -> r-brio[version='>=0.2.1'] -> r-curl[version='>=1.95']   - bioconda/linux-64::r-bcbiobase==0.4.1=r351_1 -> r-basejump[version='>=0.7.2'] -> r-brio[version='>=0.2.1'] -> r-jsonlite[version='>=1.6']   - bioconda/linux-64::r-bcbiobase==0.4.1=r351_1 -> r-basejump[version='>=0.7.2'] -> r-brio[version='>=0.2.1'] -> r-readr[version='>=1.3'] -> r-r6   - bioconda/linux-64::r-bcbiornaseq==0.2.8=r351_0 -> r-bcbiobase -> r-basejump==0.7.2 -> bioconductor-annotationhub -> bioconductor-interactivedisplaybase[version='>=1.20.0,<1.21.0'] -> r-shiny -> r-httpuv[version='>=1.5.0'] -> r-promises -> r-r6   - bioconda/linux-64::r-bcbiornaseq==0.2.8=r351_0 -> r-bcbiobase -> r-basejump==0.7.2 -> bioconductor-annotationhub -> r-curl   - bioconda/linux-64::r-brio==0.1.8=r351_0 -> r-curl[version='>=1.95']   - bioconda/linux-64::r-brio==0.1.8=r351_0 -> r-jsonlite[version='>=1.6']   - bioconda/linux-64::r-brio==0.1.8=r351_0 -> r-readr[version='>=1.3'] -> r-r6   - bioconda/linux-64::r-goalie==0.2.9=r351_0 -> r-curl[version='>=3.2.*']   - bioconda/linux-64::r-sleuth==0.30.0=r351_1 -> r-dplyr -> r-r6   - bioconda/linux-64::r-syntactic==0.1.5=r351_0 -> r-goalie[version='>=0.2.9'] -> r-curl[version='>=3.2']   - bioconda/linux-64::r-transformer==0.1.6=r351_0 -> r-goalie[version='>=0.2.9'] -> r-curl[version='>=3.2']   - bioconda/noarch::bioconductor-annotationhub==2.14.2=r351_0 -> bioconductor-interactivedisplaybase[version='>=1.20.0,<1.21.0'] -> r-shiny -> r-httpuv[version='>=1.5.0'] -> r-promises -> r-r6   - bioconda/noarch::bioconductor-annotationhub==2.14.2=r351_0 -> r-curl   - bioconda/noarch::bioconductor-biomart==2.38.0=r351_0 -> r-httr -> r-curl[version='>=0.9.1']   - bioconda/noarch::bioconductor-biomart==2.38.0=r351_0 -> r-httr -> r-r6   - bioconda/noarch::bioconductor-bubbletree==2.12.0=r351_0 -> bioconductor-biovizbase[version='>=1.30.0,<1.31.0'] -> bioconductor-ensembldb[version='>=2.6.0,<2.7.0'] -> bioconductor-genomicfeatures[version='>=1.34.0,<1.35.0'] -> bioconductor-biomart[version='>=2.38.0,<2.39.0'] -> r-httr -> r-curl[version='>=0.9.1']   - bioconda/noarch::bioconductor-bubbletree==2.12.0=r351_0 -> bioconductor-biovizbase[version='>=1.30.0,<1.31.0'] -> bioconductor-ensembldb[version='>=2.6.0,<2.7.0'] -> bioconductor-genomicfeatures[version='>=1.34.0,<1.35.0'] -> bioconductor-biomart[version='>=2.38.0,<2.39.0'] -> r-httr -> r-r6   - bioconda/noarch::bioconductor-clusterprofiler==3.10.1=r351_0 -> bioconductor-enrichplot[version='>=1.2.0,<1.3.0'] -> r-europepmc -> r-dplyr -> r-r6   - bioconda/noarch::bioconductor-clusterprofiler==3.10.1=r351_0 -> bioconductor-enrichplot[version='>=1.2.0,<1.3.0'] -> r-europepmc -> r-httr -> r-curl[version='>=0.9.1']   - bioconda/noarch::bioconductor-degreport==1.19.1=r351_0 -> r-broom -> r-dplyr -> r-r6   - bioconda/noarch::bioconductor-dexseq==1.28.0=r351_0 -> bioconductor-biomart[version='>=2.38.0,<2.39.0'] -> r-httr -> r-curl[version='>=0.9.1']   - bioconda/noarch::bioconductor-dexseq==1.28.0=r351_0 -> bioconductor-biomart[version='>=2.38.0,<2.39.0'] -> r-httr -> r-r6   - bioconda/noarch::bioconductor-enrichplot==1.2.0=r351_0 -> r-europepmc -> r-dplyr -> r-r6   - bioconda/noarch::bioconductor-enrichplot==1.2.0=r351_0 -> r-europepmc -> r-httr -> r-curl[version='>=0.9.1']   - bioconda/noarch::bioconductor-ensembldb==2.6.3=r351_0 -> bioconductor-genomicfeatures[version='>=1.34.0,<1.35.0'] -> bioconductor-biomart[version='>=2.38.0,<2.39.0'] -> r-httr -> r-curl[version='>=0.9.1']   - bioconda/noarch::bioconductor-ensembldb==2.6.3=r351_0 -> bioconductor-genomicfeatures[version='>=1.34.0,<1.35.0'] -> bioconductor-biomart[version='>=2.38.0,<2.39.0'] -> r-httr -> r-r6   - bioconda/noarch::bioconductor-genomicfeatures==1.34.1=r351_0 -> bioconductor-biomart[version='>=2.38.0,<2.39.0'] -> r-httr -> r-curl[version='>=0.9.1']   - bioconda/noarch::bioconductor-genomicfeatures==1.34.1=r351_0 -> bioconductor-biomart[version='>=2.38.0,<2.39.0'] -> r-httr -> r-r6   - bioconda/noarch::bioconductor-interactivedisplaybase==1.20.0=r351_0 -> r-shiny -> r-httpuv[version='>=1.5.0'] -> r-promises -> r-r6   - bioconda/noarch::bioconductor-oligoclasses==1.44.0=r351_0 -> r-foreach   - bioconda/noarch::bioconductor-purecn==1.12.1=r351_0 -> bioconductor-genomicfeatures[version='>=1.34.0,<1.35.0'] -> bioconductor-biomart[version='>=2.38.0,<2.39.0'] -> r-httr -> r-curl[version='>=0.9.1']   - bioconda/noarch::bioconductor-purecn==1.12.1=r351_0 -> bioconductor-genomicfeatures[version='>=1.34.0,<1.35.0'] -> bioconductor-biomart[version='>=2.38.0,<2.39.0'] -> r-httr -> r-r6   - bioconda/noarch::bioconductor-snpchip==2.28.0=r351_0 -> bioconductor-oligoclasses[version='>=1.44.0,<1.45.0'] -> r-foreach   - conda-forge/linux-64::gtk2==2.24.31=h5baeb44_1000 -> gdk-pixbuf   - conda-forge/linux-64::libarchive==3.3.3=hc3f970e_1004 -> zstd[version='>=1.3.3,<1.3.4.0a0']   - conda-forge/linux-64::r-curl==3.3=r351h96ca727_0   - conda-forge/linux-64::r-dplyr==0.8.0.1=r351h29659fb_1000 -> r-r6[version='>=2.2.2']   - conda-forge/linux-64::r-ggforce==0.2.1=r351h29659fb_0 -> r-ggplot2[version='>=3.0.0'] -> r-scales[version='>=0.5.0'] -> r-r6   - conda-forge/linux-64::r-ggraph==1.0.2=r351h29659fb_1002 -> r-dplyr -> r-r6   - conda-forge/linux-64::r-httpuv==1.4.5.1=r351hf484d3e_1000 -> r-promises -> r-r6   - conda-forge/linux-64::r-jsonlite==1.6=r351h96ca727_1000   - conda-forge/linux-64::r-pkgload==1.0.2=r351h29659fb_1000 -> r-desc -> r-r6   - conda-forge/linux-64::r-processx==3.3.0=r351hcdcec82_0 -> r-r6   - conda-forge/linux-64::r-promises==1.0.1=r351h29659fb_1000 -> r-r6   - conda-forge/linux-64::r-readr==1.3.1=r351h29659fb_1000 -> r-r6   - conda-forge/linux-64::r-readxl==1.3.1=r351h0357c0b_0 -> r-progress -> r-r6   - conda-forge/linux-64::r-scales==1.0.0=r351h29659fb_1001 -> r-r6   - conda-forge/linux-64::r-tidyr==0.8.3=r351h0357c0b_0 -> r-dplyr[version='>=0.7.0'] -> r-r6   - conda-forge/linux-64::zstd==1.3.3=1   - conda-forge/noarch::r-broom==0.5.2=r351h6115d3f_0 -> r-dplyr -> r-r6   - conda-forge/noarch::r-callr==3.2.0=r351h6115d3f_0 -> r-processx[version='>=3.2.1'] -> r-r6   - conda-forge/noarch::r-crosstalk==1.0.0=r351h6115d3f_1001 -> r-r6   - conda-forge/noarch::r-dbplyr==1.3.0=r351h6115d3f_1000 -> r-dplyr[version='>=0.7.5'] -> r-r6   - conda-forge/noarch::r-desc==1.2.0=r351h6115d3f_1001 -> r-r6   - conda-forge/noarch::r-devtools==2.0.2=r351h6115d3f_0 -> r-httr[version='>=0.4'] -> r-curl[version='>=0.9.1']   - conda-forge/noarch::r-devtools==2.0.2=r351h6115d3f_0 -> r-httr[version='>=0.4'] -> r-r6   - conda-forge/noarch::r-domc==1.3.5=r351_1000 -> r-foreach[version='>=1.2.0']   - conda-forge/noarch::r-dt==0.5=r351h6115d3f_0 -> r-crosstalk -> r-r6   - conda-forge/noarch::r-europepmc==0.3=r351h6115d3f_1000 -> r-dplyr -> r-r6   - conda-forge/noarch::r-europepmc==0.3=r351h6115d3f_1000 -> r-httr -> r-curl[version='>=0.9.1']   - conda-forge/noarch::r-foreach==1.4.4=r351h6115d3f_1001   - conda-forge/noarch::r-ggplot2==3.1.1=r351h6115d3f_0 -> r-scales[version='>=0.5.0'] -> r-r6   - conda-forge/noarch::r-gh==1.0.1=r351h6115d3f_1001 -> r-httr -> r-curl[version='>=0.9.1']   - conda-forge/noarch::r-gh==1.0.1=r351h6115d3f_1001 -> r-httr -> r-r6   - conda-forge/noarch::r-httr==1.4.0=r351h6115d3f_1000 -> r-curl[version='>=0.9.1']   - conda-forge/noarch::r-httr==1.4.0=r351h6115d3f_1000 -> r-r6   - conda-forge/noarch::r-modelr==0.1.4=r351h6115d3f_0 -> r-broom -> r-dplyr -> r-r6   - conda-forge/noarch::r-pkgbuild==1.0.3=r351h6115d3f_0 -> r-callr[version='>=2.0.0'] -> r-processx[version='>=3.2.1'] -> r-r6   - conda-forge/noarch::r-progress==1.2.0=r351h6115d3f_1002 -> r-r6   - conda-forge/noarch::r-r6==2.4.0=r351h6115d3f_0   - conda-forge/noarch::r-rcmdcheck==1.3.2=r351h6115d3f_1000 -> r-callr[version='>=2.0.0'] -> r-processx[version='>=3.2.1'] -> r-r6   - conda-forge/noarch::r-rdrop2==0.8.1=r351h6115d3f_1001 -> r-dplyr -> r-r6   - conda-forge/noarch::r-rdrop2==0.8.1=r351h6115d3f_1001 -> r-httr -> r-curl[version='>=0.9.1']   - conda-forge/noarch::r-reprex==0.2.1=r351h6115d3f_1000 -> r-callr[version='>=2.0.0'] -> r-processx[version='>=3.2.1'] -> r-r6   - conda-forge/noarch::r-rio==0.5.16=r351h6115d3f_1000 -> r-curl[version='>=0.6']   - conda-forge/noarch::r-rvest==0.3.2=r351h6115d3f_1001 -> r-httr[version='>=0.5'] -> r-curl[version='>=0.9.1']   - conda-forge/noarch::r-rvest==0.3.2=r351h6115d3f_1001 -> r-httr[version='>=0.5'] -> r-r6   - conda-forge/noarch::r-selectr==0.4_1=r351h6115d3f_1000 -> r-r6   - conda-forge/noarch::r-shiny==1.2.0=r351_1000 -> r-httpuv[version='>=1.4.4'] -> r-promises -> r-r6   - conda-forge/noarch::r-tidyverse==1.2.1=r351h6115d3f_1001 -> r-broom[version='>=0.4.2'] -> r-dplyr -> r-r6   - conda-forge/noarch::r-tidyverse==1.2.1=r351h6115d3f_1001 -> r-httr[version='>=1.3.1'] -> r-curl[version='>=0.9.1']   - conda-forge/noarch::r-usethis==1.5.0=r351h6115d3f_0 -> r-curl[version='>=2.7']   - conda-forge/noarch::r-usethis==1.5.0=r351h6115d3f_0 -> r-desc -> r-r6   - conda-forge/noarch::r-xopen==1.0.0=r351h6115d3f_1000 -> r-processx -> r-r6   - conda-forge/noarch::r==3.5.1=r351_1000       Traceback (most recent call last):   File "BCBIO/v1.1.5/bin/bcbio_nextgen.py", line 221, in     install.upgrade_bcbio(kwargs["args"])   File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/bcbio/install.py", line 60, in upgrade_bcbio     anaconda_dir = _update_conda_devel()   File "BCBIO/v1.1.5/anaconda/lib/python3.6/site-packages/bcbio/install.py", line 297, in _update_conda_devel     ["bcbio-nextgen>=%s" % version.version.replace("a0", "a")])   File "BCBIO/v1.1.5/anaconda/lib/python3.6/subprocess.py", line 291, in check_call     raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['BCBIO/v1.1.5/anaconda/bin/conda', 'install', '--quiet', '--yes', 'bcbio-nextgen>=1.1.5a']' returned non-zero exit status 1.

Please help me to resolve this issue.

Thanks In Advance Fazulur Rehaman

chapmanb commented 5 years ago

Fazulur; Sorry about these upgrade issues. That's a difficult one, as it looks like conda got in an unhappy state somehow. The initial error about numpy and py27 indicates this might have happened due to the python 2 to 3 transition, and now something is out of sync. What was the upgrade path for this install?

In these cases the easiest approach is often just to re-install bcbio and the tools, leaving the data in place. You can move the existing anaconda directory and then run the installer to add them back in reusing the rest of the install:

mv /path/to/bcbio/anaconda /path/to/bcbio/anaconda.orig
python bcbio_nextgen_install.py /path/to/bcbio --tooldir /path/to/tooldir -u development

Hope this gets it installing cleanly for you.

roryk commented 5 years ago

Thanks, we had a bunch of problems with installation of tools around this time that should be cleaned up now. I've done a full install and upgrade a few days ago and everything is all set now. Hopefully this is no longer an issue, please re-open if this isn't resolved for you.