Upgrading to 2017.12 breaks `q2-dada2`/`dada2`

[jakereps]

Under qiime2=2017.12 q2-dada2 does not work (tested on multiple conda environments, both local (macOS) and remote (Ubuntu 16.04 on EC2). It appears to be missing a GenomeInfoDbData requirement and will fail to load.

Here's an R terminal attempt to load dada2 independent of qiime2 to show the error (same error spit into terminal through q2-dada2):

Unsure if it's something in the q2-dada2 installation that is failing, or if a dependency of dada2 itself has become unreachable? Only relevant issue I could find was this on the forum.

---

edit: this remains as an issue even after conda env update --file 2017.12 from here

[thermokarst]

Thanks @jakereps ! I just tried to reproduce in a fresh macos install, using conda 4.4 and a brand new environment and was not able to raise this same error. What are the contents of the env (conda list)? What version of conda are you using? Do you see the same thing when installing a fresh env? We will get to the bottom of this!

[jakereps]

Currently upgrading (2017.9->2017.12) one more local (macOS 10.13.2) env to see if it works, but the EC2 is:

(biota) jorden@ip-867-53-0-9:~$ conda --version
conda 4.4.7
(biota) jorden@ip-867-53-0-9:~$ conda list
<Attached as gist>[1]
(biota) jorden@ip-867-53-0-9:~$ R -e "library('dada2');"

...
> library('dada2');
Loading required package: Rcpp
Error: package or namespace load failed for ‘dada2’ in loadNamespace(i, c(lib.loc, .libPaths()), versionCheck = vI[[i]]):
 there is no package called ‘GenomeInfoDbData’
Execution halted

[1] https://gist.github.com/jakereps/d24ff42df8a0f137b89c412a872ef9ac

[jakereps]

After updating (macOS 2017.9 -> 2017.12) it breaks dada2 (this is on the macOS 12.13.2 mentioned above, which worked pre-2017.12 update)

(biota) [16:55]:~$ R -e "library('dada2');"

...
> library('dada2');
Loading required package: Rcpp
Error: package or namespace load failed for ‘dada2’ in loadNamespace(i, c(lib.loc, .libPaths()), versionCheck = vI[[i]]):
 there is no package called ‘GenomeInfoDbData’
Execution halted
(biota) [16:55]:~$ 

Full terminal session: https://gist.github.com/jakereps/f728a04ed769ebef33503c0a2cb40f1e

[jakereps]

Update: It works as expected only on brand new environments. All (Ubuntu/macOS) environments it broke under were existing environments being updated.

[17:36]:~$ conda env create -n qiime2-2017.12 --file qiime2-2017.12-py35-osx-conda.yml
...yada yada...
[17:42]:~$ act qiime2-2017.12
(qiime2-2017.12) [17:43]:~$ R -e "require('dada2');"

...
> require('dada2');
Loading required package: dada2
Loading required package: Rcpp
> 
> 

(qiime2-2017.12) [17:43]:~$ 

[thermokarst]

Thanks @jakereps! I don't have a great answer for you, but, we have observed a lot of churn over on the conda issue tracker related to the new 4.4 release --- I wouldn't be surprised if this was somehow related. Stay tuned!

[ebolyen]

I think I know what might be happening here. There is probably a dependency once-removed that added GenomeInfoDBData to a newer version (pretty sure it was just GenomeInfoDB). On bioconda, the bioconductor packages don't seem to have a way to ensure they are installed alongside the other release in their respective train-release, so conda figures out update plan which keeps some of your old dependencies, but upgrades others. These are incompatible and lead to the missing dep above.

What doesn't really make sense is why the environment file isn't updating everything since I can see bioconductor-genomeinfodbdata listed in there. What does conda list on a broken environment show?

[jakereps]

@ebolyen I did notice that running it on a newer environment (not the huge .9->.12 jump) the installation list was relatively short, so it does seem to be piecewise updates. The first gist is of a broken env.

[jakereps]

@thermokarst The macOS update error is in 4.3 here, so it seems to occur in both 4.3 and 4.4. 4.3.25 failed to upgrade to 2017.12 entirely because it couldn't find mkl I guess?