build can't find devtools?

[juliasilge]

The commits I've pushed this morning have all failed on AppVeyor; it looks like this is because it can't find devtools?

Rscript -e 'library(devtools); library(methods); options(repos=c(CRAN="https://cran.rstudio.com")); install_deps(dependencies = c("Depends", "Imports", "LinkingTo", "Suggests"))'
Error in library(devtools) : there is no package called 'devtools'
Execution halted
Command exited with code 1
7z a failure.zip *.Rcheck\*

Here is the link: https://ci.appveyor.com/project/juliasilge/tidytext/build/1.0.41

There is a new version of devtools; I expect this is related somehow?

[juliasilge]

Wait, is this just because the binary for the new version of devtools isn't built on CRAN yet?

[krlmlr]

Yes, I think this will auto-resolve tomorrow. It would work if Rtools was installed for this build, but it's not because you don't have compiled code. You can force it, but this will slow regular builds.

[juliasilge]

Thanks so much for the response! I tried pushing commits today, and now the build can find devtools but it says that it can't install my actual package. There is a new version of my package as of Saturday but all the binaries appear to be built: https://cran.r-project.org/web/packages/tidytext/index.html This is the message I am getting when the AppVeyor build fails:

checking whether package 'tidytext' can be installed ...Warning: running command '"c:/R/bin/i386/Rcmd.exe" INSTALL -l "C:/projects/tidytext/tidytext.Rcheck" --build --no-html --no-multiarch "C:\projects\tidytext\TIDYTE~1.RCH\00_PKG~1\tidytext"' had status 1
 ERROR
Installation failed.
See 'C:/projects/tidytext/tidytext.Rcheck/00install.out' for details.

Is this because of my new version on CRAN? Do I wait longer again? Other thoughts? I appreciate your help a lot.

Here is the link to AppVeyor: https://ci.appveyor.com/project/juliasilge/tidytext/

[juliasilge]

I've been staring at the AppVeyor logs trying to find out what is going on here because I am now waiting on AppVeyor to pass to do the GitHub release for the new version of our package. I have been comparing the last build that passed on AppVeyor: https://ci.appveyor.com/project/juliasilge/tidytext/build/1.0.39 To any of the ones after that: https://ci.appveyor.com/project/juliasilge/tidytext/build/1.0.50 Nothing looks very different right before the failure; the two logs do look different when the dependencies are being installed, I guess because of the new version of devtools.

I just ran build_win() again, to see if the package really couldn't be installed on Windows, but build_win() came back completely clean.

[krlmlr]

For some reason, the second build doesn't install the DBI package which seems necessary. The artifacts of failed builds have a zip file of the check directory.

If winbuilder succeeds, I think it's ok to release to CRAN.

[juliasilge]

Thanks so much for helping me with this! I see that dplyr imports DBI , and my package imports dplyr. I am not sure why DBI is not being installed on the AppVeyor build without me telling it specifically to do this since it is a dependency of a dependency. Is there some way for me to force the AppVeyor build to install DBI like there is on Travis?

-r_binary_packages:

And then whatever I want to install? Or something else you think I should try?

(Just FYI, this morning I did try deleting the whole project on AppVeyor and then adding it back in, but it still failed.)

[krlmlr]

You can call travis-tool.sh install_r DBI in the build_script section as a workaround, but this really should be unnecessary: DBI should get installed when dplyr is installed. I still have no idea what's broken here.

[juliasilge]

I just tried doing a version like this:

build_script:
  - travis-tool.sh install_deps
  - travis-tool.sh install_r DBI

But it is still failing in the same spot: https://ci.appveyor.com/project/juliasilge/tidytext/build/1.0.2

In the artifacts, it still says:

*** moving datasets to lazyload DB
** preparing package for lazy loading
Error in loadNamespace(i, c(lib.loc, .libPaths()), versionCheck = vI[[i]]) : 
  there is no package called 'DBI'
ERROR: lazy loading failed for package 'tidytext'
* removing 'C:/projects/tidytext/tidytext.Rcheck/tidytext'

It looks like trying to force DBI to install did not work. There haven't been any recent updates to DBI, it looks like? Can you think of any reason why DBI is not installing? My package does not actually use any relational database systems, for the record; it only imports dplyr.

[krlmlr]

DBI seems to be installed; could you please look into failure.zip in the artifacts section of your build to see what's wrong now?

[juliasilge]

Ah, you are right; I was looking in the wrong directory before. Apologies!

I have gone through and added manually via install_r first DBI, then it said that it couldn't find assertthat. Then I added manually assertthat, and it failed because it could not find Rcpp. Then I added Rcpp manually and it failed because it could not find tibble.

These are all dependencies of dplyr, which just had a new version released. It looks like the dependencies of dplyr are no longer getting installed correctly since the new release.

[juliasilge]

I had the idea to try manually installing dplyr via install_r and that appears to have successfully installed dplyr + its dependencies, but now it is failing because it can't find SnowballC, which is a dependency of ANOTHER of my package's dependencies, quanteda:

** preparing package for lazy loading
Error in loadNamespace(j <- i[[1L]], c(lib.loc, .libPaths()), versionCheck = vI[[j]]) : 
  there is no package called 'SnowballC'
ERROR: lazy loading failed for package 'tidytext'
* removing 'C:/projects/tidytext/tidytext.Rcheck/tidytext'

So it seems like it is failing to install the packages that my package's dependencies depend on, if that makes sense. When I look at the console of the latest build, for example, it is installing the direct dependencies but not the ones one level back.

[juliasilge]

I was playing around with this a bit more today, and I got my package to build on appveyor by installing via install_r a whole bunch of my package's dependencies which have other dependencies. It looks like this now:

build_script:
  - travis-tool.sh install_r dplyr
  - travis-tool.sh install_r quanteda
  - travis-tool.sh install_r broom
  - travis-tool.sh install_r topicmodels
  - travis-tool.sh install_r ggplot2
  - travis-tool.sh install_deps

I added these, one at a time, after seeing which dependency one level back was missing (for example, gtable for ggplot2, psych for broom, etc).

My package is building successfully on AppVeyor now, but this isn't right, correct? Something is not behaving as it is supposed to with installing dependencies?

[krlmlr]

Looks like this is a devtools problem after all: https://github.com/hadley/devtools/issues/1246. I'll keep this open, because r-appveyor temporarily might want to install devtools from GitHub.

[jennybc]

What is the status of this? Will r-appveyor (temporarily) install devtools from GitHub? Or is there a way for me to request that in my own appveyor.yml? I might not have @juliasilge's patience to develop the list of dependencies with dependencies for manual installation!

[jennybc]

In case it's useful to anyone else while this situation sorts itself out, @jimhester showed me how to direct appveyor to grab devtools from GitHub. Have this in your package's appveyor.yml:

build_script:
  - travis-tool.sh install_github hadley/devtools
  - travis-tool.sh install_deps

This, in turn, requires that Rtools be present, so to force installation, even in the absence of a src directory, do something like this:

environment:
  global:
    USE_RTOOLS: true

[surmann]

A general solution for this issue would be fine. Currently I'm adding one package after the other to the import session of the DESCRIPTION file to run AppVeyor. But this way and the other ones the previous posts are only workarounds. Is there a glue how to solve this?

Btw.: I have the same problem on my local PC. For me it looks like a general issue with Windows.

[jennybc]

@surmann For your local problem, is it not fixed by installing devtools from GitHub? The general solution comes from r-appveyor doing likewise (at least temporarily) or devtools updating on CRAN, whichever comes first.

[surmann]

@jennybc Thanks! It fixed my problem with AppVeyor.

Unfortunately not on my local Windows PC. I now use devtools 1.12.0.9000 and tried to install a GitHub package A with an import dep to package B, which has an import dep itself to package C. Package C is not installed and I get the message "there is no package called 'C'". I think it makes sense to issue this at GitHub devtools. What do you think?

[jimhester]

@surmann There is already an issue for this https://github.com/hadley/devtools/issues/1265 and it should be fixed with the latest devtools. Make sure to restart your R session after installing devel devtools before testing.

[surmann]

@jimhester Obviously, it was a missed restart. Thanks for your hint.

[andrechalom]

Thanks a lot @jennybc and @jimhester, the changes to appveyor.yml above have allowed me to successfully build one of my current projects in appveyor and to uncover a bug in the Windows build of another project.

[krlmlr]

Thanks for your insights.

@jennybc @jimhester: I thought installing devtools from GitHub on a Windows machine requires some special action. I'm just surprised this code works.

Obviously, the easiest "fix" is to release devtools to CRAN. An alternative is to put a binary build of a properly working devtools version somewhere on the web, and have r-appveyor install devtools specifically from there by default. (A drat repository would be the super-luxury variant.) @jimhester: What do you suggest?

[surmann]

@krlmlr I simply install the CRAN version of devtools and after that the GitHub version via devtools::install_github("hadley/devtools"). After restarting R all other packages from GitHub can be installed without any problems. For Appveyor, I add - travis-tool.sh install_github hadley/devtools to build_script section as mentioned by @jennybc.

[krlmlr]

@surmann: That's a workaround, but actually everything should work "out of the box".

[surmann]

@krlmlr That´s right. However, this is the only way it works currently on a windows machine.

[krlmlr]

@surmann: I'm aware of this, and I'd like to help change the state of affairs.

[jimhester]

I talked with @hadley about a small devtools release, it sounds like it might be hard to squeeze in in the immediate future.

We could add a step to the devtools appveyor.yml that puts the latest windows binary in the github releases for r-appveyor after the build succeeds. Of course that would require devtools to successfully build on appveyor (which is currently does not due to this issue :wink:).

We would have to expand the build matrix in the devtools appveyor to build on release, devel and old-release to support all versions of appveyor as well.

I don't believe drat repositories provide binary builds, so we would have to do something custom for this use case if we went that route.

[krlmlr]

If this is a regression, I could just grab an older devtools release from CRAN by default.?

[jimhester]

That is true, might be the simplest solution for now

On Jul 29, 2016 10:34 AM, "Kirill Müller" notifications@github.com wrote:

If this is a regression, I could just grab an older devtools release from CRAN by default.?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/krlmlr/r-appveyor/issues/69#issuecomment-236196841, or mute the thread https://github.com/notifications/unsubscribe-auth/AAMh22IYLwnkBufqiYQagS5TX1W9DAoqks5qag9xgaJpZM4I99Wv .

[krlmlr]

Do you know, by chance, which version would work best?

[jimhester]

1.11.1 should work fine.

CRAN only keeps sources from previous versions of packages, however there is a binary for R-3.1 at https://cran.r-project.org/bin/windows/contrib/3.1/devtools_1.11.1.zip which you could try. I wouldn't be surprised if it breaks though.

[krlmlr]

I've pushed a fix to the r-travis project that installs a custom version of devtools (krlmlr/devtools@c6387931a37dc, current master with some testthat-related changes undone). The technical part seems to work, does the updated version of devtools pick up the dependencies in your packages?

[jennybc]

Works for me: https://ci.appveyor.com/project/jennybc/cellranger/build/1.0.37

I was able to revert to my previous appveyor.yml.

[juliasilge]

Also works for me: https://ci.appveyor.com/project/juliasilge/tidytext/build/1.0.26

I removed the lines from my appveyor.yml that installed Rtools and that installed the GitHub version of devtools.

I am up for closing this issue? Seems fixed to me? :heavy_check_mark:

[krlmlr]

Thanks to everyone for your help, I'm sorry for the pain this caused.

[surmann]

Thanks, this works for me, too.