Multiple R packages in a single repo

[csgillespie]

Is it possible to have multiple packages in a single repository? For example, I have multiple (very small) R packages that I use for teaching.

I suppose it would be nice for each package to be validated, but I suppose a global validation would be fine.

[krlmlr]

Have you tried

script:
  - cd subpackage; ../travis-tool.sh run-tests; cd ..

in .travis.yml?

[csgillespie]

@krlmlr Your suggestion works - see below for the script snippet. Two things to note:

1. travis-tool.sh bootstrap requires you to be in a package directory. So we need to switch directories in the before_install command
2. Under install if fakepackage2 had different dependencies from fakepackage1 , then you would need switch directories and run install_deps for each package.

script:
  - ../travis-tool.sh run-tests; cd ../fakepackage2; ../travis-tool.sh run-tests;

#cd fakepackage1; ./travis-tool.sh run_tests; cd ..; cd fakepackage2; ./travis-tool.sh run_tests
before_install:
    - curl -OL http://raw.github.com/craigcitro/r-travis/master/scripts/travis-tool.sh
    - chmod 755 ./travis-tool.sh
    - cd fakepackage1
    - ../travis-tool.sh bootstrap
install:
    - ../travis-tool.sh install_deps

[krlmlr]

1. I'm not convinced -- bootstrapping works for me even if I'm not in the package directory, cf. https://travis-ci.org/krlmlr/travis-test-csgillespie/builds/13551351
2. True.

All in all, the case of multiple packages per repo isn't even supported by devtools. Any reason not to split these into individual repos?

[craigcitro]

hi colin,

yeah, i think this one's going to be too tough to support "out of the box" in r-travis itself.

oh, here's one idea that might just be crazy enough to work: let the build matrix do it for you. in your scripts, do something like cd $REPO_TO_TEST, and in the build matrix, have a block like

- env:
  - REPO_TO_TEST=repo1
  - REPO_TO_TEST=repo2
  ...

you'll need to check on it, but that should do what you want with a little fiddling. if it doesn't work, feel free to point me at a failing travis build.

that said, let's turn this around. it's easy enough to test a single R package with r-travis, and you can put a bunch on github. i'm guessing that the reason that you want to have multiple in one place is so that you can do a single clone and have a bunch of repos available -- have you considered using something like git submodule or git subtree? it's just pushing the problem around to some degree, but it'd be an easy way to have a number of "package collections" without duplicating the packages themselves. (FWIW, i'd recommend subtree over submodule. )

i'll leave this open until we hear what works best -- and then could i ask you to add that to the wiki?

[csgillespie]

@krlmlr The rather weak reason for not splitting these into multiple repos is that I have around ten very small packages I use for teaching. It's just convenient to keep them together (which is what I currently do on r-forge).

@craigcitro I went down the route of using the build matrix. You can see the output at: https://travis-ci.org/csgillespie/travis-test/builds/13592740

The .travis.yml script is nice and simple:

language: c
script: ./travis-tool.sh run_tests

before_install:
    - cd $REPO_TO_TEST
    - curl -OL http://raw.github.com/craigcitro/r-travis/master/scripts/travis-tool.sh
    - chmod 755 ./travis-tool.sh
    - ./travis-tool.sh bootstrap
install:
    - ./travis-tool.sh install_deps
    - ./travis-tool.sh github_package assertthat

env:
  matrix:
        - REPO_TO_TEST=badpackage 
        - REPO_TO_TEST=goodpackage 

The downside to this approach is that a change in a single package will cause travis to run on all packages. I suppose once we can use language R this won't be such a big deal.

---

I looked at submodule and subtree. I suspect that for most people this might be not the best option, since

• submodule doesn't look that great (according to my O'Reilly git book)
• subtree isn't part of the git,

[csgillespie]

@craigcitro Are you happy for me to add this solution to the wiki?

[craigcitro]

@csgillespie yes! this looks awesome. if you wanted to add a copy of that text above, along with a pointer to your project, that would be fantastic. thanks!

[csgillespie]

@craigcitro @krlmlr I was thinking of expanding that repo to contain a bunch of different scenarios that a number of people would be interested in. For example:

• multiple projects (covered above)
• XML and rgdal (https://github.com/craigcitro/r-travis/issues/66)
• Packages that use Bioconductor
• knitr or sweave documents

I'll try and mock up a few simple examples tomorrow.

[csgillespie]

I've added an example set-up to: https://github.com/csgillespie/travis-examples and added a line to the wiki. We could include the code on the wiki, however this has two downsides:

• The recipes page will get large very quickly.
• It will be hard to spot when things break.

So I've just added a link to the example on the recipe page (feel free to change/alter/clone).

[craigcitro]

fantastic, that sounds great -- thanks colin!