KentonWhite / ProjectTemplate

A template utility for R projects that provides a skeletal project.
http://projecttemplate.net
GNU General Public License v3.0
622 stars 159 forks source link

Added support for feather files using `feather` and a test. #149

Closed mpancia closed 8 years ago

mpancia commented 8 years ago

This is to add a reader for the feather serialized data frame format (see).

I have included another default reader to support this using the feather package, as well as a test to check that it loads a feather-ized version of the iris data set.

mpancia commented 8 years ago

There seems to be some issue with Travis getting the feather package to build, as per this. I can't manage to get it to install the feather package, so this isn't passing at the moment. Closing until I figure it out.

KentonWhite commented 8 years ago

Thanks @mpancia, this would be a great addition. Too bad there is problems with Travis CI. I would also be hesitant adding the Feather reader to the main ProjectTemplate branch since the feather package is not stable at the moment.

One way we work with new readers, especially ones where the interface may change or having trouble with breaking the Travis CI build, is to make the reader as a separate package. The reader is then included as a library and works with ProjectTemplate. This can be done by moving the reader code you have written into a new package and including the following code snippet at the end of the package file:

 .onLoad <- function(...)
  {
    .add.extension('es', es.reader)
  }

Then you can have the convenience of a feather reader, share it with others, and not worry about the interface changing or problems with Travis CI. As the reader matures and becomes stable, then we can work to integrate it into the CRAN version of ProjectTemplate.

You can see an example on github at kentonwhite/esReader

mpancia commented 8 years ago

@KentonWhite Yeah, thanks! In hindsight, I agree with you about not necessarily incorporating feather into the main branch, given its alpha status as a file format. I'll follow your suggestion for making a separate package.

However, I did manage to get Travis to work appropriately with my fork that includes feather (and a bunch of other nonsense, ATM). Regardless, it might be worth transitioning the main ProjectTemplate repo to use the native Travis CI support as opposed to the (now-deprecated) older one that is currently used. The .travis.yml file that works to replace your current one is actually fairly minimal, and works with the feather compilation as well:

language: r
sudo: required 
cache: packages
warnings_are_errors: true
addons:
  apt:
    sources:
      - ubuntu-toolchain-r-test
    packages:
      - gcc-4.9
      - g++-4.9
      - r-cran-rodbc
before_install: |
  mkdir ~/.R
  cat <<EOF > ~/.R/Makevars
  CXX=g++-4.9
  CXX1X=g++-4.9
  CXX1XSTD=-std=c++11
r_github_packages:
  - johnmyleswhite/log4r

No need to incorporate this, but just letting you know.