Save and retrieve customized global.dcf

[rsangole]

I would like to request a feature to save a user-customized version of global.dcf someplace. For each project I create, I end up editing the libraries portion of the file and add all my go-to libraries. It's a pain to do everytime, so today, I first open an old project and copy-paste global.dcf into my new project. A bit cumbersome, and some thing we can program our way out of.

[Hugovdberg]

There is work underway to create custom templates, which includes the global.dcf, perhaps @connectedblue can give an update on his work.

Op 28 aug. 2017 20:00 schreef "Rahul S" notifications@github.com:

I would like to request a feature to save a user-customized version of global.dcf someplace. For each project I create, I end up editing the libraries portion of the file and add all my go-to libraries. It's a pain to do everytime, so today, I first open an old project and copy-paste global.dcf into my new project. A bit cumbersome, and some thing we can program our way out of.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/johnmyleswhite/ProjectTemplate/issues/204, or mute the thread https://github.com/notifications/unsubscribe-auth/AGn634GwznR5-QKn02VPh2jOAiI6dOH4ks5scwBSgaJpZM4PE2vh .

[connectedblue]

Hi all,

Yes, this use case and others would be easily accommodated with the work I have done on custom template. In fact, that's exactly the reason why I wanted to build this feature.

My progress so far is mixed. I submitted an initial version which @kentonwhite pointed out would not pass CRAN checks because it wrote files to the R lib directory. We briefly discussed whether it would be possible to create this in a seperate package and link in somehow.

I have such a package in my github which I've been using successfully on projects in my workplace. It has been redesigned to write a config file to the users home directory, which is still in breach of CRAN guidelines. There are also some implementation issues which use non exported functions from devtools which would cause the Travis test to complain.

But the biggest challenge is how to link it back to the master version. I've been very busy in a new start up this year and haven't had much time to progress.

Perhaps I should raise a seperate issue to start a community discussion on this feature.

[rsangole]

@connectedblue Thanks for the update. Perhaps one solution is as follows: we develop and maintain a curated list of packages, perhaps a few lists each with a different purpose. These lists could be developed by user surveys or by analyzing existing projects on github. The lists could be selectable in the create.project() as well as via a new function, say load.libraries().

For example:

1. For projects focusing on exploratory data analysis, for the most part, we need: tidyverse, reshape2, lattice, forcats, lubridate, magrittr, corrplot, zoo
2. If we need to perform some modeling, we can add packages like: caret, xgboost, MASS, e1071, pROC etc
3. For projects dealing with quality control, we can add packages: qcc, fitdist... etc

Perhaps the base load.project() comes with option (1). Then, if the user needs (2 and 3), we can call something like load.libraries(append = c(2,3)).

This gives us flexibility in staying within 1 package, while offering a few options to the users,

Thoughts?

[KentonWhite]

Maybe we are overthinking this. While CRAN doesn't like packages that write files to directories the package doesn't own, we could still save common config files in our home directories and then pass the path to create.project().

@connectedblue can we use your modifications just without the write function?

[Hugovdberg]

Is there an issue with packages writing files to the home directory, or with writing files in general? It seems there is no issue with the fact that ProjectTemplate writes a complete directory structure to a given path. Is creating this feature much more complicated than the following?

• Add a global option ProjectTemplate.templatedir, which can be set by the user through .Rprofile
• Create a function create.template which is more or less an alias for create.project(getOption('ProjectTemplate.templatedir', '.'), minimal=TRUE)
• Alter create.project to replace argument minimal=FALSE with template='full', and lookup the template by looking for a directory with the given name in list(getOption('ProjectTemplate.templatedir'), system.file("defaults", package = "ProjectTemplate")) (use the option if it is set and otherwise look in the installation directory)

That way the location where the template is created is controlled by the user, and no data has to be stored in either the installation directory or the home directory.

[KentonWhite]

Unfortunately there is an historical issue with CRAN having packages writing to arbitrary directories. If this was universally allowed, it would be possible for a package on CRAN to infect machines with viruses and trojans! We can explain to CRAN why we need to write to a directory -- it just slows down the process. If we can make a solution that doesn't require writing more files that would be preferred.

I think your proposed solution would work. I know @connectedblue has done some work on this feature so we could hopefully use much of his efforts.

[connectedblue]

@kentonwhite it certainly is possible to have a basic custom template feature without writing to any directory - it involves an environment variable pointing to a pre existing config file containing details of the template. This config file is dcf format and would have to be coded by hand (my version provides some helper functions to add entries and also a default template option).

I also have a function to directly download a template from github rather than the local file system - I see this as a key feature to enable small template blocks to be combined and distributed. This requires some devtools unexported functions which is also a CRAN no-no.

However, the initial version could do without this also.

Let me raise a pull request with a skeleton proposal and see what people think.

[rsangole]

Thanks @KentonWhite and @connectedblue for the quick responses on this request. @connectedblue , if you'd like to collaborate on any codes etc, please reach out... would be happy to help.

[Hugovdberg]

@connectedblue what do you need the configuration for? The project goes by the motto of 'convention over configuration', how exactly do you see these small blocks being used? To me a custom template would simply be an empty directory structure (with at least a number of fixed directories) in which you store your commonly used global.dcf, and perhaps one or two default helper scripts. As @rsangole said there are some different use cases for ProjectTemplate so someone will probably setup one or two templates, with mostly a different set of libraries in global.dcf, or different logging configurations, but I'm not convinced the flexibility of modular templates is necessary. Please say so if I misunderstand you completely, but from your explanation it feels a little overly complex. But then again, please raise the pull request and I'll be happy to look into it!

[connectedblue]

You're right @hugovdberg.

It's probably easier if you submit your version.

[Hugovdberg]

@connectedblue I submitted a pull request, I hope I didn't offend you with my criticism. I realised there could be merit to adding github support, but only for create.template to download a complete template from a repo. Could you check see if support for this feature could be added to the new functionality as it stands and otherwise suggest how create.template should be changed such that GitHub support can be added?

[Hugovdberg]

This issue can be closed since #206 was merged.