search

brad-cannell / pathfinder

The goal of pathfinder is to help track file dependencies in research projects created in RStudio.
Other
0 stars 0 forks source link

Makefile #3

Open mbcann01 opened 6 years ago

mbcann01 commented 6 years ago

I need to learn more about makefiles and decide if incorporating some make-like functionality would be useful.

mbcann01 commented 6 years ago

I don’t want to have to manually set targets in a file like with Drake. What if you create a “development “ or maybe “private” or “ignore” folder that you don’t want touched. Everything else is fair game. Then when you save a change somewhere, any other change you make downstream is updated - as long as it isn’t in the ignore folder.

Sent with GitHawk

mbcann01 commented 6 years ago

Here's a good place to start: http://stat545.com/automation00_index.html

I'm not sure that recreating "make" for R is the best thing to do with Pathfinder. For starters, it's already been done by people that are much better at it than I am. Secondly, if you are new to make (and most scientists/data analysts/data scientists probably are) it can be confusing and intimidating and a lot to learn. Rather, we want Pathfinder to feel natural and easy if you are used to working in R.

Having said that, we probably do want to have some system for either manually or automatically pushing changes downstream when upstream files change.

I'm thinking maybe track file changes in an R data frame. Give each version of a dataset a number, associate that number somehow with the version used in a given file, then prompt to update of the version number in a downstream file is lower than a version number in an upstream file.

mbcann01 commented 5 years ago

Is there some way to take advantage of the ordering and updates that Git is already aware of (hashes) to know when a change in one file needs to be propagated through other files?