bnosac / cronR

A simple R package for managing your cron jobs.
Other
290 stars 38 forks source link

Compatibility with Project Template repo srtucture #28

Closed r-leyshon closed 4 years ago

r-leyshon commented 4 years ago

Hi there,

I'm unsure if you have any plans to achieve this, but it would definitely help me schedule a la project template website

If this is of interest, some detail. I've toyed around with scheduling the 'initialise.R' script in the src folder. Looking at the logs generated, this appears to run and then sources the first of the munge scripts used to generate urls to scrape in the successive munge scripts. As you will see execution halts after munge script 1, which is perplexing as this is not a behaviour when manually executing the source file. Next steps for me are to try a cron job and failing that, go back to cronR but with a single script layout. Running R version 4.0.1 (2020-06-06) -- "See Things Now" on Mac. Below is the output from that log:

Project name: DSCA_course_catalogue_dev Loading project configuration Autoloading packages Loading package: plyr Loading required package: plyr Loading package: dplyr Loading required package: dplyr

Attaching package: ‘dplyr’

The following objects are masked from ‘package:plyr’:

arrange, count, desc, failwith, id, mutate, rename, summarise,
summarize

The following objects are masked from ‘package:stats’:

filter, lag

The following objects are masked from ‘package:base’:

intersect, setdiff, setequal, union

Loading package: rvest Loading required package: rvest Loading required package: xml2 Loading package: selectr Loading required package: selectr Loading package: xml2 Loading package: stringr Loading required package: stringr Loading package: jsonlite Loading required package: jsonlite Loading package: tidyr Loading required package: tidyr Loading package: purrr Loading required package: purrr

Attaching package: ‘purrr’

The following object is masked from ‘package:jsonlite’:

flatten

The following object is masked from ‘package:rvest’:

pluck

The following object is masked from ‘package:plyr’:

compact

Loading package: httr Loading required package: httr Loading package: httpuv Loading required package: httpuv Loading package: rlist Loading required package: rlist Loading package: gmailr Loading required package: gmailr

Attaching package: ‘gmailr’

The following object is masked from ‘package:dplyr’:

id

The following object is masked from ‘package:plyr’:

id

The following object is masked from ‘package:utils’:

history

The following objects are masked from ‘package:base’:

body, date, labels, message

Autoloading helper functions Running helper script: globals.R Running helper script: helpers.R Autoloading data Munging data Running preprocessing script: 01_GET_repo_urls.R [1] "There are 30 URLs to scrape link from: " [1] "https://github.com/datasciencecampus/access-to-services"
[2] "https://github.com/datasciencecampus/algorithmia-segment"
[3] "https://github.com/datasciencecampus/algorithmia-vegetation"
[4] "https://github.com/datasciencecampus/anomaly-detection"
[5] "https://github.com/datasciencecampus/awesome-campus"
[6] "https://github.com/datasciencecampus/clapp"
[7] "https://github.com/datasciencecampus/coding-standards"
[8] "https://github.com/datasciencecampus/coffee-and-coding"
[9] "https://github.com/datasciencecampus/coffee-and-coding-ldn"
[10] "https://github.com/datasciencecampus/community-visualizations"
[11] "https://github.com/datasciencecampus/consultation-analysis-nafw"
[12] "https://github.com/datasciencecampus/deploy-dash-with-gcp"
[13] "https://github.com/datasciencecampus/DSCA_advNLPpython"
[14] "https://github.com/datasciencecampus/DSCA_ARP"
[15] "https://github.com/datasciencecampus/DSCA_DSWR"
[16] "https://github.com/datasciencecampus/DSCA_ggplot-leaflet-map-training"
[17] "https://github.com/datasciencecampus/DSCA_Intro-R"
[18] "https://github.com/datasciencecampus/DSCA_introNLPpython"
[19] "https://github.com/datasciencecampus/DSCA_intropython"
[20] "https://github.com/datasciencecampus/DSCA_ML_theory"
[21] "https://github.com/datasciencecampus/DSCA_NLPWR"
[22] "https://github.com/datasciencecampus/DSCA_Stats4DS"
[23] "https://github.com/datasciencecampus/DSCA_Version-control-with-Git-Github"
[24] "https://github.com/datasciencecampus/DSCA_Version-controlled-RStudio-projects" [25] "https://github.com/datasciencecampus/eclipse"
[26] "https://github.com/datasciencecampus/employmentProspects"
[27] "https://github.com/datasciencecampus/energy-efficiency"
[28] "https://github.com/datasciencecampus/finbins"
[29] "https://github.com/datasciencecampus/gcp-function-deployment"
[30] "https://github.com/datasciencecampus/google-mobility-reports-data"

Execution halted

jwijffels commented 4 years ago

I have no clue based on your input where your error is coming from. Best is to check your code with options(echo = TRUE) and see where it fails. The schedule has apparently run, so it is not a cronR issue as all cronR does is scheduling R scripts making no assumptions on where things are running. So it is an issue in your script highly likely your using relative paths instead of absolute paths.

jwijffels commented 4 years ago

As this seems not an issue of the cronR package as the schedule runs but your own script fails, I'll close this. Feel free to reopen if this is not the case.