search

BiologicalRecordsCentre / ukotas

UK Overseas Territories Alien Species website work - https://dev-ceh-ukotas.pantheonsite.io/
GNU General Public License v3.0
0 stars 0 forks source link

Automated process to run R script on checklists #37

Open DavidRoy opened 1 month ago

DavidRoy commented 1 month ago

Process would be:

  1. Export a UKOT checklist dataset in the same format as the import spreadsheets
  2. R rscript to import and process checklist, producing a .pdf report @bowlerbear script
  3. pdf report available via a page
  4. run process on a schedule, e.g. every month or when there is a change to the underlying data

@johnvanbreda could you comment on the feasbility of this

DavidRoy commented 1 month ago

@JimBacon also including you as I'm not sure if you've got experience of running R script on our webservers

JimBacon commented 1 month ago

I'm unclear from the above if you anticipate doing this on the indicia warehouse webserver, which I don't suppose has R installed, or on a Pantheon webserver, where I don't suppose you have any possibility of running R. Either way, it sounds ugly.

If you want the pdf reports on a Drupal site then how about a. Making the R script a service hosted on an R platform. I'm thinking like R-Shiny apps. b. Writing a Drupal module which sends data to the service and makes the results available on the site.

JimBacon commented 1 month ago

I'm presuming the R script is complex and would be hard to rewrite in PHP. If that is a false assumption then you could do away with the R script and rewrite it in PHP in the Drupal module.

DavidRoy commented 1 month ago

We need the flexility of R I think as we have a lot of expertise and ready-built processing and visualisation approaches. I have no pre-conception of how to do this but the website is Drupal site on Pantheon so no link to Indicia.

From what you say, going through an R shiny server is probably more achievable (secure?) if it's not feasible to run R scripts on Pantheon.

@BirenRathod - do you know whether there are examples or if it is possible to run R scrips on Pantheon websites?

johnvanbreda commented 1 month ago

There was a module for Drupal 7 that could run an R script, but it relied on an installed version of R on the server and I doubt that Pantheon would support this. I think an R shiny server is the way to go. The R script could pull the UKOT checklist direct from the Drupal REST API so we would just need server side configuration to support this. I presume it is possible to run a script inside R shiny on a schedule?

JimBacon commented 1 month ago

We know we can schedule from the Drupal end so it could always trigger the R script. Checkout https://www.rplumber.io/ and the hosting options. UKCEH has an in-house Posit Connect server, I understand.

DavidRoy commented 1 month ago

@JimBacon could you take this on as it needs interaction with different parts of UKCEH? No great rush but ideally a solution by end of January 25

BirenRathod commented 1 month ago

@BirenRathod - do you know whether there are examples or if it is possible to run R scrips on Pantheon websites?

@DavidRoy I never heard of running R script on Pantheon using the recent Drupal 10 & 11. Drupal 7 will be out of service soon, so that is out of option.

Also, about your query generating pdf reports, UKCEH is planning using Flourish for data visualisation. See this link -> https://cehacuk.sharepoint.com/sites/hub-news/SitePages/New-UKCEH-website.aspx if that something helpful to integrate with website is possible.