Possible improvements in the Export to Google buckets

[rdstern]

@lilyclements The installation needs 3 packages installed as follows:

devtools::install_github("IDEMSInternational/epicsadata")
devtools::install_github("IDEMSInternational/epicsawrap")
devtools::install_github("IDEMSInternational/rpicsa")

We are trying to install them in R or RStudio and finding some do not have devtools. I am not sure why we are not installing them in R-Instat which (at least on my machine) does have devtools?

I don't see why we can't install them through R-Instat? I know that Lily has been extending R-Instat Tools Import to import from github and perhaps this is why?

I suggest it may be simpler to add the above file into the new R-Instat script library and then run it a line at a time?

This is what I have done.

We have then completed the dialog as follows:

And the sub-dialog like this:

I am trying to run again. It took ages to complete the dialog, so I used To Script, kept that file and ran it later. This ran for me and James, but not for the participants - I think Lily found the problem and has reported it. Here is the script:

# Dialog: Export Climatic Definitions

epicsawrap::gcs_auth_file(filename="C:/Users/rdste/SSD Dropbox/Roger Stern/Roger/Malawi/E_PICSA 2024/Day 3/Day 3/epicsa_token.json")
observation_data_unstacked_by_stationName_s_year1 <- data_book$get_data_frame(data_name="observation_data_unstacked_by_stationName_s_year1")
annual_rain <- epicsawrap::reformat_annual_summaries(data=observation_data_unstacked_by_stationName_s_year1, station_col="stationName", year_col="s_year1", start_rains_doy_col="start_rain", start_rains_date_col="start_rain_date", end_rains_doy_col="end_rains", end_rains_date_col="end_rains_date", seasonal_rain_col="sum_PRECIP", season_length_col="length")

epicsawrap::export_r_instat_to_bucket(summaries=c("annual_rainfall"), 
annual_rainfall_data=annual_rain, data_by_year="observation_data_unstacked_by_stationName_s_year1", 
start_rains_column="start_rain", data="observation_data_unstacked", 
end_rains_column="end_rains", rain="PRECIP", year="s_year1", month="month_abbr", 
seasonal_length_column="length", include_summary_data=TRUE, station_id="Malawi_Roger", 
definitions_id="10", country="ml_test")

rm(list=c("annual_rain", "observation_data_unstacked_by_stationName_s_year1"))

The dialog could be made simpler to use, I hope, and definitely needs help. The Help ID for the main dialog is 692and 700 for the help on the sub-dialog.

More to come. .

[lilyclements]

@rdstern thanks for the feedback. It's absolutely sensible to update the packages on R-Instat (assuming there is devtools on there)

Suggestions to simplify the dialog are really welcome!

And yes, the issue has been reported here. I hope it can be fixed if we try updating curl to be >= version 5.1.0? (This can be achieved by running install.packages("curl"))

[rdstern]

@lilyclements more on your export and also on the metadata that you export for the definitions.

Here is the main dialog:

a) I think the Data receiver (the first) is for the daily data. If so, then please could it be for the summary data instead? I note that you don't ask for the Summary data and I assume you get that, because of the links. Could you do this "the other way round"? I think you may also only need the daily data frame when asking for crop probabilities. So (if that is true), then a data file with just the dummary data frame is all you really need if you aren just producing annual rainfall summaries - which we were? b) I am also interested in your JSON metadata. I think this is either one large definitions file, or multiple small JSON files and these are metadata attached to the summary data frame. Either way I would like to have more details of which dialogs they are associated with the what the metadata looks like. We have 4 dialogs to manage the other metadata Prepare > R Objectsmenu and I would like the definitions metadata to be included here if possible? c) Here is an exercise I do often, namely get the Start and the End and then show I could either use the length dialog to get the lengths, or use the calculator. I assume I need to use the Lenhtg to generate (or add to) the JSON file? d) The definition used by DCCMS for the start is the first day, from 1 October with more than 25mm in 3 days, and no dry spell in the next 21 days of 10 days or more. I often get it girst without the dry spells condition (first planing opportunity) ns also with (successful planting). Then (in R-Instat - not e-picsa) we look at the difference which gives the risk of having to replant. I assume only the second definition gets into the definitions file, so we need to do those in the right order? e) I would later, like to g=have more suggestions to simplify completing the dialog. Some items could have pull-downs with suggestions and maybe some tooltips would be useful? f) I note you have the logical variables included - great. Is that for the future, or have you also fixed this in the dialogs?

.