Issues in comparing station and satellite data

[rdstern]

I describe the steps I am taking to compare satellite and station data for new situations in preparing to the AIMS teaching. I describe the route and the problems I am encountering. I expect these to split later into separate issues, but feel it is also useful to show the problems in their context.

I hope someone could start by repeating these steps to see if they get the same problems. The data are here: 1.04W-5.12N_SARAH-2_SDU_ds.zip These are satellite data and I will be comparing them with station data. The station data are in the R-Instat library > Climatic > Ghana

1. Open the Ghana station data - no problem here. The data are for 2 stations and the fact it is for multiple stations is a new feature in the following tasks.

2. Use File > Import and Tidy Netcdf for the satellite data. Worked fine for the first station (5.12N), but then gave an error for the second file, i.e. Tamale. I had to do a reset on the dialogue before it would work. Then it imported fine.

3. At the bottom tab rename the sheets to Saltpond and Tamale. That worked fine.

4. Use Prepare > Data Reshape > Append and append the satellite data for the 2 stations. That worked fine.

5. Now merge with the station data. I had to merge with 2 variables - for me for the first time. I just merged by adding only one variable from the second (satellite) file. This didn't work. When I used To Script I deleted a comma at the start of one of the arguments:: col_names=c(,"SDU","Station","time_date"). Then it worked fine.

6. Now I continue

[lloyddewit]

I tried to recreate the issue above.

Step 1 worked as expected. In step 2, when I tried to open the '5.12N' file, I got an error message:

I tried various ways to install the ncdf4.helpers package but without success. I reinstalled R from scratch, and I also reinstalled all R packages from scratch. However I am still left with the following message when I start R-Instat:

Does anyone else have the same issue?

CRAN states here:

Package ‘ncdf4.helpers’ was removed from the CRAN repository. Formerly available versions can be obtained from the archive.

I'll try and see if I can install from the archive.

[lloyddewit]

The solution to the ncdf4.helpers issue above is to install the package from the CRAN archive. To do this, run the following commands in the R-Instat script window:

packageurl <- "http://cran.r-project.org/src/contrib/Archive/ncdf4.helpers/ncdf4.helpers_0.3-3.tar.gz"
install.packages(packageurl, repos=NULL, type="source")
test1 <- require("ncdf4.helpers")
test1

The last 2 lines are just to confirm that the package was installed. If test1 is true then the install was successful.

I raised #5768 for the ncdf4.helpers issue.

[lloyddewit]

Now back to the original issue ...

In step 2, when I open the 2nd file (Tamale) I get the following error:

If I click Reset and then open the file again, then it works correctly.

Step 3 and 4 worked correctly without any problems.

For step 5, I wasn't sure how to do the exact same steps as @rdstern. I selected Prepare / Data Reshape / Merge .... I then selected 'Full Join' with the following 'Merge Options':

This generated the following R code:

ghana <- data_book$get_data_frame(data_name="ghana")
Append1 <- data_book$get_data_frame(data_name="Append1")
merge1 <- dplyr::full_join(x=ghana, y=Append1, suffix=c("ghana", "Append1"), by=c("station"="id", "date"="time_date"))
data_book$import_data(data_tables=list(merge1=merge1))

rm(list=c("merge1", "ghana", "Append1"))

As far as I can see this is correct.

@rdstern The only bug I'm aware of so far is the need to reset before importing the 2nd CDF file. Please could you provide a few more details to recreate any issues I'm missing? thanks