Test against the correct .gdx on PIK's cluster

[0UmfHxcvx5J7JoaOhFSs5mncnisTJJ6q]

This https://github.com/pik-piam/remind/blob/936776b26461d1e180ff47d7db72e10406d202ca/tests/testthat/test-convGDX2mif.R#L16-L19 returns a random .gdx. Is that intended?

[giannou]

yes, it is supposed to find the newest one. but probably the location has to change as people set validation runs in a different place now

[giannou]

should I relabel the issue?

[0UmfHxcvx5J7JoaOhFSs5mncnisTJJ6q]

yes, it is supposed to find the newest one. but probably the location has to change as people set validation runs in a different place now

No. find does not return results in any particular order. It depends on the internal state of the file system and should be considered random.

Try

gdxs <- system(
  command = paste("find /p/projects/remind/runs/r*",
                  "-wholename /p/projects/remind/runs/r*fulldata.gdx",
                  "-not -wholename \"*magpie*\"",
                  "-exec stat -t \\{\\} \\; 2>/dev/null |",
                  "sort -k13nr | head -n 1 | cut -d \" \" -f 1"),
  intern = TRUE)

instead. That gives you the newest fulldata.gdx file under any ./r* directory in /p/projects/remind/runs/ that does not have "magpie" in its path. But that file also didn't change in more than a year, so you might hard-code it :p

-rw-rw-r-- 1 dklein rdev 9.5M May 13  2019 /p/projects/remind/runs/r8565-R20-C/output/r8565-R20-C_Base-rem-5/fulldata.gdx

should I relabel the issue?

I dunno. I'm just trying to figure out why the automated test on remind-the-R-package/REMIND-EU fails.

[0UmfHxcvx5J7JoaOhFSs5mncnisTJJ6q]

While we are at it: This scheme of people selecting their test-gdxes manually obviously doesn't work. Since /p/projects/remind/runs/0AA_DONTDELETEME_MO/fulldata.gdx should always be tested, I suggest including it in the repository (but ignore it for package building).

[0UmfHxcvx5J7JoaOhFSs5mncnisTJJ6q]

Since /p/projects/remind/runs/0AA_DONTDELETEME_MO/fulldata.gdx should always be tested, I suggest including it in the repository (but ignore it for package building).

It already is: https://github.com/pik-piam/remind/blob/master/inst/extdata/old.gdx Why don't we test against that?

[0UmfHxcvx5J7JoaOhFSs5mncnisTJJ6q]

I forget all the stuff I did … must be the age.

https://github.com/pik-piam/remind/blob/936776b26461d1e180ff47d7db72e10406d202ca/tests/testthat/test-convGDX2mif.R#L8-L10

[giannou]

... obviously doesn't work.

I don't have access to wherever this leads

[giannou]

yes, it is supposed to find the newest one. but probably the location has to change as people set validation runs in a different place now

No. find does not return results in any particular order. It depends on the internal state of the file system and should be considered random.

when I tested it it was always giving me the newest one (maybe because all folders had the same name convention?), anyway thanks for the suggestion

[0UmfHxcvx5J7JoaOhFSs5mncnisTJJ6q]

... obviously doesn't work.

I don't have access to wherever this leads

Sorry. Everybody seems to be in the REMIND-EU group, so I assumed everybody is. :/ Marianna "accidentality" tested remind-the-R-package/REMIND-EU on the cluster and the test failed on a commit that was done two months ago. So she was awarded the distinguished code-testing cross :medal_military: for being the first to test the package branch in at least two months.

The point is: people don't test, unless they are forced to.

[Loisel]

The point is: people don't test, unless they are forced to.

This is one of the most important lessons :) Maybe the GDXs downloaded from RSE and tested whenever not on the cluster should also be copied to the cluster for consistency @giannou

[giannou]

I ran the tests on the cluster and they are run with the GDXs from the RSE server so I guess the code part that looks for a GDX on the cluster can be deleted, no?

[Loisel]

yes, nice that rse server is reachable from the cluster. I second removing this part.