search

RETURN-project / makeDataCube

Data management
Apache License 2.0
0 stars 0 forks source link

Environment-dependent line in vignette #20

Closed PabRod closed 3 years ago

PabRod commented 3 years ago

The first time this line of vignettes/make_Landsat_cube.Rmd is run:

LSscenes <- dllLS(l1folder, queuefolder, queuefile, tmpfolder, landsatlogfile, ext, starttime, endtime, sensors, tiers, cld)

it downloads a set of files and populates the variable LSscenes. If executed again (when the files have already been downloaded), LSscenes returns an empty list.

This environment-dependent behavior is dangerous, and it may cause bugs.

PabRod commented 3 years ago

By digging a bit into dllLS, we find that the output assignation happens in this line of R/Landsat.R:

scenes <- dllLandsat(queuefolder, queuefile, tmpfolder, logfile, ext, starttime, endtime, sensors, tiers, cld)

The dllLandsat function is available in inst/python/dllLandsat.py.

PabRod commented 3 years ago

@wanda: it seems that dllLandsat.py has been copied directly from another source. Can you give me some context?

Thanks in advance!

wandadk commented 3 years ago

dllLandsat.py only downloads data that haven't been added to the queue already (lines 30-35 in dllLandsat.py). This was done to prevent that we download data (and add them to the cube in lines 131 of the vignette) if it is not really needed (since the scenes were already added to the cube).

I wrote dllLandsat.py myself (inspired by these examples), but added it the the /inst folder, so R recognizes the python function when building the R package.

PabRod commented 3 years ago

I see. Please check if this pull request makes sense for you.

It contains minimal changes, actually, it is a one-liner. I haven't changed the way the data is managed, so the downloads will still happen only if needed. The only change is the output: it will be always the list of scenes, regardless of the fact of them having been downloaded or not.

wandadk commented 3 years ago

Thanks!

The list of scenes is used in the vignette to add the scenes to the queue and process them to a level-2 product (line 131 in vignette).

Landsat2L2(paramfolder, paramfile, l2folder, LSscenes, logfolder, Sskiplogfile, Ssuccesslogfile, Smissionlogfile, Sotherlogfile)

Although I did not test it, I'm afraid that providing the total list of scenes will result in errors. Maybe a solution could be to output both the total list of scenes and also the list of scenes that have been downloaded. The latter list can then be used as input for the Landsat2L2 function.

PabRod commented 3 years ago

I'll consider this closed after the accepted pull request.