search

bcgov / wqbench

R package to generate download and compile data from EPA ECOTOX database
https://bcgov.github.io/wqbench/
Apache License 2.0
3 stars 2 forks source link

`wqb_download_epa_ecotox()` fix hanging folder name issue #2

Closed aylapear closed 1 year ago

aylapear commented 1 year ago

Folders copy correctly but something is going wrong as the folders are nested. Potentially an issue with the temp directory not clearing out. 

'data_dl/ecotox_ascii_09_15_2022/ecotox_ascii_03_10_2022/AsciiDownloadHelp5.pdf', reason 'No such file or directory'

Full set of calls to the function

> library(wqbench)
> wqb_download_epa_ecotox(file_path = "data_dl", version = 3)
Downloading...
  |==========================================================================================================================| 100%
[1] "output_folder"
[1] "data_dl/ecotox_ascii_06_15_2022"
> wqb_download_epa_ecotox(file_path = "data_dl", version = 4)
Downloading...
  |==========================================================================================================================| 100%
[1] "output_folder"
[1] "data_dl/ecotox_ascii_03_10_2022"
> wqb_download_epa_ecotox(file_path = "data_dl", version = 1)
Downloading...
  |==========================================================================================================================| 100%
[1] "output_folder"
[1] "data_dl/ecotox_ascii_12_15_2022"
> wqb_download_epa_ecotox(file_path = "data_dl", version = 1)
Downloading...
  |==========================================================================================================================| 100%
[1] "output_folder"
[1] "data_dl/ecotox_ascii_12_15_2022"
Warning message:
In dir.create(file.path(move_location, "validation"), recursive = TRUE) :
  'data_dl/ecotox_ascii_12_15_2022/validation' already exists
> wqb_download_epa_ecotox(file_path = "data_dl", version = 2)
Downloading...
  |==========================================================================================================================| 100%
[1] "output_folder"
[1] "data_dl/ecotox_ascii_09_15_2022"
There were 50 or more warnings (use warnings() to see the first 50)
> warnings()
Warning messages:
1: In file.create(to[okay]) :
  cannot create file 'data_dl/ecotox_ascii_09_15_2022/ecotox_ascii_03_10_2022/AsciiDownloadHelp5.pdf', reason 'No such file or directory'
2: In file.create(to[okay]) :
  cannot create file 'data_dl/ecotox_ascii_09_15_2022/ecotox_ascii_03_10_2022/chemical_carriers.txt', reason 'No such file or directory'
3: In file.create(to[okay]) :
  cannot create file 'data_dl/ecotox_ascii_09_15_2022/ecotox_ascii_03_10_2022/dose_response_details.txt', reason 'No such file or directory'
4: In file.create(to[okay]) :
  cannot create file 'data_dl/ecotox_ascii_09_15_2022/ecotox_ascii_03_10_2022/dose_response_links.txt', reason 'No such file or directory'
aylapear commented 1 year ago

Clearly it does have to do with the temp directory not clearing out after the function has finished executing:

> library(wqbench)
> wqb_download_epa_ecotox(file_path = "data_dl", version = 1)
Downloading...
  |====================================================================================================================================| 100%
[1] "unzip sub"
[1] "/var/folders/yr/tq_q43k50m795hb4y0njhyth0000gn/T//RtmpFnCDb1/unzip"
[1] "dirs"
[1] ""                                   "ecotox_ascii_12_15_2022"            "ecotox_ascii_12_15_2022/validation"
> wqb_download_epa_ecotox(file_path = "data_dl", version = 2)
Downloading...
  |====================================================================================================================================| 100%
[1] "unzip sub"
[1] "/var/folders/yr/tq_q43k50m795hb4y0njhyth0000gn/T//RtmpFnCDb1/unzip"
[1] "dirs"
[1] ""                                   "ecotox_ascii_12_15_2022"            "ecotox_ascii_12_15_2022/validation"
[4] "validation"                        
There were 50 or more warnings (use warnings() to see the first 50)
> warnings()
Warning messages:
1: In file.create(to[okay]) :
  cannot create file 'data_dl/ecotox_ascii_09_15_2022/ecotox_ascii_12_15_2022/AsciiDownloadHelp5.pdf', reason 'No such file or directory'
aylapear commented 1 year ago

If the R session is restarted then it is fine:

Restarting R session...

> library(wqbench)
> wqb_download_epa_ecotox(file_path = "data_dl", version = 2)
Downloading...
  |====================================================================================================================================| 100%
[1] "unzip sub"
[1] "/var/folders/yr/tq_q43k50m795hb4y0njhyth0000gn/T//RtmpLvuHGt/unzip"
[1] "dirs"
[1] ""           "validation"