Download error from R CMD check

[mattdowle]

Hi Filipe,

I'm checking revdeps as part of releasing data.table. skynet is failing for me with the following error. But I see it's passing fine on CRAN. Maybe I'm just unlucky and the file it's trying to fetch is not available starting today, perhaps. When I got the error with the new version of data.table, I tried again with the current CRAN version of data.table (1.12.8) and I still see this error with skynet. So I don't think it is due to data.table, but if you don't see it, then it might be due to my computer or environment. Any ideas?

Thanks, Matt

$ Rdevel CMD check skynet_1.3.8.tar.gz 
* using log directory ‘/home/mdowle/build/revdeplib/skynet.Rcheck’
* using R Under development (unstable) (2020-07-14 r78854)
* using platform: x86_64-pc-linux-gnu (64-bit)
* using session charset: UTF-8
* checking for file ‘skynet/DESCRIPTION’ ... OK
* checking extension type ... Package
* this is package ‘skynet’ version ‘1.3.8’
* package encoding: UTF-8
* checking package namespace information ... OK
* checking package dependencies ... OK
* checking if this is a source package ... OK
* checking if there is a namespace ... OK
* checking for executable files ... OK
* checking for hidden files and directories ... OK
* checking for portable file names ... OK
* checking for sufficient/correct file permissions ... OK
* checking whether package ‘skynet’ can be installed ... OK
* checking installed package size ... OK
* checking package directory ... OK
* checking ‘build’ directory ... OK
* checking DESCRIPTION meta-information ... OK
* checking top-level files ... OK
* checking for left-over files ... OK
* checking index information ... OK
* checking package subdirectories ... OK
* checking R files for non-ASCII characters ... OK
* checking R files for syntax errors ... OK
* checking whether the package can be loaded ... OK
* checking whether the package can be loaded with stated dependencies ... OK
* checking whether the package can be unloaded cleanly ... OK
* checking whether the namespace can be loaded with stated dependencies ... OK
* checking whether the namespace can be unloaded cleanly ... OK
* checking loading without being on the library search path ... OK
* checking dependencies in R code ... NOTE
Namespace in Imports field not imported from: ‘maps’
  All declared Imports should be used.
* checking S3 generic/method consistency ... OK
* checking replacement functions ... OK
* checking foreign function calls ... OK
* checking R code for possible problems ... OK
* checking Rd files ... OK
* checking Rd metadata ... OK
* checking Rd cross-references ... OK
* checking for missing documentation entries ... OK
* checking for code/documentation mismatches ... OK
* checking Rd \usage sections ... OK
* checking Rd contents ... OK
* checking for unstated dependencies in examples ... OK
* checking contents of ‘data’ directory ... OK
* checking data for non-ASCII characters ... OK
* checking data for ASCII and uncompressed saves ... OK
* checking installed files from ‘inst/doc’ ... OK
* checking files in ‘vignettes’ ... OK
* checking examples ... OK
* checking for unstated dependencies in ‘tests’ ... OK
* checking tests ...
  Running ‘testthat.R’
 ERROR
Running the tests in ‘tests/testthat.R’ failed.
Last 13 lines of output:
  Backtrace:
   1. skynet::download_t100(2011, "mkt")
   2. httr::POST(...)
   3. httr:::request_perform(req, hu$handle$handle)
   5. httr:::request_fetch.write_memory(req$output, req$url, handle)
   6. curl::curl_fetch_memory(url, handle = handle)

  ══ testthat results  ═══════════════════════════════════════════════════════════
  [ OK: 70 | SKIPPED: 0 | WARNINGS: 2 | FAILED: 3 ]
  1. Error: Download Ontime (@test_download_ontime.R#6) 
  2. Error: Download T100 (@test_download_t100.R#6) 
  3. Error: Download T100 (@test_download_t100.R#12) 

  Error: testthat unit tests failed
  Execution halted
* checking for unstated dependencies in vignettes ... OK
* checking package vignettes in ‘inst/doc’ ... OK
* checking running R code from vignettes ...
  ‘skynet.Rmd’ using ‘UTF-8’... OK
 NONE
* checking re-building of vignette outputs ... OK
* checking PDF version of manual ... OK
* DONE

Status: 1 ERROR, 1 NOTE
See
  ‘/home/mdowle/build/revdeplib/skynet.Rcheck/00check.log’
for details.

[FilipeamTeixeira]

@mattdowle my apologies for the late reply.

I've tested on my computer and I got no errors or notes. What happens when you run one of those functions individually? Does it give you an error as well or returns a blank data frame?

Try to run the download_ontime or the download_t100 functions outside of the test environment and tell me if you still have any issues.

[mattdowle]

Sure thing.

> require(skynet)
Loading required package: skynet
> download_ontime()
Error in paste("https://transtats.bts.gov/PREZIP/On_Time_Reporting_Carrier_On_Time_Performance_1987_present_",  : 
  argument "y" is missing, with no default
> download_t100()
Please select mkt - market or seg - segment database.
Error in download_t100() : 
> skynet::download_t100(2011, "mkt")
Error in curl::curl_fetch_memory(url, handle = handle) : 
  error:141A318A:SSL routines:tls_process_ske_dhe:dh key too small

[FilipeamTeixeira]

Thank you for the extra info.

This seems to be a bug with openssl and it's not something I can change in R. However, I would try to suggest to follow this here https://stackoverflow.com/questions/61517347/net-core-httpclient-dh-key-too-small-on-ubuntu-20-04. This fix is for Ubuntu, but it is the same for any OS. All you have to do is to find that openssl.cnf file.

Let me know if it worked so I can close this issue.

Thank you

[mattdowle]

Many thanks! Yes that worked. If I understand correctly, Ubuntu 20.04 (and one comment said Apple too) have increased the default security level to 2 deliberately and expect website owners to upgrade their level 1 certificates to level 2. I'm indeed running a distro based on Ubuntu 20.04. I don't mind downgrading to level 1 at all for my purposes. Perhaps you can let the owner of the website your package is downloading from that this change is on the horizon and they might wish to update their certificate.

[FilipeamTeixeira]

I'm happy to hear that it worked. The data provider is the US Bureau of Transport Statistics so I wouldn't really expect them to upgrade their certificates anytime soon. In any case I'll drop my contacts a quick message so they're aware of that. Thank you.