r-lib / httr

httr: a friendly http package for R
https://httr.r-lib.org
Other
986 stars 1.99k forks source link

Can't get content for FTP directory listing #731

Closed pepijn-devries closed 10 months ago

pepijn-devries commented 1 year ago

Thanks for your work on httr, it really makes accessing online information in R so much easier. However, I came across the following problem.

When I use httr to get a directory list of an FTP site, I will get an error when trying to obtain the requested content using the content function from httr. This is illustrated in the following reprex:

library(httr)

# ftp site obtained from https://dlptest.com/ftp-test/
ftp_list <-
  GET("ftp://ftp.dlptest.com/",
      authenticate("dlpuser", "rNrKYTX9g7z3RgJRmxWuGHbeu"),
      dirlistonly = TRUE)

## this doesn't work and throws an error:
content(ftp_list)

## this does work:
cat(rawToChar(ftp_list$content))

Is this expected behaviour, or can this be fixed?

hadley commented 10 months ago

FTP is not officially supported by httr2, sorry.

pepijn-devries commented 10 months ago

That is fair enough. However, note that my issue is reported under httr and not httr2. Furthermore, the error message is cryptic (I've recreated the reprex with the reprex package to make this more clear). A more informative error-message would be helpful here.

library(httr)

# ftp site obtained from https://dlptest.com/ftp-test/
ftp_list <-
  GET("ftp://ftp.dlptest.com/",
      authenticate("dlpuser", "rNrKYTX9g7z3RgJRmxWuGHbeu"),
      dirlistonly = TRUE)

## this doesn't work and throws an error:
content(ftp_list)
#> Error in x$headers[["Content-Type"]]: subscript out of bounds

## this does work:
cat(rawToChar(ftp_list$content))
#> drwxr-xr-x    3 1001     1001           16 Nov 01 19:42 11
#> -rw-r--r--    1 1001     1001     37032115 Nov 01 19:45 CGR_WAV_102023.wav
#> -rw-r--r--    1 1001     1001          718 Nov 01 19:40 Export_PostVenta_64552.csv
#> -rw-r--r--    1 1001     1001         1492 Nov 01 19:40 Export_PostVenta_64553.csv
#> -rw-r--r--    1 1001     1001          878 Nov 01 19:44 Export_PostVenta_64612.csv
#> -rw-r--r--    1 1001     1001         1169 Nov 01 19:44 Export_PostVenta_64613.csv
#> -rw-r--r--    1 1001     1001        27781 Nov 01 19:40 Export_PostVenta_64711.csv
#> -rw-r--r--    1 1001     1001          187 Nov 01 19:40 Export_PostVenta_64712.csv
#> -rw-r--r--    1 1001     1001          168 Nov 01 19:40 Export_PostVenta_64713.csv
#> -rw-r--r--    1 1001     1001          345 Nov 01 19:44 Export_PostVenta_64714.csv
#> -rw-r--r--    1 1001     1001          178 Nov 01 19:44 Export_PostVenta_64715.csv
#> -rw-r--r--    1 1001     1001     10485760 Nov 01 19:44 NetAllyTest.txt
#> -rw-r--r--    1 1001     1001         1149 Nov 01 19:43 TestUpload.kml
#> drwxr-xr-x    3 1001     1001           18 Nov 01 19:40 Vorne
#> -rw-r--r--    1 1001     1001       593920 Nov 01 19:41 \conexao.sdf
#> -rw-r--r--    1 1001     1001        10000 Nov 01 19:45 _11471977
#> drwxr-xr-x    2 1001     1001           29 Nov 01 19:40 input
#> -rw-r--r--    1 1001     1001         6895 Nov 01 19:43 test-cron-2023-11-01.xlsx
#> -rw-r--r--    1 1001     1001          144 Nov 01 19:40 test.zip

Created on 2023-11-01 with reprex v2.0.2

hadley commented 10 months ago

That was a typo; it's not supported by httr either. I agree that the error message could be better but (a) httr is no longer under active development and (b) I don't think many people would expect ftp to work with a package called httr.

pepijn-devries commented 10 months ago

OK. Fair enough. Thanks for considering.