search

fangfufu / httpdirfs

A filesystem which allows you to mount HTTP directory listings or a single file, with a permanent cache. Now with Airsonic / Subsonic support!
Other
763 stars 59 forks source link

URLS with : in the path Not working #127

Closed kingofkya closed 4 months ago

kingofkya commented 11 months ago

Found this URL with : in it that is apparently valid. Just logging it incase anyone tries the same thing. suse should probably be replacing : with %3A on there end. Httpdirfs seams to not be able to read theses links, likely similar to the spaces issue for %20 in urls.

httpdirfs --cache https://download.opensuse.org/repositories/Cloud:/Images:/ /opensuse


libcurl SSL engine: OpenSSL/3.0.2
CacheSystem_init(): directory: /root/.cache/httpdirfs/https%3A%2F%2Fdownload.opensuse.org%2Frepositories%2FCloud%3A%2FImages%3A%2F
LinkTable_new(): disk_linktbl->num: 1, linktbl->num: 1
LinkTable_invalid_reset(): 0 invalid links
LinkTable_uninitialised_fill(): ... Done!
--------------------------------------------
 LinkTable 0x55d256fe1310 for https://download.opensuse.org/repositories/Cloud:/Images:/
--------------------------------------------
0 H 0  https://download.opensuse.org/repositories/Cloud:/Images:/
--------------------------------------------
LinkTable_print(): Invalid link count: 0, https://download.opensuse.org/repositories/Cloud:/Images:/.
--------------------------------------------
fangfufu commented 11 months ago

Could you format your ticket properly please? Are you reporting that the URL is working? Or is it not working?

kingofkya commented 11 months ago

Sorry about that hit save before I was done.

kingofkya commented 11 months ago

Also if needed here is an example of a mirror of the same repo for that that does it properly, and has no issues with httpdirfs. httpdirfs --cache https://provo-mirror.opensuse.org/repositories/Cloud%3A/Images%3A/ /opensuse

fangfufu commented 11 months ago

Thanks. I will label this as a bug. I will look at it when I have some more time.

fangfufu commented 5 months ago

Probably a duplicate of https://github.com/fangfufu/httpdirfs/issues/138

fangfufu commented 5 months ago

I can confirm what the ticket described is correct - only affects cached mode.

fangfufu commented 5 months ago

@kingofkya , I believe this has been fixed via 28293b5ccdc4f65e5721ba2fe978f1cff066d0d7, sorry for the delay.

fangfufu commented 5 months ago

I have manually verified that this is resolved. Having said that, I don't think this particular server supports range request, I could be wrong. Feel free to re-open the ticket. 

fangfufu@smithsonian:~/projects/httpdirfs$ ./httpdirfs -f --cache https://download.opensuse.org/repositories/Cloud:/Images:/ tmp
Debug:src/cache.c:102:CacheSystem_init: /home/fangfufu/.cache/httpdirfs/https%3A%2F%2Fdownload.opensuse.org%2Frepositories%2FCloud%3A%2FImages%3A%2F
Debug:src/link.c:577:LinkTable_new: linktbl->index_time: 1714876057
Debug:src/link.c:56:Link_to_curl: https://download.opensuse.org/repositories/Cloud:/Images:/
Debug:src/link.c:750:LinkTable_disk_open: linktbl->index_time: 1714875893
Debug:src/link.c:609:LinkTable_new: disk_linktbl->num: 14, linktbl->num: 14
Debug:src/link.c:513:LinkTable_invalid_reset: 0 invalid links
Debug:src/link.c:228:LinkTable_uninitialised_fill:  ...... Done!
Debug:src/link.c:692:LinkTable_disk_save: linktbl->index_time: 1714875893
Debug:src/link.c:649:LinkTable_new: !!!!Calling LinkTable_new for the 0 time!!!!
LinkTable_print: --------------------------------------------
LinkTable_print:  LinkTable 0x555efbd966f0 for https://download.opensuse.org/repositories/Cloud:/Images:/
LinkTable_print: --------------------------------------------
LinkTable_print: 0 H 0  https://download.opensuse.org/repositories/Cloud:/Images:/
LinkTable_print: 1 D 0 Leap_15.0 https://download.opensuse.org/repositories/Cloud:/Images:/Leap_15.0/
LinkTable_print: 2 D 0 Leap_15.1 https://download.opensuse.org/repositories/Cloud:/Images:/Leap_15.1/
LinkTable_print: 3 D 0 Leap_15.2 https://download.opensuse.org/repositories/Cloud:/Images:/Leap_15.2/
LinkTable_print: 4 D 0 Leap_15.3 https://download.opensuse.org/repositories/Cloud:/Images:/Leap_15.3/
LinkTable_print: 5 D 0 Leap_15.4 https://download.opensuse.org/repositories/Cloud:/Images:/Leap_15.4/
LinkTable_print: 6 D 0 Leap_15.5 https://download.opensuse.org/repositories/Cloud:/Images:/Leap_15.5/
LinkTable_print: 7 D 0 Leap_15.6 https://download.opensuse.org/repositories/Cloud:/Images:/Leap_15.6/
LinkTable_print: 8 D 0 Leap_42.1 https://download.opensuse.org/repositories/Cloud:/Images:/Leap_42.1/
LinkTable_print: 9 D 0 Leap_42.2 https://download.opensuse.org/repositories/Cloud:/Images:/Leap_42.2/
LinkTable_print: 10 D 0 Leap_42.3 https://download.opensuse.org/repositories/Cloud:/Images:/Leap_42.3/
LinkTable_print: 11 D 0 openSUSE_13.1 https://download.opensuse.org/repositories/Cloud:/Images:/openSUSE_13.1/
LinkTable_print: 12 D 0 openSUSE_13.2 https://download.opensuse.org/repositories/Cloud:/Images:/openSUSE_13.2/
LinkTable_print: 13 D 0 SLES_12.3 https://download.opensuse.org/repositories/Cloud:/Images:/SLES_12.3/
LinkTable_print: --------------------------------------------
LinkTable_print:  Invalid link count: 0
LinkTable_print: --------------------------------------------
fangfufu@smithsonian:~/projects/httpdirfs$ ./httpdirfs -f --cache https://provo-mirror.opensuse.org/repositories/Cloud%3A/Images%3A/ tmp
Debug:src/cache.c:102:CacheSystem_init: /home/fangfufu/.cache/httpdirfs/https%3A%2F%2Fprovo-mirror.opensuse.org%2Frepositories%2FCloud%253A%2FImages%253A%2F
Debug:src/link.c:577:LinkTable_new: linktbl->index_time: 1714876092
Debug:src/link.c:56:Link_to_curl: https://provo-mirror.opensuse.org/repositories/Cloud%3A/Images%3A/
Debug:src/link.c:736:LinkTable_disk_open: Linktable at /home/fangfufu/.cache/httpdirfs/https%3A%2F%2Fprovo-mirror.opensuse.org%2Frepositories%2FCloud%253A%2FImages%253A%2F/meta///.LinkTable does not exist.Debug:src/link.c:466:LinkTable_fill: Filling https://provo-mirror.opensuse.org/repositories/Cloud%3A/Images%3A/
Debug:src/link.c:228:LinkTable_uninitialised_fill:... Done!
Debug:src/link.c:692:LinkTable_disk_save: linktbl->index_time: 1714876092
Debug:src/link.c:649:LinkTable_new: !!!!Calling LinkTable_new for the 0 time!!!!
LinkTable_print: --------------------------------------------
LinkTable_print:  LinkTable 0x5603142698f0 for https://provo-mirror.opensuse.org/repositories/Cloud%3A/Images%3A/
LinkTable_print: --------------------------------------------
LinkTable_print: 0 H 0  https://provo-mirror.opensuse.org/repositories/Cloud%3A/Images%3A/
LinkTable_print: 1 D 0 Leap_15.0 https://provo-mirror.opensuse.org/repositories/Cloud%3A/Images%3A/Leap_15.0/
LinkTable_print: 2 D 0 Leap_15.1 https://provo-mirror.opensuse.org/repositories/Cloud%3A/Images%3A/Leap_15.1/
LinkTable_print: 3 D 0 Leap_15.2 https://provo-mirror.opensuse.org/repositories/Cloud%3A/Images%3A/Leap_15.2/
LinkTable_print: 4 D 0 Leap_15.3 https://provo-mirror.opensuse.org/repositories/Cloud%3A/Images%3A/Leap_15.3/
LinkTable_print: 5 D 0 Leap_15.4 https://provo-mirror.opensuse.org/repositories/Cloud%3A/Images%3A/Leap_15.4/
LinkTable_print: 6 D 0 Leap_42.1 https://provo-mirror.opensuse.org/repositories/Cloud%3A/Images%3A/Leap_42.1/
LinkTable_print: 7 D 0 Leap_42.2 https://provo-mirror.opensuse.org/repositories/Cloud%3A/Images%3A/Leap_42.2/
LinkTable_print: 8 D 0 Leap_42.3 https://provo-mirror.opensuse.org/repositories/Cloud%3A/Images%3A/Leap_42.3/
LinkTable_print: 9 D 0 SLES_12.3 https://provo-mirror.opensuse.org/repositories/Cloud%3A/Images%3A/SLES_12.3/
LinkTable_print: 10 D 0 openSUSE_13.1 https://provo-mirror.opensuse.org/repositories/Cloud%3A/Images%3A/openSUSE_13.1/
LinkTable_print: 11 D 0 openSUSE_13.2 https://provo-mirror.opensuse.org/repositories/Cloud%3A/Images%3A/openSUSE_13.2/
LinkTable_print: --------------------------------------------
LinkTable_print:  Invalid link count: 0
LinkTable_print: --------------------------------------------