umap-project / umap

uMap lets you create maps with OpenStreetMap layers in a minute and embed them in your site.
https://umap-project.org
Other
1.19k stars 226 forks source link

Cannot import geojson from dropbox #489

Closed bagage closed 1 month ago

bagage commented 7 years ago

I'm trying to import GeoJSON from a shared Dropbox link. Tested on umap.openstreetmap.fr.

Steps to reproduce

  1. Open "Data Import" tab
  2. Enter URL: https://www.dropbox.com/s/l2g8kit4felnznv/26-statistics.geojson?dl=0
  3. Choose GeoJSON format and layer
  4. Hit "Import"

Expected behavior

File is downloaded/imported and displayed on the map.

Actual behavior

Nothing happens (not even a loading spinner or such). Is the link somewhat invalid?

At first I wanted to use "Remote data" Url to keep track of changes, but the same behavior applies so I cannot use it (yet!).

yohanboniface commented 7 years ago

I don't use nor now anything about Dropbox©®, but I see the link you are using it not pointing to the geojson itself, but to an html page where one can then download the geojson. No idea how to get the direct file link, but this is what you need.

bagage commented 7 years ago

The URL doesn't matter; you can try yourself:

wget https://www.dropbox.com/s/l2g8kit4felnznv/26-statistics.geojson just download the file. Actually this kind of link is also used for Seafile, an opensource alternative to Dropbox. But since UMap fails to download from both of them (while wget succeed even if I don't know exactly how), it's hard to have URL changes tracking with UMap today and I'd like to remedy that. If you don't consider it an issue, would you mind pointing me to the sourcecode handling the download please? Thanks!

yohanboniface commented 7 years ago

Fun. Works with wget but not with httpie. I still think this is an issue on Dropbox side and I hope there is a less magical link out there, but let's reopen until we investigated. You are welcome to do any PR on uMap! Here is a pointer: https://github.com/umap-project/Leaflet.Storage/blob/master/src/js/leaflet.storage.layer.js#L605

yohanboniface commented 7 years ago
$ http --headers 'https://www.dropbox.com/s/l2g8kit4felnznv/26-statistics.geojson' --follow
HTTP/1.1 200 OK

While

$ wget -d 'https://www.dropbox.com/s/l2g8kit4felnznv/26-statistics.geojson'
HTTP/1.1 302 Found
Location: https://dl.dropboxusercontent.com/content_link/Q5u3oZw2VSZgTBRzBdprC0SUDworNMrKIrBPLsDyZhtpDWIP8RpJDcXUtp5BvJ9L/file
bagage commented 7 years ago

Thanks for the pointer - in the mean time, here is the equivalent link for seafile:

https://cloud.damsy.net/f/a0c9fa2a59/?dl=1

As you can see responses are very similar -- I'll investigate leaflet storage hopefully simply the redirection must be handled :).

PS: thanks for reopening that, much appreciated.

yohanboniface commented 7 years ago
$ http --headers 'https://www.dropbox.com/s/l2g8kit4felnznv/26-statistics.geojson' User-Agent:"Wget/1.0"
HTTP/1.1 302 Found
Location: https://dl.dropboxusercontent.com/content_link/nEvuscY2SPxrwHO9own0zIPydTgTurGihl8JMrgf7OlgTaThAdFJqh3VVusipUpC/file

So seems Dropbox is changing the response according to User-Agent sniffing, which sounds really ugly.

yohanboniface commented 7 years ago

I'll investigate leaflet storage hopefully simply the redirection must be handled

Hope so, but I'm afraid you will fall into CORS :/

bagage commented 7 years ago

@yohanboniface: While I am investigating the issue, I think there is at least another issue here: see #493

bagage commented 7 years ago

Update: the seafile ilnk issue was fixed on the server side by tweaking nginx for CORS support.

On the drobox side, one workaround is to replace hostname from www.dropbox.com to dl.dropboxusercontent.com. See the fiddle.

It's a bit hacky but it seems that it's a valid workaround since mid-2015 and easy enough to put in place - what do you think @yohanboniface?

yohanboniface commented 7 years ago

It's a bit hacky but it seems that it's a valid workaround since mid-2015 and easy enough to put in place - what do you think @yohanboniface?

If you mean adding in the code this kind of replacement, I'd say that's not a good idea: it's out of scope of Leaflet.Storage, and if we add one we will be asked to add many more, and I feel this unmaintainable. But we sure can add somewhere in the documentation or in a FAQ the info about replacing the domain when trying to use Dropbox®©. :)

bagage commented 7 years ago

It's out of scope of Leaflet.Storage

Fair enough. But... (see below ;))

But we sure can add somewhere in the documentation or in a FAQ the info about replacing the domain when trying to use Dropbox®©. :)

I'am afraid the FAQ and/or documentation is more a developer stuff than user - do you think people will find this hint by themselves?

What about dealing with this directly in UMap (this repository)? Since it handles user inputs, maybe it could have some "intelligence" (or black magic depending on your point of view…) to replace the hostname before invoking importFromUrl from Leaflet.Storage? Dropbox is a very popular service for users, I feel it is unfortunate that it won't work "out of the box" for them. (same thing applies for google drive, microsoft drive).

But I get the point about maintainability - so I cannot enforce you to agree with me nor can I estimate the number of requests for other (smaller) services that doesn't support CORS. Feel free to close if you think it's out of the scope of the project -- at least thanks to you my very personal issue is fixed :-D.

yohanboniface commented 1 year ago

Six years later, is this issue still relevant ?

github-actions[bot] commented 1 month ago

This issue is stale because it has been open for 30 days with no activity.

github-actions[bot] commented 1 month ago

This issue was closed because it has been inactive for 14 days since being marked as stale.