opendatateam / udata

Customizable and skinnable social platform dedicated to open data.
http://udata.readthedocs.org
GNU Affero General Public License v3.0
239 stars 87 forks source link

Chrome blocking some downloads when the external URL for a resource is provided in HTTP #2760

Open AlainVagner opened 2 years ago

AlainVagner commented 2 years ago

Chrome blocks the downloads of some resources. These resources are external and their URL is in HTTP and not HTTPS.

Detailed Description

More details about the change in Chrome here: https://blog.chromium.org/2020/02/protecting-users-from-insecure.html

This happens only for resources which should be downloaded by the browser (if the resource is a web page, e.g. for a documentation, the problem does not happen)

How to reproduce

You can for example check this dataset in Chrome: https://data.public.lu/en/datasets/57068838a30b1b33caf65976/

If you click on the download button on the first resource (in ods format), Chrome will block the download with the following error in the console: Mixed Content: The site at 'https://data.public.lu/' was loaded over a secure connection, but the file at 'https://elections.public.lu/opendata/referendum/2015/referendum-2015-ods.ods' was redirected through an insecure connection. This file should be served over HTTPS. This download has been blocked. See https://blog.chromium.org/2020/02/protecting-users-from-insecure.html for more details.

If we change the URL by replacing http by https, the issue is fixed.

The same problem happens also with our udata 4 instance.

Context

Some users are complaining because they cannot download some files on our udata instance. We still have lots of old datasets with external resources in HTTP, and nothing prevent data producers to add new resources with HTTP urls.

Possible Implementation

Your Environment

maudetes commented 2 years ago

Thank you for this detailed issue.

I see that we have the same issue on data.gouv.fr, ex: https://www.data.gouv.fr/fr/datasets/liste-des-points-de-recharge-de-la-carte-pastel/.

Some inputs on this issue and the possible implementation:

Let me know if this is clear enough for you.

AlainVagner commented 2 years ago

@maudetes thanks for your answer. We will check the URLS_ALLOWED_SCHEMES setting. In the mean time, we have developed a script available in this repo, to find all URLs possibly impacted by this issue: https://github.com/opendatalu/udata-fetching-scripts

maudetes commented 2 years ago

Thank you for the script! I'd be interested to know how you've dealt with the http resources found. Did you plan to contact the producers or check if upgrading to https would return the same content for example?

AlainVagner commented 2 years ago

We have tried for ourselves to upgrade the urls to https, whenever it was possible, we just did the change ourselves. For the non-upgradable ones, we have contacted the producers. We did the check manually, because the list was quite short. Would it have been bigger, we may have scripted the check.

AlainVagner commented 2 years ago

We have tried removing http from the URL_ALLOWED_SCHEMES setting and it is working. The error message is however a bit terse. error It would be helpful to precise what's wrong with this URL and help people fix the issue, e.g. give the list of authorised protocols in the error message. This is also mandatory for the digital accessibility (cf criterion 11.11 of the RGAA)

maudetes commented 2 years ago

It would be helpful to precise what's wrong with this URL and help people fix the issue, e.g. give the list of authorised protocols in the error message.

Indeed! We actually have a proper error message with the validation error reason, that is replaced with this Invalid URL (with localization) here: https://github.com/opendatateam/udata/blob/63f78489f5cb7011eb99bd20483785e3465ca7ee/udata/forms/fields.py#L141 We could return the full validation error instead (they aren't localized though).

AlainVagner commented 2 years ago

It would make sense. Could the internationalization be done directly in uris.py? https://github.com/opendatateam/udata/blob/63f78489f5cb7011eb99bd20483785e3465ca7ee/udata/uris.py

The message "Invalid URL" is not false, but not precise enough. It could be kept, and we could extend the error message with some extra informations.

So the error message coming from the validator would be: "Invalid scheme http". I think it would be good to clarify what the authorised schemes are. For example: 'Invalid URL: invalid scheme "http". The authorised schemes in an URL are: "https", "ftp", "ftps"'.
Does it seam feasible?

maudetes commented 2 years ago

Seems feasible indeed! We can use _('Invalid URL: %(reason)s', reason=reason) patterns to build the localized error string. We won't be able to take this ticket in the current sprint iteration however. You are welcome if you want to give it a shot and open a PR.

Let us know if you try it out, else we can plan this later to pick it for future sprint iterations.

AlainVagner commented 2 years ago

Here is a draft PR which has not been tested yet (I have not been able to install the development environment so far): https://github.com/opendatateam/udata/pull/2768