search

datasets / house-prices-uk

UK house prices dataset
https://datahub.io/core/house-prices-uk
12 stars 46 forks source link

[Automation] Remote server SSL verify failed. #6

Open AcckiyGerman opened 6 years ago

AcckiyGerman commented 6 years ago

The datapackage-pipeline cannot download and process the source.

Acceptance criteria

Analysis

source link: https://www.nationwide.co.uk/~/media/MainSite/documents/about/house-price-index/downloads/uk-house-price-since-1952.xls

automation log: https://testing.datahub.io/AcckiyGerman/house-prices-uk/v/8

tabulator.exceptions.HTTPError: HTTPSConnectionPool(host='www.nationwide.co.uk', port=443): Max retries exceeded with url: /~/media/MainSite/documents/about/house-price-index/downloads/uk-house-price-since-1952.xls (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)'),))

File could be downloaded via browser, but I failed to download it with curl Definitely the bug is on the remote site. (I just wonder why could I get the file via browser without any security warnings)

rufuspollock commented 6 years ago

@AcckiyGerman this may be blocking bots. Have you replicated with data cat url or simple python script.

AcckiyGerman commented 6 years ago

Python "urlretreave" failed as well as curl.

On Jan 15, 2018 6:10 PM, "Rufus Pollock" notifications@github.com wrote:

Reopened #6 https://github.com/datasets/house-prices-uk/issues/6.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/datasets/house-prices-uk/issues/6#event-1424745496, or mute the thread https://github.com/notifications/unsubscribe-auth/ALKlIshLoNpPwwAlJCPf3itt0_e4fVbXks5tK4aCgaJpZM4ReEJ3 .

AcckiyGerman commented 6 years ago

"data cat " fails too

On Jan 15, 2018 6:33 PM, "Dmitry German" dmitry.german@datopian.com wrote:

Python "urlretreave" failed as well as curl.

On Jan 15, 2018 6:10 PM, "Rufus Pollock" notifications@github.com wrote:

Reopened #6 https://github.com/datasets/house-prices-uk/issues/6.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/datasets/house-prices-uk/issues/6#event-1424745496, or mute the thread https://github.com/notifications/unsubscribe-auth/ALKlIshLoNpPwwAlJCPf3itt0_e4fVbXks5tK4aCgaJpZM4ReEJ3 .

rufuspollock commented 6 years ago

@AcckiyGerman have you tried sending a non-bot header with url retrieve or similar?

AcckiyGerman commented 6 years ago

@rufuspollock no, I haven't and reasons are: