search

sensiblecodeio / scraperwiki-python

ScraperWiki Python library for scraping and saving data
https://scraperwiki.com
BSD 2-Clause "Simplified" License
160 stars 69 forks source link

Fix status update failures #94

Closed StevenMaude closed 8 years ago

StevenMaude commented 8 years ago

Use QuickCode URL for status updates.

The previous https://scraperwiki.com/api/status URL now redirects to https://app.quickcode.io/login and the status update fails silently (the site does respond, but doesn't cause a HTTP error).

StevenMaude commented 8 years ago

curl -L -v --data "type=error" https://scraperwiki.com/api/status

* Ignoring the response-body
* Connection #1 to host app.quickcode.io left intact
* Issue another request to this URL: 'https://app.quickcode.io/login'
* Found bundle for host app.quickcode.io: 0xe2e020
* Re-using existing connection! (#1) with host app.quickcode.io
* Connected to app.quickcode.io (54.229.175.226) port 443 (#1)
> GET /login HTTP/1.1
> User-Agent: curl/7.35.0
> Host: app.quickcode.io
> Accept: */*
StevenMaude commented 8 years ago

It might also be better to have the correct redirect from the ScraperWiki URL to the QuickCode URL specified too.

frabcus commented 8 years ago

@StevenMaude I've made the status endpoint use a 307 redirect, so old boxes will carry on working.

StevenMaude commented 8 years ago

@frabcus Great, as that was broken for everyone.

I think this should still get merged to reflect the current configuration accurately.

StevenMaude commented 8 years ago

Merging to reflect the state of things.