scalingexcellence / scrapybook

Scrapy Book Code
http://scrapybook.com/
475 stars 209 forks source link

Can't deploy 'properties' spider to scrapinghub.com from the docker container (chapter 6) #64

Open nherriot opened 4 years ago

nherriot commented 4 years ago

Hi scrapybook.

Firstly - the book is fantastic. Wish I had bought it before starting with scrapy. Would have save a lot of work!

In chapter 6 I'm slightly confused and can't get to work the example provided. And I'm guessing the portal has moved on since the book was written. But since my docker container is still using the exact versions of shub it should communicate with scrapinghub.com in the same way and deploy my spider. It does not and I get an error when I specifiy my project ID like this from my docker container:

root@dev:~/book/ch06/properties# shub deploy 451072
Unknown target: 451072

If i deploy as suggested in the book:

root@dev:~/book/ch06/properties# shub deploy 
Packing version 1589885829
Deploying to Scrapy Cloud project "properties"
Error: Deploy failed (500):
 <!DOCTYPE html><html lang=en><head>  <title>Scrapinghub · Scrapinghub</title>  <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
<meta name="Copyright" content="2010-2020 Scrapinghub Limited" />
.......
........
.........
<!-- Begin Hotjar integration -->
<script>
    (function(h,o,t,j,a,r){
        h.hj=h.hj||function(){(h.hj.q=h.hj.q||[]).push(arguments)};
        h._hjSettings={hjid:1117371,hjsv:6};
        a=o.getElementsByTagName('head')[0];
        r=o.createElement('script');r.async=1;
        r.src=t+h._hjSettings.hjid+j+h._hjSettings.hjsv;
        a.appendChild(r);
    })(window,document,'https://static.hotjar.com/c/hotjar-','.js?sv=');
</script>
<!-- END Hotjar integration -->
  </body></html>

The book scrapy.cfg I have looks like this:


[settings]
default = properties.settings

[deploy]
#url = http://dash.scrapinghub.com/api/scrapyd/
#username = nherriot
project = properties
#project = 451072

I've tried all combinations of with and without url, username, project=projectID and project=project name. But they all fail. The version of shub on the docker machine is 1.5

Do you know why this is?

Which looks like a web page being returned as part of the error. Any help in getting the book examples to run on the most recent version of scrapinghub.com would be great! :-)

Kind regards, Nicholas.