search

scrapinghub / shub

Scrapinghub Command Line Client
https://shub.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
125 stars 79 forks source link

Add `deploy-reqs` session to shub docs #152

Closed josericardo closed 7 years ago

josericardo commented 8 years ago

deploy-reqs isn't mentioned on the shub docs

http://doc.scrapinghub.com/shub.html

jdemaeyer commented 8 years ago

Repeating this here for completeness: This is somewhat on purpose. deploy-reqs is very reckless. It re-builds and uploads all dependency eggs on every invocation, disregarding what libraries are available on SC anyways (http://support.scrapinghub.com/topic/205467-supported-libraries-on-scrapy-cloud/). There are also additonal issues with binaries eggs that have to be built server-side, some of which have been solved in Kumo.

This is definitely far from optimal, but I suggest we wait for Kumo's release and then see how we want to proceed.

See also #56