scrapyd-client doesn't support basic auth

getorca commented 5 years ago

it doesn't seem like scrapyd-client supports basic auth. my scrapyd-deploy worked fine, but scrapyd-client is returning a 401.

Digenis commented 5 years ago

@getorca, it does support it. You just specify username and password for your target. What do you mean by "it works but returns a 401"? How do you add authentication to scrapyd? You proxy it through a webserver, right?

PhakphumV commented 4 years ago

Got similar issue here.

Here are configuration that I set

Remote server

install scrapyd and run it in background
Setup a reverse proxy with nginx to scrapyd
Configure basic_auth in this virtualhost with username and password

Local machine

Install scrapy and scrapy-deploy
Create a scrapy project
Set scrapy.cfg file with the credentials

So when I run scrapyd-deploy command, the message that I received is 401 - Authorization Required

jpmckinney commented 3 years ago

I have a scrapy.cfg file that is the same as in the readme. See:

Without a netrc file: https://github.com/scrapy/scrapyd-client#deploying-a-project
With a netrc file: https://github.com/scrapy/scrapyd-client#global-settings

I recommend attempting to login to your service through a browser, to be sure the basic authentication is set up correctly.

Then, please check that you don't have a ~/.netrc file with conflicting credentials. Or, if you have only a scrapy.cfg file, try moving the credentials to a ~/.netrc file (it shouldn't make a difference, but it's worth a shot).

Also, ensure that your ~/.netrc file is chmod 400, just in case it's not being read/trusted due to weak permissions.

scrapy / scrapyd-client

scrapyd-client doesn't support basic auth #56