Closed fuqiang-code closed 1 month ago
I just committed a change to configure these with environment variable, e.g.
env SCRAPYD_HTTP_PORT=8080 SCRAPYD_BIND_ADDRESS=0.0.0.0 scrapyd
This allows for more flexible configuration.
Do you still need CLI options?
Closing as no response, but we can consider CLI options in addition to envvars, if needed.
Introduction
Scrapyd is a popular tool used for deploying and running Scrapy spiders. While Scrapyd provides various configuration options, two commonly used parameters,
--bind-address
and--http_port
, require modifying the Scrapyd startup command. This article proposes enhancements to Scrapyd to add these parameters as command-line options, simplifying the configuration process.Adding
--bind-address
parameterCurrently, Scrapyd listens on the localhost (127.0.0.1) by default, limiting its accessibility. By introducing the
--bind-address
parameter, users will have the flexibility to specify the IP address on which Scrapyd should listen. This allows Scrapyd to accept connections from different hosts on a network, facilitating better integration with other services.Example usage:
In the above example, Scrapyd will bind to all available network interfaces, making it accessible from any IP address.
Adding
--http_port
parameterScrapyd uses port 6800 for its HTTP API by default. However, certain environments or security configurations may require using a different port. By introducing the
--http_port
parameter, users will have the ability to specify a custom port for Scrapyd's HTTP API.Example usage:
In the above example, Scrapyd will start the HTTP API on port 8080 instead of the default port 6800.
Conclusion
By adding the
--bind-address
and--http_port
parameters as command-line options in Scrapyd, users will have more flexibility in configuring the network interface and port for Scrapyd's operation. This enhancement simplifies the process of modifying these commonly used parameters, making it more convenient for users to customize Scrapyd's behavior to their specific requirements.