Scrapoxy is a super proxy aggregator, allowing you to manage all proxies in one place π―, rather than spreading it across multiple scrapers πΈοΈ. It also smartly handles traffic routing π to minimize bans and increase success rates π.
When the scrapoxy instance is overloaded, we could have some socket hang up.
Expected Behavior
We do not have any error.
Steps to Reproduce
Send as many request as possible and wait
Failure Logs
ERROR [MasterService] request_error: socket hang up from proxy
### Scrapoxy Version
4.9
### Custom Version
- [ ] No
- [X] Yes
### Deployment
- [X] Docker
- [ ] Docker Compose
- [ ] Kubernetes
- [ ] NPM
- [ ] Other (Specify in Additional Information)
### Operating System
- [X] Linux
- [ ] Windows
- [ ] macOS
- [ ] Other (Specify in Additional Information)
### Storage
- [X] File (default)
- [X] MongoDB & RabbitMQ
- [ ] Other (Specify in Additional Information)
### Additional Information
It's not an easy bug and if I investigate the TCP traffic with Wireshark after disabling the SSL between scrapoxy and proxy, I could find this error send back from the proxy to scrapoxy:
HTTP/1.1 500 connect_error
X-Scrapoxy-Proxyerror: write after end
![image](https://github.com/fabienvauchelles/scrapoxy/assets/593562/8059c697-0521-493e-9d26-b687694edc04)
You can assign me this issue as I'm working on it.
Current Behavior
When the scrapoxy instance is overloaded, we could have some socket hang up.
Expected Behavior
We do not have any error.
Steps to Reproduce
Failure Logs
HTTP/1.1 500 connect_error X-Scrapoxy-Proxyerror: write after end