search

root-gg / plik

Plik is a temporary file upload system (Wetransfer like) in Go.
https://plik.root.gg
Other
1.47k stars 168 forks source link

Use custom User-Agent in client library #432

Closed pjakuszew closed 2 years ago

pjakuszew commented 2 years ago

Plik client library uses standard "Go-http-client/1.1" user agent in its HTTP requests. This may cause issues when Plik server is hosted behind a reverse proxy which filters out various User-Agent strings used by malicious bots, "Go-http-client/1.1" being such example.

This change adds an additional User-Agent header to Plik client library, so it can access Plik server instances running behind reverse proxies implementing the User-Agent filtering.

pjakuszew commented 2 years ago

This should explain the problem a bit more clearly:

I configured a private Plik instance on shared hosting service and noticed that Plik client keeps receiving HTTP 403 Forbidden errors. After talking to technical support, I learned that Go-http-client/1.1 is added to their bot blocklist, because it is a User-Agent string commonly used by bots.

See the following example:

# Standard curl User-Agent works fine
$ curl -X POST https://plik.example.org/upload
unable to create upload : anonymous uploads are disabled

# Go-http-client is blocked
$ curl -X POST -A "Go-http-client/1.1" https://plik.example.org/upload
<html>
<head><title>403 Forbidden</title></head>
<body>
<center><h1>403 Forbidden</h1></center>
<hr><center>nginx</center>
</body>
</html>

This pull request aims to fix that issue by presenting a plik_client/<version> User-Agent string instead.

camathieu commented 2 years ago

LGTM, waiting for the test suit to complete to merge

pjakuszew commented 2 years ago

I missed a spot in server/context/errors.go:

-var userAgents = []string{"wget", "curl", "python-urllib", "libwwww-perl", "php", "pycurl", "go-http-client"}
+var userAgents = []string{"wget", "curl", "python-urllib", "libwwww-perl", "php", "pycurl", "go-http-client", "plik_client"}

Custom user agents need to be added in there, otherwise the server will return the 404 in a format which testcases don't like...

pjakuszew commented 2 years ago

c5a3882 fixes the problem, make test ends successfully on my machine.

camathieu commented 2 years ago

Thanks for the contribution.