azukaar / Cosmos-Server

☁️ The Most Secure and Easy Selfhosted Home Server. Take control of your data and privacy without sacrificing security and stability (Authentication, anti-DDOS, anti-bot)
https://cosmos-cloud.io
Other
3.6k stars 129 forks source link

[FEAT]: Option to remove URLs strip slashes #126

Open bruj0 opened 1 year ago

bruj0 commented 1 year ago

What happened?

I am trying to use a paywall remover service: https://github.com/kubero-dev/ladder

Everything works as expected except when the URL is added to the request, one of the / is stripped:

Original: https://paywall.example.net/https://example.com

Redirected: https://paywall.example.net/https:/example.com

What should have happened?

The original URL in the request should be forwarded as is.

How to reproduce the bug?

  1. Add this as a service: https://github.com/kubero-dev/ladder/blob/main/docker-compose.yaml
  2. Try to load an URL

Relevant log output

No response

Other details

No response

System details

azukaar commented 1 year ago

This is not a bug but a feature as those are not valid HTTP links, and should not be used. The author of ladder should use something like /?path=... instead

I will see if I can add a way to disable the slash stripping (can't guarantee it's possible)

bruj0 commented 1 year ago

Thanks for taking a look!

This is not a bug but a feature as those are not valid HTTP links, and should not be used. The author of ladder should use something like /?path=... instead

That is debatable however this is a valid link and is still treated the same:

https://paywall.example.net/https%3A%2F%2Fexample.com

Ie, converting :// to url encoding.

I will see if I can add a way to disable the slash stripping (can't guarantee it's possible)

One possible user friendly option would be that if "Block common Bots" is disabled then no changes are made to the request and passed as is.

Cheers!

azukaar commented 1 year ago

The thing is I think this it is a built-in Go behaviour, because even with StrictSlash(false) I still observe the behaviour, but will investigate