net4people / bbs

Forum for discussing Internet censorship circumvention
3.23k stars 76 forks source link

Issues with Trading & Banking Apps and Google Services #375

Open underdog-03 opened 2 weeks ago

underdog-03 commented 2 weeks ago

Thank you all for the great work you are doing.

I'm experiencing some issues when using Xray-core and Sing-box (home-made VPN). Certain websites, services, and apps, such as Google Analytics and some digital trading and banking apps primarily from America or Europe, don't open when the VPN is on. I have tried Warp and Routing, specifically targeting the domains and IPs I wanted, and forcing them to use IPv4. Unfortunately, this didn't work. I also forced all my traffic to go through Cloudflare Warp, but that didn't solve the issue either. I ensured there are no DNS, WebRTC, or IP Address leaks, and I verified that the VPS IP is clean with no blacklist history or bans. Despite all this, it still doesn't work!

For example, if you try to open Google Analytics, it still returns an HTTP ERROR 503. Similarly, some digital trading websites and international banks, mostly American and English, have the same problem. Somehow, they detect the VPN and proxy connections.

Interestingly, these sites and services work fine with other VPN services like ExpressVPN or NordVPN. Is there any solution you can offer to resolve these problems?

Thank you in advance for your help.

KatouMegumi-osu commented 2 weeks ago

Probably a VPS/hosting company IP range blacklist. Your best bet is to get a friend to host your proxy on their home connection. I often experience that too in Russia, not that it's much of a problem for me.

underdog-03 commented 1 week ago

Thank you, @KatouMegumi-osu, for your insight! However, as I mentioned in my previous message, I have ensured there are no DNS, WebRTC, or IP address leaks. I verified that the VPS IP addresses are clean, with no blacklist history or bans. Despite this, it still doesn't work! I have tried using various IP addresses from different locations and even different VPS providers.

There must be a solution to this common problem. Has anyone else experienced similar issues? Are there any other potential solutions or workarounds I could try?

uuonda commented 1 week ago

Are those websites available at all if you request them directly from your VPS?

underdog-03 commented 1 week ago

@uuonda @mmmray

Yes, that's correct. When I use the curl -I command to fetch headers from websites like https://analytics.google.com/, all of them are accessible directly on my VPS server.

root@:~# curl -I https://analytics.google.com/

HTTP/2 301 
location: https://analytics.google.com/analytics/web/
cross-origin-resource-policy: cross-origin
x-content-type-options: nosniff
server: sffe
content-length: 240
x-xss-protection: 0
date: Sun, 07 Jul 2024 15:29:27 GMT
expires: Sun, 07 Jul 2024 15:59:27 GMT
cache-control: public, max-age=1800
content-type: text/html; charset=UTF-8
age: 1779
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
root@:~# ping analytics.google.com

PING analytics.google.com(2001:4860:4802:38::181 (2001:4860:4802:38::181)) 56 data bytes
64 bytes from 2001:4860:4802:38::181 (2001:4860:4802:38::181): icmp_seq=1 ttl=117 time=1.84 ms
64 bytes from 2001:4860:4802:38::181 (2001:4860:4802:38::181): icmp_seq=2 ttl=117 time=1.16 ms
64 bytes from 2001:4860:4802:38::181 (2001:4860:4802:38::181): icmp_seq=3 ttl=117 time=1.16 ms
64 bytes from 2001:4860:4802:38::181 (2001:4860:4802:38::181): icmp_seq=4 ttl=117 time=1.15 ms
64 bytes from 2001:4860:4802:38::181 (2001:4860:4802:38::181): icmp_seq=5 ttl=117 time=1.21 ms
64 bytes from 2001:4860:4802:38::181 (2001:4860:4802:38::181): icmp_seq=6 ttl=117 time=1.33 ms
64 bytes from 2001:4860:4802:38::181 (2001:4860:4802:38::181): icmp_seq=7 ttl=117 time=1.28 ms
^C
--- analytics.google.com ping statistics ---
11 packets transmitted, 11 received, 0% packet loss, time 10014ms
rtt min/avg/max/mdev = 1.129/1.248/1.838/0.195 ms

However, when accessed through VPN client software, the pages fail to load and return errors consistently.

I've extensively tested various client software on both Mac and PC platforms to eliminate it as a client software issue, but the problem persists. Despite trying more than 20 different VPS providers, all accessible via my VPS, the issue remains unresolved. I've also verified the certificate on my VPS to ensure it is valid. The problem does not seem to be related to the VPS, IP configuration, setup, or client software.

I recommend you perform a similar test with your VPS to see if you encounter similar challenges. Here are a few examples of websites I'm having trouble accessing through my VPN:

https://analytics.google.com/ https://developers.google.com/ https://www.halifax.co.uk https://semrush.com/

PS: The strange phenomenon is that while services like analytics.google.com and developers.google.com are unavailable through my VPN connection, other Google services function smoothly with no problem at all with the same VPN connection. 🤔 🥴

wkrp commented 1 week ago

My first thought, like @KatouMegumi-osu, was that this is a case of server-side blocking based on the source IP address, which is not uncommon. Online providers maintain their own risk scores for various IP address ranges (VPS versus residential, for example), so you may experience difficulty even if you don't find your IP address on any blacklist. Here's some past research on server-side blocking:

If, however, you can access the resources directly from the VPS, and you see the same result despite trying many different VPS IP addresses, and there's no problem when using a commercial VPN, then it's most likely some issue with the proxy software you are using, and if so, I'm afraid debugging that is off topic for this forum. It's better to discuss it with the developers. There is a variety of potential causes: a non-browser TLS fingerprint, TLS fingerprint or other side channel information that is inconsistent with the HTTP User-Agent, inconsistencies with information cached by the browser such as cookies or local storage. It could be any of these causes in combination with the VPS IP address. I don't know about developers.google.com, but it would not surprise me if analytics.google.com tried to discern signs of proxying. Banks often employ obscure and questionably effective client fingerprinting techniques in the name of preventing fraud.

In short, server-side blocking based on client IP address is something experienced by many proxy and VPN users, and there's not much you can do about it besides changing where the proxy is located; but if you've eliminated that as a cause, it must be something specific to your setup or the software you're using.

underdog-03 commented 1 week ago

@wkrp Truly appreciate you taking the time to explain this matter in depth and helping me understand the issue better, especially considering your busy schedule with more important matters in this forum. Your advice has been incredibly valuable.

Following your advice and suggestions, I will delve deeper into this issue. I'll follow the pointers you provided and conduct thorough research with the resources you've shared. I will reach out to the developers to see if they have any technical advice or solutions to resolve this issue.

As you mentioned, it's likely something specific to my setup or the proxy software. Thank you once again for your patience and support Rest assured, if I manage to resolve this issue, I will provide an update here.

Many thanks