net4people / bbs

Forum for discussing Internet censorship circumvention
3.19k stars 75 forks source link

Large scale blocking of Reality and (relayed) WS + TLS protocols in Iran (MCCI; AS43358) #277

Open NikolajHansen23 opened 10 months ago

NikolajHansen23 commented 10 months ago

Starting a few days ago, there have been widespread reports that MCCI is blocking Reality-based VPNs in a matter of hours and with low traffic. Additionally, WS + TLS VPNs that take advantage of CDNs (e.g., Cloudflare) are barely working. Even if you use a domestic (Iranian) CDN (e.g., Arvan Cloud), which generally has access to a less censored Internet, they almost don't work anymore.

Here are some of my observations:

So, what are the next steps now? When this new system gets used on all the main providers in Iran, I don't think there will be much more options left. Using a domestic relay (whether a CDN or server) was always deemed the last resort to escape censorship.

mmmray commented 10 months ago

Additionally, WS + TLS VPNs that take advantage of CDNs (e.g., Cloudflare) are barely working

I am not sure if it still applies, but I managed with this scheme last week:

  1. do not use TLS, but plain HTTP. then, use vmess inside ws for privacy
  2. point clients to specific IPv4 addresses, not the domain itself. there is a large variance in which IP addresses work and which don't. . anything starting with 104. is broken, but cloudflare typically gives you two A records for their proxy
  3. configure cloudflare to have a wildcard (*.foo.example.com), and randomize host header in client config (a.foo.example.com, etc)

also, plain tcp+vmess continues to work for as long as you can afford to switch IP regularly.

i want to try ipv6-only proxies next

Phoenix-999 commented 10 months ago

MCI is blocking REALITY servers in Iran #2451

Just wanted to shed some more light on this situation. Fired up a couple of brand-new VPS servers with clean IPs, and just to be extra safe, I threw in a CDN certificate for good measure. And to top it off, I went ahead and blocked all IR domains and IPs directly on the server.

No big traffic spikes here, just a handful of small video files being sent back and forth for a legit speed test. And to keep things interesting, I've got the VPN running on a couple of devices as well.

Rest assured, all the devices are locked down tight. Our client apps and software are up-to-date and set up perfectly. I've only used "Reality" on 443 port and I've got us covered with whitelisted SNI, and I've gone the extra mile to implement every possible security measure to the best of my understanding.

Out of the three VPS instances, one got the Blocked within just 4 hours, while the other two managed to hold on for about a day before they were also blocked.

The second and third VPS instances started acting up with erratic and unusually high ping, causing a significant slowdown in speed. After a few hours of this instability, they eventually ended up completely blocked. It's starting to look like a potential DDoS attack on the servers.

I hope the information above sheds some light on the subject.

us254 commented 10 months ago

Problems:

  1. Blocking of Reality-based VPNs: There are reports of MCCI (Iranian ISP) blocking Reality-based VPNs within a short timeframe and with low traffic.

  2. WS + TLS VPNs with CDNs: WS + TLS VPNs utilizing CDNs (like Cloudflare) are experiencing limited functionality and are barely working.

  3. Ineffectiveness of Domestic CDNs: Even using domestic Iranian CDNs (e.g., Arvan Cloud) is not providing reliable access to a less censored internet.

  4. Unchanged Connection to Uncensored Websites: The new blocking system seems to be able to detect specific protocols rather than causing general disruption to all connections. Access to previously uncensored websites remains unaffected.

  5. SNI Detection Issues: The SNI (Server Name Indication) used in Reality connections is not working effectively anymore. Switching to new SNIs has had mixed success.

  6. Direct WS + TLS Connections Unusable: Direct WS + TLS connections have been nearly unusable for the past 6 months.

  7. Potential DDoS Attack: Some VPS instances experienced erratic behavior and high ping, leading to slowdowns and eventual blocking. This behavior could indicate a potential Distributed Denial of Service (DDoS) attack.

Solutions and Suggestions:

  1. Use Plain HTTP and VMess Inside WS: One user suggests using plain HTTP instead of TLS, and then utilizing VMess inside WebSocket (WS) for privacy. This approach may help in bypassing the blocking mechanisms.

  2. Point to Specific IPv4 Addresses: Instead of relying on domain names, pointing clients to specific IPv4 addresses can potentially bypass the blocking. Cloudflare's proxy provides multiple A records that might have varying levels of effectiveness.

  3. Randomize Host Headers: Configuring Cloudflare with a wildcard (*.foo.example.com) and randomizing the host header in client configurations can help avoid detection.

  4. Consider IPv6-Only Proxies: There's a suggestion to try using IPv6-only proxies as a potential solution.

  5. Whitelisted SNI: Implementing whitelisted SNI (Server Name Indication) can potentially help in evading detection and blocking.

NikolajHansen23 commented 10 months ago

Problems:

  1. Blocking of Reality-based VPNs: There are reports of MCCI (Iranian ISP) blocking Reality-based VPNs within a short timeframe and with low traffic.
  2. WS + TLS VPNs with CDNs: WS + TLS VPNs utilizing CDNs (like Cloudflare) are experiencing limited functionality and are barely working.
  3. Ineffectiveness of Domestic CDNs: Even using domestic Iranian CDNs (e.g., Arvan Cloud) is not providing reliable access to a less censored internet.
  4. Unchanged Connection to Uncensored Websites: The new blocking system seems to be able to detect specific protocols rather than causing general disruption to all connections. Access to previously uncensored websites remains unaffected.
  5. SNI Detection Issues: The SNI (Server Name Indication) used in Reality connections is not working effectively anymore. Switching to new SNIs has had mixed success.
  6. Direct WS + TLS Connections Unusable: Direct WS + TLS connections have been nearly unusable for the past 6 months.
  7. Potential DDoS Attack: Some VPS instances experienced erratic behavior and high ping, leading to slowdowns and eventual blocking. This behavior could indicate a potential Distributed Denial of Service (DDoS) attack.

Solutions and Suggestions:

  1. Use Plain HTTP and VMess Inside WS: One user suggests using plain HTTP instead of TLS, and then utilizing VMess inside WebSocket (WS) for privacy. This approach may help in bypassing the blocking mechanisms.
  2. Point to Specific IPv4 Addresses: Instead of relying on domain names, pointing clients to specific IPv4 addresses can potentially bypass the blocking. Cloudflare's proxy provides multiple A records that might have varying levels of effectiveness.
  3. Randomize Host Headers: Configuring Cloudflare with a wildcard (*.foo.example.com) and randomizing the host header in client configurations can help avoid detection.
  4. Consider IPv6-Only Proxies: There's a suggestion to try using IPv6-only proxies as a potential solution.
  5. Whitelisted SNI: Implementing whitelisted SNI (Server Name Indication) can potentially help in evading detection and blocking.

Regarding the solutions and suggestions:

  1. Plain HTTP and VMess inside WS doesn't work for me at all. (Both on MCCI and MTN)
  2. Using specific Cloudflare IPs has mixed results for MTN (it seems the GFW eases or aggravates connections) but almost doesn't work at all on MCI.
  3. I have yet to try this one.
  4. As far as I know, IPv6 only works for MTN thus IPv6-based solutions won't work on MCCI where new solutions are most needed.
  5. AFAIK, speedtest.net is one of the whitelisted SNIs in Iran for almost all providers. I suppose there's more to this new detection system than just using a whitelisted SNI.

However, One possible solution could be choosing an SNI that's hosted on your VPS provider. So if you have a VPS from Google Cloud, you must choose an SNI from the whitelisted websites hosted on GCP. (e.g., gmail.com)

Phoenix-999 commented 10 months ago

Just an Update

These serverless methods continue to function with MCI. ✔️ Workers + Fragment ✔️ Warp app by changing endpoints IP

hawshemi commented 10 months ago

This is not a protocol blocking. MCI and TCI just limit the DL/UL speed after some high-traffic usage from the server. and after 2-3 days they block the IP address based on the traffic usage. It is the upgraded Iran's GFW.

NikolajHansen23 commented 10 months ago

This is not a protocol blocking. MCI and TCI just limit the DL/UL speed after some high-traffic usage from the server. and after 2-3 days they block the IP address based on the traffic usage. It is the upgraded Iran's GFW.

If they were arbitrarily limiting the speed to IPs that see high-traffic, it should have severely impacted non-VPN servers (i.e., normal websites) too. My observation (and inspection with Netreach) shows that normal websites are almost unaffected by MCCI.

Phoenix-999 commented 10 months ago

Update - Thu 17 Aug - 13:50

Multiple reports from various locations across Iran indicate a common issue: encountering notable throttling, severe disruption, and substantial fluctuations on different ISPs. This scenario strongly suggests the possibility of a coordinated attack, potentially orchestrated by MCI.

We speculate that the recent events and peculiar behavior exhibited by ISPs in Iran could be indicative of some form of preparation or demonstration of power, potentially in anticipation of the upcoming commemoration of Mahsa Amini and other brave individuals who tragically killed and murdered during last year's uprising against the dictatorial regime.

As a wise man once said: "We can never forget that no matter how much it hurts, how dark it gets, or how far we fall, we are never out of the fight." ✌🏾

hawshemi commented 10 months ago

This is not a protocol blocking. MCI and TCI just limit the DL/UL speed after some high-traffic usage from the server. and after 2-3 days they block the IP address based on the traffic usage. It is the upgraded Iran's GFW.

If they were arbitrarily limiting the speed to IPs that see high-traffic, it should have severely impacted non-VPN servers (i.e., normal websites) too. My observation (and inspection with Netreach) shows that normal websites are almost unaffected by MCCI.

They are not affected because they are whitelisted. You can test it now. Buy a new VPS and setup s simple nginx webserver. And then upload some heavy files on that from MCI after 1Gb or 2Gb data the DL/UL speed will be limited to 0.1 mb/s. This new behavior is for MCI and TCI ISPs. They just 'Graylist' the IP address unless it's listed as a whitelisted IP address from a certain database or timestap snapshot!

Phoenix-999 commented 10 months ago

Recent reports have surfaced, indicating that specific individuals have received warning messages from domestic VPS providers. These notifications inform them that their VPS IP addresses have been blocked due to the detection of VPN or proxy services on their servers. Additionally, they are advised not to generate any traffic for the next 72 hours.

Here some an example of warning messages

photo_5967755331947839123_x

Has anyone else encountered the same problem?

hawshemi commented 10 months ago

Recent reports have surfaced, indicating that specific individuals have received warning messages from domestic VPS providers. These notifications inform them that their VPS IP addresses have been blocked due to the detection of VPN or proxy services on their servers. Additionally, they are advised not to generate any traffic for the next 72 hours.

Here some an example of warning messages

photo_5967755331947839123_x

Has anyone else encountered the same problem?

Yes this is common among Iranian VPS providers. You should buy from websites that dont have limitations for tunneling and proxy servers but the traffic is expensive as hell... (1Gb=500Tomans and more...) Also don't use iptalbes or nftables or common tunneling solutions. They will be detected easily.

Phoenix-999 commented 10 months ago

Yes this is common among Iranian VPS providers. You should buy from websites that dont have limitations for tunneling and proxy servers but the traffic is expensive as hell... (1Gb=500Tomans and more...) Also don't use iptalbes or nftables or common tunneling solutions. They will be detected easily.

I'm working to establish a connection between these incidents and coincidences to ascertain whether they are linked to the coordinated attack by MCI. This is prompted by the fact that numerous individuals have reported receiving the same kind of messages over the last 2 to 3 days.

OR it's a valid point that we might be experiencing heightened paranoia and caution as a result of the simultaneous occurrence of multiple events!!!

mmmray commented 10 months ago

They are not affected because they are whitelisted. You can test it now. Buy a new VPS and setup s simple nginx webserver. And then upload some heavy files on that from MCI after 1Gb or 2Gb data the DL/UL speed will be limited to 0.1 mb/s.

I would like to see this test behind cloudflare CDN. in order for this censorship method to be effective with cloudflare, one would have to implement bandwidth limits both per destination IP and SNI. or is cloudflare currently just collateral damage?

asked another way: is bandwidth limiting done individually per-IP and per-SNI, or per tuple of (IP, SNI)? In the latter case, randomizing subdomains in SNI would help (which can be done even on cloudflare)

same question about the whitelist. do we think the whitelist is about IPs, SNI or the combination of both

mmmray commented 10 months ago

Plain HTTP and VMess inside WS doesn't work for me at all. (Both on MCCI and MTN)

I believe this has stopped working on exactly Aug 16 13:40 UTC, or at least that is the occurrence where I see a large bandwidth drop.

interesting: a proxy of mine where the outbound bandwidth is throttled to 7 MiB/s (intentionally, to control cost) in total continues to work fine at the same bandwidth usage. I think this corroborates @hawshemi's statements

NikolajHansen23 commented 10 months ago

This is not a protocol blocking. MCI and TCI just limit the DL/UL speed after some high-traffic usage from the server. and after 2-3 days they block the IP address based on the traffic usage. It is the upgraded Iran's GFW.

If they were arbitrarily limiting the speed to IPs that see high-traffic, it should have severely impacted non-VPN servers (i.e., normal websites) too. My observation (and inspection with Netreach) shows that normal websites are almost unaffected by MCCI.

They are not affected because they are whitelisted. You can test it now. Buy a new VPS and setup s simple nginx webserver. And then upload some heavy files on that from MCI after 1Gb or 2Gb data the DL/UL speed will be limited to 0.1 mb/s. This new behavior is for MCI and TCI ISPs. They just 'Graylist' the IP address unless it's listed as a whitelisted IP address from a certain database or timestap snapshot!

This is super interesting. However there are still some questions:

  1. If they are whitelisting websites based on their IPs, this means that many IPs are now "clean." As almost all websites use major providers' CDNs, this would mean that you can find a lot of "clean" IPs on big providers such as Azure, GCP, and AWS? I'm not sure, but it'd also mean that you could use Cloudflare as a reverse proxy for your Reality server and put a white-listed website that also uses Cloudflare as the SNI.
  2. The other one is I've seen some configs that had stopped working with [speedtest.net](http://speedtest.net/) SNI but started working again with [www.theverge.com](http://www.theverge.com/). How does that fit into your theory that they are only limiting IPs with high traffic?
mmmray commented 10 months ago

The other one is I've seen some configs that had stopped working with speedtest.net SNI but started working again with www.theverge.com. How does that fit into your theory that they are only limiting IPs with high traffic?

It could fit if the bandwidth throttling happens on a tuple/combination of (IP, SNI). more tests are needed

markpash commented 10 months ago

The other one is I've seen some configs that had stopped working with speedtest.net SNI but started working again with www.theverge.com. How does that fit into your theory that they are only limiting IPs with high traffic?

It could fit if the bandwidth throttling happens on a tuple/combination of (IP, SNI). more tests are needed

I won't rule this out, but it's highly unlikely imo.

I've been experimenting with reality, with around 10 clients over a few weeks. Around the same time that there were power outages in Tehran, I noticed the IP of the server was suddenly blocked across all providers. I switched a few of them to a new server, and that IP was also blocked within a few hours. I migrated one user to another server on a very little-known VPS provider and that was blocked after about 2 hours. The user didn't even pull over 800KB/s, and didn't transfer more than a few hundred MB in that time window. I didn't change the keys or SNI during that 24 hour window of all these events. There are a few possible methods of filtering that I can think of with this, but I'm not sure yet.

Could it be that they are monitoring all SNIs that seem to have a high amount of traffic, collecting the destination IPs, then batch resolving those SNIs to find real IPs, then blocking the non-real ones?

The fact that blocking or throttling doesn't happen immediately tells me there's a window where data is collected, and batch operations or waves of blocking/throttling occurs.

When throttling is happening, it's likely they observe connection counts for the IP+SNI tuple, then take that IP and throw it in a general block/throttle list. It's very unlikely to be actively done in realtime, as that kind of detection would be CPU expensive.

Keep in mind that this new mechanism that's being observed applies simultaneously on all downstream civilian internet providers. Which means that the system was either deployed and enabled at all operator exchanges or datacenters at the same time, or that the new system is operating at a higher aggregation point in the national network. I personally haven't seen it like this before, except for the basic things like the DNS manipulation system.

An idea that may both prevent the IP+SNI tuple from being detected in the first place, and also to put a lot of load on the filtering systems, is to perhaps fragment the TLS header of REALITY??? Because a filtering system would have to actively store a 4tuple (src+dest IP+port) and wait a really long time before it can build the full SNI in memory. The filtering system would have to forget the 4tuple and partial SNI after a small time, or run the risk of running out of memory. (if the number of connections doing this is high)

mmmray commented 10 months ago

Could it be that they are monitoring all SNIs that seem to have a high amount of traffic, collecting the destination IPs, then batch resolving those SNIs to find real IPs, then blocking the non-real ones?

the idea that ISPs maintain table of SNI to allowed IPs for the top N sites has been widely discussed in a lot of older threads from around this year (in this BBS but also in xray-core issuetracker). we can assume it exists, and it would still allow for the possibility that switching SNI from speedtest.net to theverge.com unblocks traffic. because theverge.com is simply not a popular enough domain to be in that table, so its IP is not validated at all.

it doesn't explain though why you can switch IPs and have a time window where the proxy works. assuming you do not switch SNI, and your old proxy has been banned via the above mechanism (ie. SNI-based) and not something else, the new IP should never work

therefore it seems more likely to me that the new mechanism you are interacting with is not SNI-based at all

hawshemi commented 10 months ago

This is super interesting. However there are still some questions:

  1. If they are whitelisting websites based on their IPs, this means that many IPs are now "clean." As almost all websites use major providers' CDNs, this would mean that you can find a lot of "clean" IPs on big providers such as Azure, GCP, and AWS? I'm not sure, but it'd also mean that you could use Cloudflare as a reverse proxy for your Reality server and put a white-listed website that also uses Cloudflare as the SNI.
  2. The other one is I've seen some configs that had stopped working with [speedtest.net](http://speedtest.net/) SNI but started working again with [www.theverge.com](http://www.theverge.com/). How does that fit into your theory that they are only limiting IPs with high traffic?

I don't know if you are currently in Iran and testing these methods but in theory, "yes" we can use those clean IPs. but currently, Iran's GFW just throttles the IPs from Cloudflare (UL speed) in the first place! so for most of the operators the idea of "CF clean IP" is almost dead (but not completely for "some" ISPs).

As of the top VPS providers:

  1. GCP is completely blocked due to sanctions (google blocked the IP addresses from Iran)
  2. Azure has clean IPs (in the EU zone) and can be used as v2ray/xray/hysteria/... protocols. But from 1 week ago (after the GFW upgrades), the IP address will be throttled after using REALITY on MCI/TCI (even with all the security measures)
  3. AWS has "some" clean IPs but you should test and change IP for like an hour or two to get a "semi-clean" IP which connects to "some" ISPs and is blocked at other ISPs!

For the SNI changing, it could be possible that the GFW just observes and save the IP/SNI/FP dict. because many of my servers are just blocked by GFW but after changing the FP (fingerprint) it connects perfectly! this theory is also valid... more tests are needed.

hawshemi commented 10 months ago

Could it be that they are monitoring all SNIs that seem to have a high amount of traffic, collecting the destination IPs, then batch resolving those SNIs to find real IPs, then blocking the non-real ones?

the idea that ISPs maintain table of SNI to allowed IPs for the top N sites has been widely discussed in a lot of older threads from around this year (in this BBS but also in xray-core issuetracker). we can assume it exists, and it would still allow for the possibility that switching SNI from speedtest.net to theverge.com unblocks traffic. because theverge.com is simply not a popular enough domain to be in that table, so its IP is not validated at all.

it doesn't explain though why you can switch IPs and have a time window where the proxy works. assuming you do not switch SNI, and your old proxy has been banned via the above mechanism (ie. SNI-based) and not something else, the new IP should never work

therefore it seems more likely to me that the new mechanism you are interacting with is not SNI-based at all

I don't think it's just one change. It has changed from a week ago with two or more mechanics updates. One of them is blocking the IP address of the server based on SNI. One of them is simply just the amount of traffic on the server. I just changed the SNI from www.speedtest.net to www.theverge.com and it is working ok now. (but no one knows for how long). The other thing is just the combination of the server IP address and SNI and Fingerprint.

computerscot commented 10 months ago

So, what are the next steps now? When this new system gets used on all the main providers in Iran, I don't think there will be much more options left. Using a domestic relay (whether a CDN or server) was always deemed the last resort to escape censorship.

At the end of my summary of methods, I listed some "last resort" methods. You might try some of those. But there's a general principle here. If a method ever becomes too popular, it gets blocked. You have to be continually looking for something new and different.

hawshemi commented 10 months ago

Discussion: https://github.com/XTLS/Xray-core/discussions/2450

NikolajHansen23 commented 10 months ago

I don't know if you are currently in Iran and testing these methods but in theory, "yes" we can use those clean IPs. but currently, Iran's GFW just throttles the IPs from Cloudflare (UL speed) in the first place! so for most of the operators the idea of "CF clean IP" is almost dead (but not completely for "some" ISPs).

As of the top VPS providers:

  1. GCP is completely blocked due to sanctions (google blocked the IP addresses from Iran)
  2. Azure has clean IPs (in the EU zone) and can be used as v2ray/xray/hysteria/... protocols. But from 1 week ago (after the GFW upgrades), the IP address will be throttled after using REALITY on MCI/TCI (even with all the security measures)
  3. AWS has "some" clean IPs but you should test and change IP for like an hour or two to get a "semi-clean" IP which connects to "some" ISPs and is blocked at other ISPs!

For the SNI changing, it could be possible that the GFW just observes and save the IP/SNI/FP dict. because many of my servers are just blocked by GFW but after changing the FP (fingerprint) it connects perfectly! this theory is also valid... more tests are needed.

Thanks for your reply.

Regarding GCP, you're unfortunately right. It's a shame for Google honestly.

For Cloudflare there are dozens of websites that are on Cloudflare and you can access them fine with MCI (e.g., discord.com) but if you use discord.com as your address in a WS + TLS config behind Cloudflare CDN, the download speed is almost zero too. So this means there's some form of recognition of normal browsing from WS + TLS for the filtering system.

What I'm looking for is to use Cloudflare as a reverse proxy for a Reality VPN server. You would use discord.com as the SNI and Cloudflare's IP for the address. Theoretically, If they are unable to detect Reality's traffic characteristics, this should work. (I'll try to test this out and report back)

NikolajHansen23 commented 10 months ago

An interesting observation:

If you find "clean" IPs for Cloudflare, WS + TLS configs work somewhat decently for MCCI. However, WS + TLS behind a domestic CDN (e.g., Arvan) almost doesn't work. Additionally, WS + TLS behind Arvan works perfectly for MTN.

Since they are not catching VPN connections to some clean CF IPs, it tells me that it's not about traffic characteristics, and it's all about IPs. However, Why doesn't it work with Arvan then?

I really can't make sense of what's happening here.

us254 commented 10 months ago
  1. Blocking Behavior:

    • MCI and TCI ISPs in Iran throttle DL/UL speeds after high-traffic usage.
    • IPs are blocked after 2-3 days of heavy traffic.
    • Non-VPN servers (normal websites) are not affected due to whitelisting.
  2. Graylisting and IP Whitelisting:

    • IPs are 'graylisted' unless whitelisted in a specific database.
    • Blocking occurs in waves or batch operations after data collection.
  3. Data Collection and Blocking Mechanism:

    • Monitoring SNIs with high traffic to collect destination IPs.
    • Batch resolving SNIs to find real IPs for blocking.
    • Throttling based on observed connection counts for IP+SNI tuple.
  4. Mitigation Strategies:

    • Fragmenting TLS header to hinder SNI detection and increase load on filters.
    • Changing IP and fingerprint to bypass GFW blocking.
  5. Cloud Service Providers (CSPs) Status:

    • GCP blocked due to sanctions; IPs from Google are inaccessible.
    • Azure throttles IPs after using REALITY on MCI/TCI.
    • AWS IPs work with varied success across ISPs; frequent FP changes help.
hawshemi commented 10 months ago
  • Non-VPN servers (normal websites) are not affected due to whitelisting.

Some users reported even with a simple nginx webserver and uploading to it, it will be limited after 2-3gb of traffic.

mmmray commented 10 months ago

What I'm looking for is to use Cloudflare as a reverse proxy for a Reality VPN server. You would use discord.com as the SNI and Cloudflare's IP for the address.

cloudflare does not allow for domain fronting so it won't happen. you can use arbitrary cloudflare IP to connect to any website, but stealing somebody else's SNI and connecting to your own server won't work

oxer-0 commented 10 months ago

An idea that may both prevent the IP+SNI tuple from being detected in the first place, and also to put a lot of load on the filtering systems, is to perhaps fragment the TLS header of REALITY??? Because a filtering system would have to actively store a 4tuple (src+dest IP+port) and wait a really long time before it can build the full SNI in memory. The filtering system would have to forget the 4tuple and partial SNI after a small time, or run the risk of running out of memory. (if the number of connections doing this is high)

Combination of XTLS and Fragment does not work

[Warning] [328153325] app/proxyman/outbound: failed to process outbound traffic > proxy/vless/outbound: XTLS only supports TLS and REALITY directly for now.

I also tested on a working VLESS + TCP + Reality instance, When I activate the fragments for Reality outbound on the client, It fails to finish SSL handshake.

* Uses proxy env variable all_proxy == 'socks5h://127.0.0.1:10801'
*   Trying 127.0.0.1:10801...
* Connected to 127.0.0.1 (127.0.0.1) port 10801 (#0)
* SOCKS5 connect to ifconfig.io:443 (remotely resolved)
* SOCKS5 request granted.
* Connected to 127.0.0.1 (127.0.0.1) port 10801 (#0)
* ALPN: offers h2,http/1.1
* (304) (OUT), TLS handshake, Client hello (1):
*  CAfile: /etc/ssl/cert.pem
*  CApath: none
* LibreSSL SSL_connect: SSL_ERROR_SYSCALL in connection to ifconfig.io:443 
* Closing connection 0
curl: (35) LibreSSL SSL_connect: SSL_ERROR_SYSCALL in connection to ifconfig.io:443 

I also tried setting sockopt.dialerProxy, Xray-core does take it into account but then setting it into an invalid value also works...

quackerd commented 10 months ago
  • Non-VPN servers (normal websites) are not affected due to whitelisting.

Some users reported even with a simple nginx webserver and uploading to it, it will be limited after 2-3gb of traffic.

Meanwhile someone reported that tunnelling 30G data per day through SSH does not result in a block. https://github.com/XTLS/Xray-core/issues/2451#issuecomment-1684935929. Does this incident mainly affect TLS stuff?

NikolajHansen23 commented 10 months ago

Okay. Did some more experiments and here are the results:

1- You need a "clean" IP for your WS + TLS config to work even behind Arvan CDN for MCI. For some IPs, you could perfectly use Arvan CDN and MCI but for some others, it just doesn't work at all. Additionally, A lot of those IPs that don't work for MCI, work perfectly with MTN.

I have no idea how this could happen other than Arvan collaborating with the filtering system, which seems unlikely honestly. So this means that Arvan's edge servers would decline to fulfill a proxy request if it's coming from MCI AND the IP is grey-listed but would serve MTN requests regardless? Moreover, one of my IPs that doesn't work with MCI + Arvan CDN is working perfectly with MCI + Cloudflare CDN.

I'd love to know what you guys think about this.

2- For those IPs that MCI + Arvan CDN work, you could have perfect direct WS + TLS connections too. (Vless, Vmess, Trojan). But for the IPs that MCI + Arvan CDN didn't work, direct WS + TLS connections don't work either.

3- I also toyed with using Reality with some SNIs that were in the same ASN as my VPN server but they didn't work. There still doesn't seem to be a systematic "method" to choose an SNI that makes it indistinguishable from the "real" traffic. A couple of SNIs work nicely for MCI, but nobody knows why they work and how detectable they are.

mmmray commented 10 months ago

I have similar experiences, I would go further and say a truly clean ip unlocks basically every protocol (reality, ws and TCP+vmess) at the moment. what is less clear to me is whether it is a particular traffic pattern or a particular protocol that gets the IP banned

us254 commented 10 months ago

I also tested on a working VLESS + TCP + Reality instance, When I activate the fragments for Reality outbound on the client, It fails to finish SSL handshake.- oxer-0

Fragment only works in VLESS_WS_TLS setup.

us254 commented 10 months ago

Sure, here's the provided text with the requested formatting:

Markdown this: Link to the GitHub Comment

感谢你们的讨论,请做一个测试,步骤如下:

1: 使用 dokodemo-door 转发服务端的 80 和 443 端口至 www.speedtest.net 2: 将 hosts 设为自己的 IP,浏览器访问几次,看会不会被封 3: 重复 1,但使用 iptables 进行转发,看会不会被封 这有助于确定 MCI 是否针对 www.speedtest.net 设置了 traffic characters 白名单


Testing Results: Someone tested, None of Three Methods Blocked.

ignoramous commented 10 months ago

Warp app by changing endpoints IP

Any Cloudflare IP as a WARP endpoint works? Even 1.1.1.1?

NikolajHansen23 commented 10 months ago

I'm thinking out loud here. Let's say they can't detect Reality's traffic characteristics. Then they get suspicious of a connection and put it into a "further analysis" bucket.

Let's say they have a list of whitelisted non-Iranian domains? Then they'd like to do some form of reverse lookup to check if that IP indeed belongs to the SNI that was sent with it. How do they do this?

Do they check the DNS ptr record? Do they access the IP to see where it'll finally redirect to? Do they use something like certificate common names (CN)?

Understanding this would be the key to making Reality almost undetectable for now.

NikolajHansen23 commented 10 months ago

I'm thinking out loud here. Let's say they can't detect Reality's traffic characteristics. Then they get suspicious of a connection and put it into a "further analysis" bucket.

Let's say they have a list of whitelisted non-Iranian domains? Then they'd like to do some form of reverse lookup to check if that IP indeed belongs to the SNI that was sent with it. How do they do this?

Do they check the DNS ptr record? Do they access the IP to see where it'll finally redirect to? Do they use something like certificate common names (CN)?

Understanding this would be the key to making Reality almost undetectable for now.

I'd like to know what you think since my comment was inspired by your comment. @markpash

computerscot commented 10 months ago

Then they'd like to do some form of reverse lookup to check if that IP indeed belongs to the SNI that was sent with it.

You can get the IP and SNI to match using the "steal oneself" technique invented by @chika0801. To implement this, you'll need to own a domain name that is either whitelisted or graylisted.

markpash commented 10 months ago

I think this is it really, with REALITY there's no solid proof yet that the way the stream looks on the wire has been identified by DPI systems. For now, our best guess is that they simply look at the pattern of traffic or the destination for these TLS connections and have some form of heuristic.

So when doing normal REALITY, when you pick a dest you need to make sure that your IP acts identically to that host in every way. So if the dest is listening on port 80, then you need to make sure to make port 80 also proxy/forward to the dest. If the real dest doesn't have an ssh server listening on port 22, then you shouldn't either. (note with ssh, make sure to use different ssh keypairs from what you use on sites like github, they can use this to find your servers, blog in persian here)

If your hosting provider does reverse dns records, delete those. If you do dig lookups for your chosen dest and the IPs are consistently the same, from the same provider, and it's normally a low-traffic website, then as soon as you put a lot of traffic on that SNI, a heuristic that does a DNS lookup on that SNI to discover the real dest will flag your server. (assuming they have such a heuristic).

The stealoneself method is a good move, assuming you can be sure that your fake site or whatever you choose to host, won't get blocked. On the other hand, rotating domains is more expensive than SNI, depending on how the filters behave with subdomains.

markpash commented 10 months ago

I also tested on a working VLESS + TCP + Reality instance, When I activate the fragments for Reality outbound on the client, It fails to finish SSL handshake.

Maybe existing tools don't allow it, but in theory REALITY is just TLS, what's stopping you from fragmenting the SNI in the REALITY ClientHello?

NikolajHansen23 commented 10 months ago

You can get the IP and SNI to match using the "steal oneself" technique invented by @chika0801. To implement this, you'll need to own a domain name that is either whitelisted or graylisted.

I couldn't find this steal oneself technique. Would you please send me a pointer to it?

computerscot commented 10 months ago

I couldn't find this steal oneself technique. Would you please send me a pointer to it?

Here are the configuration JSON files:

https://github.com/chika0801/Xray-examples/tree/main/VLESS-XTLS-uTLS-REALITY/steal_oneself

Here are my implementation notes:

https://computerscot.github.io/vless-xtls-utls-reality-steal-oneself.html

chika0801 commented 10 months ago

@computerscot

I read your post and have a question.

https://github.com/chika0801/Xray-examples/blob/main/VLESS-XTLS-uTLS-REALITY/steal_oneself/nginx.conf#L31

As this nginx configuration uses the code 80 jump to port 443. You need to tell someone in your article that if they are using acme/cerbot's standalone mode to apply for SSL certificates, please ask them to remove this piece of code or add #'s to comment out the code. Otherwise, acme/cerbot's automatic renewal of SSL certificates every 3 months will fail. (Because port 80 is occupied by nginx)

You can tell people that the 80 to 443 code is not a problem even if it is not used.

auto_update_acme.sh File permissions are 0755

#!/usr/bin/env bash
systemctl stop nginx
/root/.acme.sh/acme.sh --cron --home /root/.acme.sh
systemctl start nginx

I am using crontab to design a timed task to handle them all at the same time. It's not technical, but it's simple and works well.

crontab -e

0 0 1 * * /root/auto_update_acme.sh

tips:

https://github.com/chika0801/Xray-examples/blob/main/VLESS-XTLS-uTLS-REALITY/steal_oneself/nginx.conf#L40

ssl_reject_handshake on.

With this parameter, I tested using the tool https://github.com/XTLS/RealiTLScanner on my own VPS, and it protects my domain well (from being used as a URL in REALITY DEST)

chika0801 commented 10 months ago

In China with steal their own form, the main thing is that we will occasionally encounter when not use steal their own form, connect our own VPS, may be for no reason the speed of the time fast and slow, or not connected after a while and then recover themselves. From my personal use of VPS, I have used the form of stealing their own, did not appear above some unknown reasons for the phenomenon. If you need to try this way, you can try in your country (region) how the effect.

Of course, if your country (region) has SNI whitelist, only demand you to visit SNI whitelisted sites, your own domain name is obviously not on the SNI whitelist, which is also a problem.

chika0801 commented 10 months ago

@computerscot

https://computerscot.github.io/vless-xtls-utls-reality-steal-oneself.html

https://github.com/XTLS/Xray-core#usage

I recommend you to go to Xray's project and send a PR request to add this article request of yours to the Installation Guide project, it can help more people.

computerscot commented 10 months ago

PR created. https://github.com/XTLS/Xray-core/pull/2470 Thanks for your help!

NikolajHansen23 commented 10 months ago

Update:

It seems like the affected IPs are now working again. Both for Reality and WS + TLS behind Arvan CDN.

I still can't wrap my head around that Arvan CDN was working for MTN but not for MCI in these couple of days when these IPs were throttled/banned by MCI.

We can only speculate, but maybe they have been just testing the new system for broader deployment or deployment when the times are tough for the government.

Phoenix-999 commented 10 months ago

Update:

It appears that the previously affected Domain/Subdomains that were Filtered & Blocked are now up and running again, and they've been unblocked!

We're a bit puzzled about the reasons behind these behavior, and We suspect that this might be in preparation for the anniversary of MAHSA AMINI and the people's uprising against the dictator regime from last year, but as of now, it's all speculation.

It's possible that The Iranian regime are conducting tests on the new system for a wider rollout. While we can't definitively pinpoint the cause, this turn of events is undoubtedly positive.

It would be greatly appreciated if someone could confirm these changes.

Phoenix-999 commented 10 months ago

I've come across a new and rather strange issue that I wanted to share with you all. Just in case anyone else is experiencing the same phenomenon or perhaps has a reasonable explanation for it.

I've created a new VPS server using a clean IP, carried out my usual setup and configuration routine, and generated the Reality configuration with a clean and whitelisted SNI on port 443. (VLESS+TCP+Reality)

Everything appeared to be in order until I conducted tests across various ISPs and in different cities to ensure that everything is functioning correctly as it should. The client app and software are identical and up-to-date with the latest version.

Interestingly, the same configuration that was functioning flawlessly in cities like Tabriz, Isfahan, and Shiraz is no longer operational in Tehran, the capital city. This issue is observed across multiple ISPs in Tehran.

I've attempted various whitelisted SNI options, but the outcome remains the same, BUT when I’ve changed the port number from 433 to any 4 or 5 digit number the same config with the same SNI working in Tehran as well as other cities with consistent and decent upload and download speeds.

Has anyone else encountered a similar issue?

RPRX commented 10 months ago

https://github.com/XTLS/Xray-core/issues/2451#issuecomment-1698536776

我持续关注伊朗 GFW 的动向,看起来上周它不仅撤回了这个 issue 所讨论的严格过滤,还开放了 QUIC 协议与很多 Cloudflare 的 IP,我也想知道为什么它的做法会 180 度大转弯,有任何内幕消息吗?

I continue to follow the Iranian GFW and it looks like last week it not only withdrew the strict filtering discussed in this issue, but also opened up the QUIC protocol with a lot of Cloudflare IPs, I'd also like to know why it made a 180-degree turn in it's approach, any insider info on that?

Phoenix-999 commented 10 months ago

@RPRX

XTLS/Xray-core#2451 (comment)

I continue to follow the trend of Iran GFW. It seems that last week it not only withdrew the strict filtering discussed in this issue, but also opened up the QUIC protocol and many Cloudflare IPs. I also wonder why it made a 180-degree turn in its approach. Some Any insider tips?

Our speculation is that the Iranian regime has uses various types of psychological techniques and mind games on numerous occasions in the past same as we are witnessing now.

They intentionally downplay restrictions and limitations, while saturating the dissemination of false news through national TV channels and all government websites. This is aimed at creating a false sense of hope and positivity, ultimately leading the general public to believe that things are returning to normal. Of course, by employing these tactics, they effectively deter technical individuals from devising new strategies and solutions in case of a complete blackout of the internet.

If you're wondering why they are implementing these measures at this particular time when about 2 weeks ago we witness the Large scale blocking of Reality!

We believe that this could be linked to the upcoming anniversary of MAHSA AMINI's death and many others and the events surrounding the people's uprising against the dictatorial regime, which resulted in bloodshed last year. The regime might be aiming to prevent any potential gatherings, online discussions, or movements that could gain momentum during this sensitive time.

Exactly, it appears that the Iranian regime is taking preemptive measures to prevent people from organizing and developing new methods to bypass internet filtering and censorship ahead of the anniversary. By implementing these measures now, they aim to disrupt any potential plans or strategies that could emerge closer to the anniversary date. This way, they hope to maintain control over the flow of information and prevent the spread of news or discussions that could challenge their authority.

We must remain vigilant and prepared. A storm is approaching, and we need to have alternative options in place in case of a complete blackout of free internet.