开启SSRR后，Android 8.1/P不认Wifi

kcschan commented 6 years ago

这个故障用8.1的应该都比较熟悉了。

简单描述一下复现方法：系统版本：Android 8.1~P(DP1) 设备接在WiFi上，开启SSRR(本身服务器连接正常)

故障描述：这个时候，很多应用会认为处于移动网络/metered network而拒绝使用它们认为应该仅在WiFi下进行的操作，如： Google photos拒绝备份照片，除非允许cellular backup 下载更新时，会一直停留在install update，但实际并不会进行下载的状态

到了Android P变得更麻烦，play store无法安装应用(因为它认为处于metered network)，可以通过开启play store的unrestricted data usage来临时处理

按照今天的声明，看上去与8.1~P的VPN接口改动有关。

官方到今天才正式声明了这件事，但实际上8.1就应该已经包含了至少一部分的改动：

https://issuetracker.google.com/u/2/issues/68657525

https://developer.android.com/preview/behavior-changes.html 关注#More detailed network capabilities reporting for VPNs 这一节

看上去需要SSRR根据实际网络情况动态调用setUnderlyingNetworks()

https://developer.android.com/reference/android/net/VpnService.html#setUnderlyingNetworks(android.net.Network[])

在madeye的shadowsocks方面也有人指出这件事: https://github.com/shadowsocks/shadowsocks-android/issues/1666

Akkariiin commented 6 years ago

感谢反馈

Akkariiin commented 6 years ago

emmmmm , 根据这里的说法：

https://developer.android.com/reference/android/net/VpnService.html#setUnderlyingNetworks(android.net.Network[])

setUnderlyingNetworks

added in API level 22

boolean setUnderlyingNetworks (Network[] networks)

Sets the underlying networks used by the VPN for its upstream connections.

Used by the system to know the actual networks that carry traffic for apps affected by this VPN in order to present this information to the user (e.g., via status bar icons).

This method only needs to be called if the VPN has explicitly bound its underlying communications channels — such as the socket(s) passed to protect(int) — to a Network using APIs such as bindSocket(Socket) or bindSocket(DatagramSocket). The VPN should call this method every time the set of Networks it is using changes.

networks is one of the following:

*    a non-empty array: an array of one or more Networks, in decreasing preference order. For example, if this VPN uses both wifi and mobile (cellular) networks to carry app traffic, but prefers or uses wifi more than mobile, wifi should appear first in the array.
*    an empty array: a zero-element array, meaning that the VPN has no underlying network connection, and thus, app traffic will not be sent or received.
*    null: (default) signifies that the VPN uses whatever is the system's default network. I.e., it doesn't use the bindSocket or bindDatagramSocket APIs mentioned above to send traffic over specific channels.

This call will succeed only if the VPN is currently established. For setting this value when the VPN has not yet been established, see setUnderlyingNetworks(Network[]).

Parameters
--
networks | Network: An array of networks the VPN uses to tunnel traffic to/from its servers.

Returns
---
boolean | true on success.

这个地方说仅仅只能在已建立VPN连接时使用，而且在变更时需要重新告知系统新的底层网络是什么。（这里的变更应该是指变更使用的底层网络，也就是说存在某些可能实际上不会连接网络的VPN，也可能存在支持负载均衡同时使用移动数据和wifi的应用。）所以我觉得SS/SSR这种情况直接调用 setUnderlyingNetworks (null) 来告诉系统使用的是默认系统连接就好。

但是又说，上面这个只能在以建立连接时使用，还未建立时应该使用以下这个版本：

https://developer.android.com/reference/android/net/VpnService.Builder.html#setUnderlyingNetworks(android.net.Network[])

setUnderlyingNetworks

added in API level 22

VpnService.Builder setUnderlyingNetworks (Network[] networks)

Sets the underlying networks used by the VPN for its upstream connections.

Parameters
networks | Network: An array of networks the VPN uses to tunnel traffic to/from its servers.

Returns
VpnService.Builder | this VpnService.Builder object to facilitate chaining method calls. 

See also:
    setUnderlyingNetworks(Network[])

这个版本起调于VpnService.Builder，也就是配置vpn时使用，但却没有详细说明使用方法，而是引用了上面那个版本。所以我猜应该在启动vpn之前在VpnService.Builder上直接 setUnderlyingNetworks (null) 设置一下就好了。剩下的就不用管了。撒花~~~

kcschan commented 6 years ago

可以用官方模拟器配套的带“play”字样的镜像(不是"google apis"，前者才有play store)测试一下。

这个故障在P变得更麻烦的表现是，似乎是扯上download manager的包括第三方应用都会遇到。

kcschan commented 6 years ago

VpnService.setUnderlyingNetworksForVpn最后应该是由这里执行:

https://github.com/aosp-mirror/platform_frameworks_base/blob/master/services/core/java/com/android/server/ConnectivityService.java setUnderlyingNetworksForVpn(Network[] networks)

public boolean setUnderlyingNetworksForVpn(Network[] networks) {
    int user = UserHandle.getUserId(Binder.getCallingUid());
    final boolean success;
    synchronized (mVpns) {
        throwIfLockdownEnabled();
        success = mVpns.get(user).setUnderlyingNetworks(networks);
    }
    if (success) {
        mHandler.post(() -> notifyIfacesChangedForNetworkStats());
    }
    return success;
}

接下来看Vpn.java:

https://github.com/aosp-mirror/platform_frameworks_base/blob/master/services/core/java/com/android/server/connectivity/Vpn.java

public Builder setUnderlyingNetworks(Network[] networks) {
--
  | mConfig.underlyingNetworks = networks != null ? networks.clone() : null;
  | return this;
  | }

303行的 updateCapabilities(ConnectivityManager cm, Network[] underlyingNetworks, NetworkCapabilities caps)

里面有这么一句

        if (ArrayUtils.isEmpty(underlyingNetworks)) {
            // No idea what the underlying networks are; assume sane defaults
            metered = true;   //
            roaming = false;
            congested = false;
        } else {
            for (Network underlying : underlyingNetworks) {
                final NetworkCapabilities underlyingCaps = cm.getNetworkCapabilities(underlying);
                if (underlyingCaps == null) continue;
                for (int underlyingType : underlyingCaps.getTransportTypes()) {
                    transportTypes = ArrayUtils.appendInt(transportTypes, underlyingType);
                }

                // When we have multiple networks, we have to assume the
                // worst-case link speed and restrictions.
                downKbps = NetworkCapabilities.minBandwidth(downKbps,
                        underlyingCaps.getLinkDownstreamBandwidthKbps());
                upKbps = NetworkCapabilities.minBandwidth(upKbps,
                        underlyingCaps.getLinkUpstreamBandwidthKbps());
                metered |= !underlyingCaps.hasCapability(NET_CAPABILITY_NOT_METERED);
                roaming |= !underlyingCaps.hasCapability(NET_CAPABILITY_NOT_ROAMING);
                congested |= !underlyingCaps.hasCapability(NET_CAPABILITY_NOT_CONGESTED);
            }
        }

        caps.setTransportTypes(transportTypes);
        caps.setLinkDownstreamBandwidthKbps(downKbps);
        caps.setLinkUpstreamBandwidthKbps(upKbps);
        caps.setCapability(NET_CAPABILITY_NOT_METERED, !metered);
        caps.setCapability(NET_CAPABILITY_NOT_ROAMING, !roaming);
        caps.setCapability(NET_CAPABILITY_NOT_CONGESTED, !congested);

这么一来underlyingNetworks一旦是null和空数组，metered 一定会变成 true，然后便没有NET_CAPABILITY_NOT_METERED --> BOOM

https://github.com/aosp-mirror/platform_frameworks_base/blob/master/services/core/java/com/android/server/ConnectivityService.java 的 public NetworkCapabilities[] getDefaultNetworkCapabilitiesForUser(int userId)

也有一段类似意思的注释，大致上说VPN Service下面有什么网络，就把它们全部算上

iKirby commented 6 years ago

昨天试过在 VpnService.Builder 那里调用 setUnderlyingNetworks(null) 无效（原版 SS）。另外目前官方的模拟器在运行 P 的时候无法打开 Wi-Fi ，所以没办法用模拟器测试。

Akkariiin commented 6 years ago

@kcschan 这样呀那么这么说就是需要传当前VPN应用下层的网络类型了么？那么如何获取当前的真实的下层网络类型而不是VPN自己广播出去的类型呢？

PS: 下载管理器的行为应该和play store是一样的，因为play store其实也是调用下载管理器来下载的 PS2: 现在暂时没法开安卓虚拟机，因为这个项目现在没法在win上build，但是我又暂时没有安全的linux的虚拟机宿主机

Akkariiin commented 6 years ago

@iKirby @kcschan 我在想，现在的解决方法要么就是把wifi和蜂窝两个一起放进去，但是这样的话按照那个代码就会限速在蜂窝网络下；要么就默认传wifi；要么就给个配置按钮让使用者自己选择。

希望有官方的标准解决方法范例，或者我上面说的可以获取当前的真实的下层网络类型，那么监听一下就OK了。但是要怎么做呢？安卓我还是新手欸~~

iKirby commented 6 years ago

@Akkariiin 刚才试了获取当前活动的网络（确定是 Wi-Fi ，用 [toString()](https://developer.android.com/reference/android/net/Network.html#toString()) 方法看的）并传进去但是也无效，不知道是不是方法不对。对于 Android 我也是新手。官方文档说的又不清楚，搜索过也没找到其他例子，不知道到底应该在什么时候调用这个方法。

Akkariiin commented 6 years ago

@iKirby 摊手那么如果一直传入wifi，其他应用（比如play）会不会在流量下也识别为wifi呢如果是的话，在官方修正或提供标准做法之前，就按照上面我说的第三个方法（给个配置按钮让使用者自己选择）来做好了。

iKirby commented 6 years ago

@Akkariiin 关键是尝试了传入 Wi-Fi 没有用啊... 如果能选择那也不错。

kcschan commented 6 years ago

@Akkariiin

ss那边看上去已经找到问题了，看样子@iKirby 提到的办法在P应当有效，但在8.1无效

https://github.com/shadowsocks/shadowsocks-android/commit/3a8f863bd1f0deed081de082a674020368f30319

他们提到一个workaround:

An interesting observation is that the developer can call setUnderlyingNetworks(arrayOfNulls(1)) to make the VPN connection always unmetered

@iKirby 试试看这个办法?

iKirby commented 6 years ago

@kcschan 我是用 P 测试的，我的修改确实无效。现在正在重新编译 beta 分支的 SS 测试。

kcschan commented 6 years ago

@iKirby 在P里面用这个呢？ setUnderlyingNetworks(arrayOfNulls(1))

iKirby commented 6 years ago

@kcschan 刚才试了一下，没用。我编译 beta 分支的 apk 无法安装... master 分支没问题，但是加上了这个也没有用

Akkariiin commented 6 years ago

额........... setUnderlyingNetworks(arrayOfNulls(1)) 其实就是上面所说的 * an empty array: a zero-element array, meaning that the VPN has no underlying network connection, and thus, app traffic will not be sent or received.

iKirby commented 6 years ago

@kcschan @Akkariiin 测试过了 SS master 分支 + https://github.com/shadowsocks/shadowsocks-android/commit/3a8f863bd1f0deed081de082a674020368f30319 的代码（因为 beta 分支的我编译出来装不上）。在 Android P 上，Google Photos 和 Drive 可以正确检测到 WiFi 网络并且自动开始上传任务，但是 Play Store 仍然无法下载应用，Telegram X 还是会识别为移动网络。