Dart sockets don't respect iOS's VPN

[gaaclarke]

Originally filed for Flutter: https://github.com/flutter/flutter/issues/41500

The issue reports that using Dart to access resources over VPN doesn't work.

I looked through Dart's source code and it appears to be using posix sockets. Apple's documentation recommends against that: In iOS, POSIX networking is discouraged because it does not activate the cellular radio or on-demand VPN. Thus, as a general rule, you should separate the networking code from any common data processing functionality and rewrite the networking code using higher-level APIs.

source: https://developer.apple.com/library/archive/documentation/NetworkingInternet/Conceptual/NetworkingTopics/Articles/UsingSocketsandSocketStreams.html

We should implement a socket implementation based on CFSocket.

[gaaclarke]

cc @ZichangG

[zichangg]

I think this should be added. But I'm not sure whether VM is able to distinguish ios from mac. We only have a mac implementation so far. @a-siva @rmacnak-google

[a-siva]

We do have HOST_OS_IOS defined when TARGET_OS_IPHONE is defined, see runtime/platform/globals.h

[rmacnak-google]

We do distinguish between iOS and Mac, it's just that the implementations are so similar we put them in the same files (unlike how we have separated files for Android and Linux) that have a small number of ifdefs.

[gaaclarke]

@ZichangG The CoreNetworking framework exists for iOS and macosx. If you change iOS and macosx to use it, it will be easier on you since you'll be able to test it locally instead of having to test on the simulator / device all the time.

[dnfield]

CFSocket should be available on both iOS and macOS.

An alternative to this is https://github.com/dart-lang/sdk/issues/39104, but this is likely easier to implement and should not require any breakages in the API surface.

[zichangg]

Right. That should be easier for testing.

If we move to CFSocket, it will be another set of implementation which won't use our kqueue-based eventhandler. It is basically rewriting whole socket with higher level CFSocket and CFSocketCreateRunLoopSource.

[zichangg]

Looked at some docs and code examples. CFSocket can be embedded into our eventhandler system. Unfortunately, this link explicitly says

In iOS, using sockets directly using POSIX functions or CFSocket does not automatically activate the device’s cellular modem or on-demand VPN.

What @dnfield proposed might be a good choice.

Since the only problem is activation of modem and VPN. Is it possible to turn them on programmatically for this case?

[gaaclarke]

@ZichangG @dnfield That's a bummer. I think we are stuck with the higher level API's if CFSocket doesn't work.

39104 sounds good and necessary but the one problem is that it doesn't solve the issue for every client of Dart. This is going to be a problem for everyone that uses Dart on iOS, it would be nice to solve this problem as the default implementation on iOS.

[dnfield]

The idea with 39104 is to keep the API compatible with dart:io, so that you can just switch your import from dart:io to dart:net. I've been planning to write a doc for this, but have been delayed.

[zichangg]

We do have some discussions for splitting dart:io into multiple smaller packages. I haven't started but this is the plan for this quarter.

[tobiaszuercher]

what is the current state of this issue? i face app vpn from mobile iron at every enterprise customer. it is a show stopper in many cases if the app vpn is used to secure a connection.

[thaoula]

Hi @tobiaszuercher,

As a work around you can create a custom http client and use http overriders to return your custom client when a HttpClient is requested.

Inside the custom client you can get the device proxy so that it is dynamic.

Please see the examples below of our Proxy aware http client that utilises the device_proxy package from pub dev

• It uses the system proxy so no hardcoded string
• It updates proxy information between calls (so if you turn on or off proxy during application run)
• Its just a wrapper that implements the same interface as http client and updates the proxy before certain calls
• It seamlessly works with Charles or any other proxy that allows you to view network requests.

import 'dart:io';

import 'package:device_proxy/device_proxy.dart';

class ProxyHttpClient implements HttpClient {
  HttpClient _client;
  String _proxy;

  ProxyHttpClient({SecurityContext context, HttpClient client}) {
    _client = client != null ? client : new HttpClient(context: context);
    _client.findProxy = (uri) {
      return _proxy;
    };
    _client.badCertificateCallback = ((
      X509Certificate cert,
      String host,
      int port,
    ) =>
        // TODO Disable in release mode
        true);
  }

  Future<void> updateProxy() async {
    ProxyConfig proxyConfig = await DeviceProxy.proxyConfig;
    _proxy = proxyConfig.isEnable ? 'PROXY ${proxyConfig.proxyUrl};' : 'DIRECT';
  }

  @override
  bool get autoUncompress => _client.autoUncompress;

  @override
  set autoUncompress(bool value) => _client.autoUncompress = value;

  @override
  Duration get connectionTimeout => _client.connectionTimeout;

  @override
  set connectionTimeout(Duration value) => _client.connectionTimeout = value;

  @override
  Duration get idleTimeout => _client.idleTimeout;

  @override
  set idleTimeout(Duration value) => _client.idleTimeout = value;

  @override
  int get maxConnectionsPerHost => _client.maxConnectionsPerHost;

  @override
  set maxConnectionsPerHost(int value) => _client.maxConnectionsPerHost = value;

  @override
  String get userAgent => _client.userAgent;

  @override
  set userAgent(String value) => _client.userAgent = value;

  @override
  void addCredentials(
      Uri url, String realm, HttpClientCredentials credentials) {
    return _client.addCredentials(url, realm, credentials);
  }

  @override
  void addProxyCredentials(
      String host, int port, String realm, HttpClientCredentials credentials) {
    return _client.addProxyCredentials(host, port, realm, credentials);
  }

  @override
  set authenticate(
          Future<bool> Function(Uri url, String scheme, String realm) f) =>
      _client.authenticate = f;

  @override
  set authenticateProxy(
          Future<bool> Function(
                  String host, int port, String scheme, String realm)
              f) =>
      _client.authenticateProxy = f;

  @override
  set badCertificateCallback(
      bool Function(X509Certificate cert, String host, int port) callback) {
    // _client.badCertificateCallback = callback;
  }

  @override
  void close({bool force = false}) => _client.close(force: force);

  @override
  Future<HttpClientRequest> delete(String host, int port, String path) async {
    await updateProxy();
    return _client.delete(host, port, path);
  }

  @override
  Future<HttpClientRequest> deleteUrl(Uri url) async {
    await updateProxy();
    return _client.deleteUrl(url);
  }

  @override
  set findProxy(String Function(Uri url) f) {
    _client.findProxy = f;
  }

  @override
  Future<HttpClientRequest> get(String host, int port, String path) async {
    await updateProxy();
    return _client.get(host, port, path);
  }

  @override
  Future<HttpClientRequest> getUrl(Uri url) async {
    await updateProxy();
    return _client.getUrl(url.replace(path: url.path));
  }

  @override
  Future<HttpClientRequest> head(String host, int port, String path) async {
    await updateProxy();
    return _client.head(host, port, path);
  }

  @override
  Future<HttpClientRequest> headUrl(Uri url) async {
    await updateProxy();
    return _client.headUrl(url);
  }

  @override
  Future<HttpClientRequest> open(
    String method,
    String host,
    int port,
    String path,
  ) async {
    await updateProxy();
    return _client.open(method, host, port, path);
  }

  @override
  Future<HttpClientRequest> openUrl(String method, Uri url) async {
    await updateProxy();
    return _client.openUrl(method, url);
  }

  @override
  Future<HttpClientRequest> patch(String host, int port, String path) async {
    await updateProxy();
    return _client.patch(host, port, path);
  }

  @override
  Future<HttpClientRequest> patchUrl(Uri url) async {
    await updateProxy();
    return _client.patchUrl(url);
  }

  @override
  Future<HttpClientRequest> post(String host, int port, String path) async {
    await updateProxy();
    return _client.post(host, port, path);
  }

  @override
  Future<HttpClientRequest> postUrl(Uri url) async {
    await updateProxy();
    return _client.postUrl(url);
  }

  @override
  Future<HttpClientRequest> put(String host, int port, String path) async {
    await updateProxy();
    return _client.put(host, port, path);
  }

  @override
  Future<HttpClientRequest> putUrl(Uri url) async {
    await updateProxy();
    return _client.putUrl(url);
  }
}

A custom HttpOverrides that returns your new proxy aware client

import 'dart:io';

import 'package:modules/core/shelf.dart';

class ProxyHttpOverrides extends HttpOverrides {
  @override
  HttpClient createHttpClient(SecurityContext context) {
    return ProxyHttpClient(
      context: context,
      client: super.createHttpClient(context),
    );
  }
}

Some docs regarding httpoverrides https://api.flutter.dev/flutter/dart-io/HttpOverrides-class.html

Regards, Tarek

[tobiaszuercher]

@thaoula thank you a lot for your code! i still don't have access to an app-vpn, but i'll try as soon as the environment is there!

[GoncaloPT]

With per-apn vpn the suggested solution doesn't work, since it might not be "just" a proxy.

I've tested with VMware Airwatch, using flutter and the host cannot be reached. Bypassing the dart HTTP ( by the usage of a custom plugin that routes requests to native ios code ), the per-app VPN captures and forwards those requests/responses successfully.

[vsutedjo]

Any news on this issue? Will there be a built-in solution in the near future?

[JordiGiros]

Hello @gaaclarke Is there any timing for this issue? Thanks!

[mraleph]

@JordiGiros no

[albatrosify]

Did anyone try this with F5 BIG IP VPN? I dont have access to a working environment just yet.

[brianquinlan]

cupertino_http is a new experimental Flutter plugin that provides access to Apple's Foundation URL Loading System - which honors iOS VPN settings.

cupertino_http has the same interface as package:http Client so it is easy to use in a cross-platform way. For example:

late Client client;
if (Platform.isIOS) {
  final config = URLSessionConfiguration.ephemeralSessionConfiguration()
    # Do whatever configuration you want.
    ..allowsCellularAccess = false
    ..allowsConstrainedNetworkAccess = false
    ..allowsExpensiveNetworkAccess = false;
  client = CupertinoClient.fromSessionConfiguration(config);
} else {
  client = IOClient(); // Uses an HTTP client based on dart:io
}

final response = await client.get(Uri.https(
    'www.googleapis.com',
    '/books/v1/volumes',
    {'q': 'HTTP', 'maxResults': '40', 'printType': 'books'}));

I would really appreciate it if you can try cupertino_http out and see if it solves the VPN issues for you.

Comments or bugs in the cupertino_http issue tracker would be appreciated!

[komaxx]

Are there any news? There's quite some impact for the company I'm working with as most of their customers have some sort of VPN setup going..

[a-siva]

@komaxx have you tried https://pub.dev/packages/cupertino_http ?

[komaxx]

@a-siva Good point, I'll give it a try! However, it's still marked "experimental", right? I'm a bit reluctant to include not-yet-stable code in this app since there are business app stores and certifications in play - updating the app is not a quick process :/

[a-siva]

@komaxx it is currently marked as "experimental" as this package is fairly new and we are soliciting feedback from users, our plan is to address all the initial feedback we receive and move it out of the "experimental" state.

[GoncaloPT]

Bump.

[a-siva]

@GoncaloPT have you tried https://pub.dev/packages/cupertino_http for your VPN problem.

[GoncaloPT]

Hello @a-siva. Would it support websocket connections? Last time i checked the package was very rudimentary and lacked support for websocket connections. Also, as probably all of us that post in this thread, I use flutter in a enterprise context; so using experimental packages is not something i'm eager to jump into :)

[brianquinlan]

An interesting note about using VPN with BSD sockets: https://developer.apple.com/forums/thread/76448

[brianquinlan]

I filed a bug to track adding websocket support in package:cupertino_http.