[Solved] iOS: `Network request failed` when sync to WebDAV. Other desktop clients work fine.

ettzzz commented 6 months ago

Operating system

iOS

Joplin version

ios 12.14.8

Desktop version info

No response

Current behaviour

Joplin won't sync on target webdav server on iOS client showing Network request failed, while other desktop versions work normally.

I double checked webdav user-password configs and I'm sure all information are correct.
I tried to uninstall and reinstall joplin on my iphone and the error remains.
My webdav service is a self-hosted nginx one and I use plain http and ip address for years as my sync target, it works well just until today.
just FYI, my mac client version is 2.13.12, my windows10 client version is 2.13.13, again, they are still functional under the same specs.

Expected behaviour

iOS client can sync normally again.

Logs

personalizedrefrigerator commented 6 months ago

My webdav service is a self-hosted nginx one and I use plain http and ip address for years as my sync target, it works well just until today.

By default, React Native doesn't seem to support plain HTTP on iOS >= 9.0. From the documentation,

By default, iOS 9.0 or later enforce App Transport Secruity (ATS). ATS requires any HTTP connection to use HTTPS. If you need to fetch from a cleartext URL (one that begins with http) you will first need to add an ATS exception. If you know ahead of time what domains you will need access to, it is more secure to add exceptions only for those domains; if the domains are not known until runtime you can disable ATS completely. Note however that from January 2017, Apple's App Store review will require reasonable justification for disabling ATS. See Apple's documentation for more information.

ettzzz commented 6 months ago

My webdav service is a self-hosted nginx one and I use plain http and ip address for years as my sync target, it works well just until today.

By default, React Native doesn't seem to support plain HTTP on iOS >= 9.0. From the documentation,

By default, iOS 9.0 or later enforce App Transport Secruity (ATS). ATS requires any HTTP connection to use HTTPS. If you need to fetch from a cleartext URL (one that begins with http) you will first need to add an ATS exception. If you know ahead of time what domains you will need access to, it is more secure to add exceptions only for those domains; if the domains are not known until runtime you can disable ATS completely. Note however that from January 2017, Apple's App Store review will require reasonable justification for disabling ATS. See Apple's documentation for more information.

Thanks for the info. I just checked my phone and it's iOS 17.4.1, the iOS 9.0 is quite far away from now. Also I started using joplin with self-hosted webdav from ~2020 on crossplatform clients and it works with plain http and ip address from the beginning to yesterday. I'm not sure if this bug is all about this ATS thing.

johne4013 commented 6 months ago

i also have the same problem when use ios client to sync, but windows version is well.

personalizedrefrigerator commented 6 months ago

Here are a few things I've found while looking into this:

In the latest one or two versions, changes were made due to updated iOS App Store submission requirements:
- The latest iOS version was built with a newer version of XCode.
  - This caused https://github.com/laurent22/joplin/issues/10412, which was fixed here.
- The latest iOS version also includes a privacy manifest.
Info.plist configuration
- In the Apple documentation for NSAllowsLocalNetworking states that configuration changes might be needed to connect to IP addresses in iOS 17:
  
  The NSAllowsLocalNetworking key controls whether App Transport Security (ATS) allows your app to connect to unqualified domains, .local domains, and IP addresses using IPv4 or IPv6.
  
  In iOS 9 and macOS 10.11, ATS disallows connections to all three domain types. You can add exceptions for unqualified domains and .local domains in the NSExceptionDomains dictionary, but you can’t add IP addresses. Instead you use NSAllowsArbitraryLoads when you want to load directly from an IP address.
  
  In iOS 10 through iOS 16, iPadOS 13.1 through iPadOS 16, and macOS 10.12 through macOS 13, ATS allows all three of these connections by default, so you no longer need an exception for any of them. However, if you need to maintain compatibility with older versions of the OS, set both of the NSAllowsArbitraryLoads and NSAllowsLocalNetworking keys to YES.
  
  In iOS 17, iPadOS 17, and macOS 14, ATS no longer allows connections to IP addresses by default. Add individual IP addresses and classless inter-domain routing (CIDR) ranges in the NSExceptionDomains dictionary. (Emphasis added)
- Joplin currently does include exceptions: https://github.com/laurent22/joplin/blob/ca8fd8d7ae4eb77d65ff62d9e87ac42ab67d362c/packages/app-mobile/ios/Joplin/Info.plist#L40-L49 This file doesn't seem to have been modified recently, however.

ettzzz commented 6 months ago

Here are a few things I've found while looking into this:

* In the latest one or two versions, changes were made due to updated iOS App Store submission requirements:

  * The latest iOS version was built with a newer version of XCode.

    * This caused [IOS Application does not open #10412](https://github.com/laurent22/joplin/issues/10412), [which was fixed here](https://github.com/laurent22/joplin/commit/966fe38ae3a30d70db6a330cdf29a53e3ad9fff1).
  * The latest iOS version also includes a [privacy manifest](https://developer.apple.com/documentation/bundleresources/privacy_manifest_files).

* `Info.plist` configuration

  * In [the Apple documentation for NSAllowsLocalNetworking](https://developer.apple.com/documentation/bundleresources/information_property_list/nsapptransportsecurity/nsallowslocalnetworking?language=objc) states that configuration changes might be needed to connect to IP addresses in iOS 17:
    > The NSAllowsLocalNetworking key controls whether App Transport Security (ATS) allows your app to connect to unqualified domains, .local domains, and IP addresses using IPv4 or IPv6.
    > In iOS 9 and macOS 10.11, ATS disallows connections to all three domain types. You can add exceptions for unqualified domains and .local domains in the [NSExceptionDomains](https://developer.apple.com/documentation/bundleresources/information_property_list/nsapptransportsecurity/nsexceptiondomains?language=objc) dictionary, but you can’t add IP addresses. Instead you use [NSAllowsArbitraryLoads](https://developer.apple.com/documentation/bundleresources/information_property_list/nsapptransportsecurity/nsallowsarbitraryloads?language=objc) when you want to load directly from an IP address.
    > In iOS 10 through iOS 16, iPadOS 13.1 through iPadOS 16, and macOS 10.12 through macOS 13, ATS allows all three of these connections by default, so you no longer need an exception for any of them. However, if you need to maintain compatibility with older versions of the OS, set both of the [NSAllowsArbitraryLoads](https://developer.apple.com/documentation/bundleresources/information_property_list/nsapptransportsecurity/nsallowsarbitraryloads?language=objc) and NSAllowsLocalNetworking keys to YES.
    > **In iOS 17, iPadOS 17, and macOS 14, ATS no longer allows connections to IP addresses by default. Add individual IP addresses and classless inter-domain routing (CIDR) ranges in the NSExceptionDomains dictionary.**
    > (Emphasis added)
  * Joplin currently does include exceptions:
    https://github.com/laurent22/joplin/blob/ca8fd8d7ae4eb77d65ff62d9e87ac42ab67d362c/packages/app-mobile/ios/Joplin/Info.plist#L40-L49

    This file doesn't seem to have been modified recently, however.

Thanks again for your digging in. It makes sense now why plain http and ip addresses work fine until recent versions. I will try to modify sync connections from http to https and see what comes next.

This issue then will be closed and I will paste something later if I find anything useful.

wljince007 commented 6 months ago

I found a alternatives way: #10437

ettzzz commented 6 months ago

Here are a few things I've found while looking into this:

* In the latest one or two versions, changes were made due to updated iOS App Store submission requirements:

  * The latest iOS version was built with a newer version of XCode.

    * This caused [IOS Application does not open #10412](https://github.com/laurent22/joplin/issues/10412), [which was fixed here](https://github.com/laurent22/joplin/commit/966fe38ae3a30d70db6a330cdf29a53e3ad9fff1).
  * The latest iOS version also includes a [privacy manifest](https://developer.apple.com/documentation/bundleresources/privacy_manifest_files).

* `Info.plist` configuration

  * In [the Apple documentation for NSAllowsLocalNetworking](https://developer.apple.com/documentation/bundleresources/information_property_list/nsapptransportsecurity/nsallowslocalnetworking?language=objc) states that configuration changes might be needed to connect to IP addresses in iOS 17:
    > The NSAllowsLocalNetworking key controls whether App Transport Security (ATS) allows your app to connect to unqualified domains, .local domains, and IP addresses using IPv4 or IPv6.
    > In iOS 9 and macOS 10.11, ATS disallows connections to all three domain types. You can add exceptions for unqualified domains and .local domains in the [NSExceptionDomains](https://developer.apple.com/documentation/bundleresources/information_property_list/nsapptransportsecurity/nsexceptiondomains?language=objc) dictionary, but you can’t add IP addresses. Instead you use [NSAllowsArbitraryLoads](https://developer.apple.com/documentation/bundleresources/information_property_list/nsapptransportsecurity/nsallowsarbitraryloads?language=objc) when you want to load directly from an IP address.
    > In iOS 10 through iOS 16, iPadOS 13.1 through iPadOS 16, and macOS 10.12 through macOS 13, ATS allows all three of these connections by default, so you no longer need an exception for any of them. However, if you need to maintain compatibility with older versions of the OS, set both of the [NSAllowsArbitraryLoads](https://developer.apple.com/documentation/bundleresources/information_property_list/nsapptransportsecurity/nsallowsarbitraryloads?language=objc) and NSAllowsLocalNetworking keys to YES.
    > **In iOS 17, iPadOS 17, and macOS 14, ATS no longer allows connections to IP addresses by default. Add individual IP addresses and classless inter-domain routing (CIDR) ranges in the NSExceptionDomains dictionary.**
    > (Emphasis added)
  * Joplin currently does include exceptions:
    https://github.com/laurent22/joplin/blob/ca8fd8d7ae4eb77d65ff62d9e87ac42ab67d362c/packages/app-mobile/ios/Joplin/Info.plist#L40-L49

    This file doesn't seem to have been modified recently, however.

Thanks again for your digging in. It makes sense now why plain http and ip addresses work fine until recent versions. I will try to modify sync connections from http to https and see what comes next.

This issue then will be closed and I will paste something later if I find anything useful.

Update after some tweaks:

After switching to https+domain from http+plain_ip, iOS client does sync normally again. However this will cost you a domain and ssl certificate and some time to configure. For myself, iOS client is a terminal for reading notes instead of editing, people should make their own tradeoffs if https tweak is worthy

ettzzz commented 6 months ago

I found a alternatives way: #10437

Thanks for sharing. I thinks this is a valid workaround for now. Anyone who is interested can have a try on it.

laurent22 / joplin