Significant delay before uploading large files with multipart

[alexstyl]

• [x] I've read, understood, and done my best to follow the *CONTRIBUTING guidelines.

What did you do?

Upload big files within a multipart form request. Here is the code I am using:

AF.upload(multipartFormData: { multipartFormData in
                [ array of strings with paths to files shared via a share extension].enumerated().forEach { (index, filePath) in
                    let fileURL = URL(fileURLWithPath: filePath)
                    multipartFormData.append(fileURL, withName: "file_"+index.description)
                }
            }, to: url, method: .post, headers: headers)
            .uploadProgress { progress in
                    // TODO update UI progress
             }
            .response { response in
                    print("done")
            }

What did you expect to happen?

The upload to start instantly.

What happened instead?

There is a significant delay before the calling of AF.upload() and a 'tick' to the upload progress. I suspect this is due to saving the files to disk before uploading (after reading the encodingMemoryThreshold documentation).

I tried uploading a ~600mb file to my server and it takes 3 seconds for the upload to start.

PS: I find it weird that the files needs to be saved to disk before uploading, as the files i am trying to upload are already stored on disk (uploading files from file extension). is there a way to upload the files directly from disk that I am not aware of?

Alamofire Environment

Alamofire Version: 5.9.1 Dependency Manager: Xcode Version: 15.3 Swift Version: Platform(s) Running Alamofire: iPhone 15 Simulator, iPhone 12 Pro Max (real device) macOS Version Running Xcode: MBP M3 Max

[jshier]

Alamofire's multipart form handling has a long history. Specifically, it was designed when the library kinda sorta supported background URLSessions and so needed to be safe to use from the foreground or background. This lead to a design where either everything was encoded into memory or onto disk, from which the upload was actually started. Both have their own drawbacks, as you've seen.

The true answer is to allow for streaming uploads while either guaranteeing the upload can't happen in the background, or creating some sort of mechanism to allow it to smoothly transition to the background while in progress. This is a huge unsolved area in Alamofire 5 due to the time investment required to fully understand the limits of background URLSessions and building the API in Alamofire to properly support them. And of course, building a wrapper around the proper Stream types to allow streaming various types of data to a single upload. (While we're at it, it may be useful to recreate AFNetworking's old streaming API's that allowed throttling the upload over time.) This is not a nice API. But it may be a good idea, long term, for Alamofire to support more streaming, even for URLRequest-based requests.

I'm going to mark this as a feature request in case someone wants to investigate, but I don't know when I'll have the time to work on it myself.

[alexstyl]

I am assuming that by 'background' you mean iOS background restrictions. I am new to iOS so I don't know much about that space.

However, I see on the Github repo that Alamofire supports a bunch of different OSes and platforms. Because of this it doesn't make sense to me to prioritize this background/foreground feature from a scoping perspective.

I believe it would make more sense to break this into smaller pieces of work such as:

1. A thin layer on top of native apis to support multiform requests. Let developers handle the iOS bg limitations on their own (which I am assuming they already are if they are using Apple's APIs)
2. iOS specific iteration with foreground/background handling which is opt-in (which uses the API from 1.)

Overall there seems like a huge gap in the iOS space for doing multipart form requests with streaming. I talked to a few iOS dev friend of mine with years of experience to see what they use for the job and they either implement their own or use Alamofire (with the limitation mentioned above).

[jshier]

Background handling isn't really optional for large uploads on iOS. No user will sit with your app open while a 600MB file uploads. They will immediately switch to doing something else, or close their phone entirely. Really you'd want to start the upload using a background-configured URLSession so the upload can continue regardless of app state. For that we need full background support in Alamofire. And that assumes a stream is even possible in the background.

And while Alamofire technically builds for the other platforms Swift supports, swift-corelibs-foundation's URLSession is so bad on those platforms I'd be surprised if anyone regularly uses Alamofire in the first place. iOS is still the vast majority of Swift and Alamofire usage. So while we could create a solution that could work on macOS and other desktop or server OSes, it wouldn't solve your problem, or the problems of most Alamofire users.