Parallelized builds stopped working in 3.4.1

hybridcattt commented 6 years ago

[x] I've read, understood, and done my best to follow the *CONTRIBUTING guidelines.

What did you do?

Updated AlamofireImage to 3.4.1, where Alamofire is not a nested project anymore.

I'm running a project where AlamofireImage is added manually. AlamofireImage and Alamofire are both added as target dependencies to the app target. "Parallelize build" setting is on, "Find implicit dependencies" is off.

What did you expect to happen?

Project can successfully build in such environment (as it did before 3.4.1)

What happened instead?

Project occasionally (most of the time) fails with "No such module Alamofire": in the build log it can be seen that AlamofireImage starts to build in parallel with Alamofire, and AlamofireImage fails to build because Alamofire is not yet available. The reason for such failure is that there is no dependency between AlamofireImage and Alamofire, so Xcode can't detect that one should wait for the other to finish building.

My project contains other target dependencies, so turning off parallelization is not a viable option.

Alamofire Environment

Alamofire version: 4.7.3 Xcode version: 9.4.1, 10.0 Swift version: 4.1, 4.2 Platform(s) running AlamofireImage: iOS macOS version running Xcode: Mojave, High Sierra

Demo Project

The issue is easy to reproduce, by disabling "Find implicit dependencies" in iOS Example project. I was not able to alter project settings to make it work with parallelized builds.

Additional notes:

This is a regression after Xcode project refactor PR (#329). Since there is no explicit dependency between AlamofireImage and Alamofire, it's not possible to enable build parallelization without enabling Find implicit dependencies. A WWDC video about dependencies in the build process, first 13 min: https://developer.apple.com/videos/play/wwdc2018/415 A WWDC video about build parallelization: https://developer.apple.com/videos/play/wwdc2018/408 Apple encourages to provide as much as possible dependency information, so the build system can use that to optimise the build sequence. It's not possible to alter the build sequence manually with parallel builds.

jshier commented 6 years ago

Why isn't enabling "Find implicit dependencies" an option?

Also, how are you integrated AlamofireImage?

@cnoon What do you think here?

hybridcattt commented 6 years ago

Hi @jshier , thanks for a reply.

how are you integrated AlamofireImage?

The setup is quite basic, pretty much the same as in your iOSExample project. Both AlamofireImage and Alamofire are added as nested projects, and corresponding targets are added as target dependencies to the project that uses AlamofireImage.

In our production use-case, we have an image library project that uses AlamofireImage. In turn, there are two apps (projects) that use that image library.

Why isn't enabling "Find implicit dependencies" an option?

There are a few reasons, let me try to explain.

Finding implicit dependencies setting lets Xcode use some empirical strategies to determine relationships and dependencies between targets. Having it enabled in a complex enough project can lead to various issues (one being if there are a few targets available that produce products with same name).

There's one more downside to using Find implicit dependencies - it works on scheme level (not target level), so it has to be enabled by every scheme that (even indirectly) uses AlamofireImage. So enabling Find implicit dependencies in our image lib project is not enough - it has to be enabled in every project that uses this it too.

We have quite complicated project where we have many libraries (some of which depend on others) - an image library is just one of. So having Find implicit dependencies disabled gives us the following benefits:

fine-grained control of what gets built when, as well as clear overview what depends on what (as opposed to dependencies being inferred from source code imports)
not depending on fiddly behaviours that break every other Xcode version (there are multiple reports of this feature causing various issues in complex projects from Xcode 4 through Xcode 9)

As a workaround for now we had to enable Find implicit dependencies, but would prefer to keep it off. Now it's enabled only to make AlamofireImage build.

In general Find implicit dependencies is probably okay to keep on for most developers, but I hope that for more advanced use cases AlamofireImage won't force us to use that setting.

cnoon commented 4 years ago

Hi @hybridcattt, thank you for taking the time to lay out the issue in full detail. Much appreciated!

With that said, there's not much we can do here. You've reached a point of complexity in your project where a dependency manager is essentially a must. Without it, you struggle with these types of problems across all projects. Any of your dependencies could make the change that we've made (for good reason I might add) which will bring your entire project down.

Instead, I'd encourage you to stop fight that particular issue, and embrace a dependency management solution like CocoaPods, Carthage, or SPM (if possible). They will eliminate this problem altogether, and will also relieve you of the problem of having multiple dependency naming collisions.

While I really don't like suggesting this as your only practical solution, I really think it is. There's no way you can practically expect every open-source project to change their projects so that it perfectly matches with your strategy. We swapped the direct dependency for workspace linking to allow AFI users to use newer versions of AF than AFI was using. That's a change that needed to happen.

If Find implicit dependencies has other impacts on your project that don't allow you to be successful, then I'd say the solution is not to put AFI back the way it was, but consider a different dependency management strategy for your project that is more resilient to change.

Alamofire / AlamofireImage