tuist fetch fails with `The artifact for binary target…cannot be found` when removing an SPM package with binary artifacts

jeffremer commented 1 year ago

What happened?

If you add an SPM package with binary artifacts to Tuist/Dependencies.swift then later remove it, subsequent calls to tuist fetch fail with an error along the lines of the following, even if the missing artifact is not referenced in Tuist/Dependencies.swift.

The error is along the lines of this, but the exact package will vary…

The artifact for binary target gRPC-Core of package gRPC cannot be found.

You must run tuist clean dependencies to allow tuist fetch to then run successfully.

While inspecting the SwiftPackageManager/.build directory I can see that when running tuist fetch after the package with binary artifacts is removed from Tuist/Dependencies.swift that the directories for those artifacts are in place but it appears that tuist fetch cleans out the xcframeworks underneath each target.

We currently have a somewhat naive wrapper around tuist so that developers and CI machines need only run the wrapper to both tuist fetch and tuist generate (as well as some unrelated things such as Cocoapods…).

I know we are calling tuist fetch too often and I have a plan to mitigate that, but regardless, this still feels like a bug - tuist fetch shouldn't fail simply because a dependency was removed. I didn't first encounter this by removing, adding, and then removing a dependency - instead I found this issue in CI where we .gitignore everything under Tuist/Dependencies except for the Lockfiles. This means that between CI runs the result of tuist fetch between builds is left in place on disk. I had added the Firebase SDK on a branch, my branch was run in CI, then later builds on those same machines encountered this error.

How do we reproduce it?

Use the attached sample project - TuistDependenciesError.zip
Run tuist fetch to fetch the existing FMDB package - see it succeed
Edit Tuist/Dependencies.swift
Uncomment the line to add the Firebase iOS SDK package
Run tuist fetch - see it succeed and now you have FMDB and Firebase
Edit Tuist/Dependencies.swift and comment out the Firebase package
Run tuist fetch

Expected: the final tuist fetch should succeed Actual: the final tuist fetch fails with message like the below

Screenshot 2023-06-28 at 5 34 25 PM

Error log

Attached is the output of the final tuist fetch --verbose call - fetch.log

Here is the terse version…

Resolving and fetching plugins.
Plugins resolved and fetched successfully.
Resolving and fetching dependencies.
Installing Swift Package Manager dependencies.
The artifact for binary target FirebaseAnalytics of package Firebase cannot be found.
Consider creating an issue using the following link: https://github.com/tuist/tuist/issues/new/choose

macOS version

13.4.1

Tuist version

3.20.0

Xcode version

14.3.1

jeffremer commented 1 year ago

I would be interested in taking a whack at this bug, but any hints as to where to get started would be appreciated.

danieleformichelli commented 1 year ago

Most of the logic for this is in:

SwiftPackageManagerInteractor generates the Package.swift from Dependencies.swift and invokes swift package resolve
SwiftPackageManagerGenerator parses the fetched informatio using swift package dump-package and passes them to PackageInfoMapper
PackageInfoMapper maps the SPM project to a Tuist one. It seems it is the one throwing the actual error, because for some reason still finds Firebase in the package dump

kapitoshka438 commented 11 months ago

Any progress on this? It makes us to tuist clean every time on CI to avoid problems and that means caching doesn't make any sense.

fortmarek commented 10 months ago

Any progress on this? It makes us to tuist clean every time on CI to avoid problems and that means caching doesn't make any sense.

Running tuist clean dependencies doesn't clean the cache artifacts, so you can still utilise caching. @jeffremer it could be interesting to compare this with the behavior of a raw SPM package by creating a dummy Package.swift, run swift build, remove the binary dependency, and run swift build. I'd assume the resolution might fail in this case, too – if so, then this might be a bug in SPM, not in tuist.

fortmarek commented 8 months ago

tuist install (formerly tuist fetch) has been greatly simplified, so this no longer should be an issue. Feel free to reopen if you're still running into this.

tuist / tuist