Not all outputs were created or valid when building for IOS device

[mattdornfeld]

I'm trying to use rules_apple + rules_spm to build and deploy a Swift ios app to an IPhone. I'm able to build it for my laptop and run it in the IPhone simulator using this command

bazel build :build_preview_app --apple_platform_type=ios

However when I try to build it for an IPhone device by specifying the CPU architecture

bazel build :build_preview_app --apple_platform_type=ios --ios_multi_cpus=arm64

I get a bunch of errors that look like this (I'm just posting a random sampling of lines). There are probably hundreds of error lines.

ERROR: /private/var/tmp/_bazel_johnsmith/bc251fba59d5612c8a8ba9752f7cffc6/external/swift_pkgs/BUILD.bazel:11155:12: output 'external/swift_pkgs/spm_build/arm64-apple-macosx/release/CNIOBoringSSL.build/ssl/ssl_key_share.cc.o' was not created
ERROR: /private/var/tmp/_bazel_johnsmith/bc251fba59d5612c8a8ba9752f7cffc6/external/swift_pkgs/BUILD.bazel:11155:12: output 'external/swift_pkgs/spm_build/arm64-apple-macosx/release/CNIOBoringSSL.build/ssl/ssl_lib.cc.o' was not created
ERROR: /private/var/tmp/_bazel_johnsmith/bc251fba59d5612c8a8ba9752f7cffc6/external/swift_pkgs/BUILD.bazel:11155:12: output 'external/swift_pkgs/spm_build/arm64-apple-macosx/release/NIOTransportServices.build/NIOTSNetworkEvents.swift.o' was not created
ERROR: /private/var/tmp/_bazel_johnsmith/bc251fba59d5612c8a8ba9752f7cffc6/external/swift_pkgs/BUILD.bazel:11155:12: output 'external/swift_pkgs/spm_build/arm64-apple-macosx/release/NIOTransportServices.build/SocketAddress+NWEndpoint.swift.o' was not created
ERROR: /private/var/tmp/_bazel_johnsmith/bc251fba59d5612c8a8ba9752f7cffc6/external/swift_pkgs/BUILD.bazel:11155:12: output 'external/swift_pkgs/spm_build/arm64-apple-macosx/release/NIOTransportServices.build/StateManagedChannel.swift.o' was not created
ERROR: /private/var/tmp/_bazel_johnsmith/bc251fba59d5612c8a8ba9752f7cffc6/external/swift_pkgs/BUILD.bazel:11155:12: output 'external/swift_pkgs/spm_build/arm64-apple-macosx/release/NIOTransportServices.build/TCPOptions+SocketChannelOption.swift.o' was not created
ERROR: /private/var/tmp/_bazel_johnsmith/bc251fba59d5612c8a8ba9752f7cffc6/external/swift_pkgs/BUILD.bazel:11155:12: output 'external/swift_pkgs/spm_build/arm64-apple-macosx/release/XCTestDynamicOverlay.swiftdoc' was not created
ERROR: /private/var/tmp/_bazel_johnsmith/bc251fba59d5612c8a8ba9752f7cffc6/external/swift_pkgs/BUILD.bazel:11155:12: output 'external/swift_pkgs/spm_build/arm64-apple-macosx/release/XCTestDynamicOverlay.build/XCTFail.swift.o' was not created
ERROR: /private/var/tmp/_bazel_johnsmith/bc251fba59d5612c8a8ba9752f7cffc6/external/swift_pkgs/BUILD.bazel:11155:12: output 'external/swift_pkgs/spm_build/arm64-apple-macosx/release/XCTestDynamicOverlay.build/XCTIsTesting.swift.o' was not created
ERROR: /private/var/tmp/_bazel_johnsmith/bc251fba59d5612c8a8ba9752f7cffc6/external/swift_pkgs/BUILD.bazel:11155:12: Building Swift package (external/swift_pkgs) for arm64-apple-ios15.5 using SPM. failed: not all outputs were created or valid
Target //app_ios_app:build_preview_app failed to build
ERROR: /Users/johnsmith/projects/app/app_ios_app/BUILD:41:14 Linking app_ios_app/libbuild_app_preview_source.a failed: not all outputs were created or valid

Any idea on what might be causing the issue? For reference here's a simplified version of my BUILD file

load("@build_bazel_rules_apple//apple:ios.bzl", "ios_application")
load("@build_bazel_rules_swift//swift:swift.bzl", "swift_library")

swift_library(
    name = "build_app_source",
    srcs = glob([
        "AppSource/**/*.swift",
    ]),
    deps = [
        "@swift_pkgs//Bow:Bow",
        "@swift_pkgs//Bow:BowEffects",
        "@swift_pkgs//Bow:BowOptics",
        "@swift_pkgs//Cache:Cache",
        "@swift_pkgs//swift-composable-architecture:ComposableArchitecture",
    ],
    module_name = "AppSource"
)

ios_application(
    name = "build_preview_app",
    bundle_name = "AppPreview",
    bundle_id = "co.name.AppPreview",
    families = [
        "iphone",
        "ipad",
    ],
    minimum_os_version = "15.0",
    infoplists = [":App/Info.plist"],
    visibility = ["//visibility:public"],
    deps = [
        ":build_app_source",
    ],
    provisioning_profile = "f3901410-71c1-4fe8-ab63-5c2563a07f49.mobileprovision"
)

Here's my WORKSPACE file

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

http_archive(
    name = "com_github_buildbuddy_io_rules_xcodeproj",
    sha256 = "564381b33261ba29e3c8f505de82fc398452700b605d785ce3e4b9dd6c73b623",
    url = "https://github.com/buildbuddy-io/rules_xcodeproj/releases/download/0.9.0/release.tar.gz",
)

http_archive(
    name = "cgrindel_rules_spm",
    sha256 = "03718eb865a100ba4449ebcbca6d97bf6ea78fa17346ce6d55532312e8bf9aa8",
    strip_prefix = "rules_spm-0.11.0",
    url = "https://github.com/cgrindel/rules_spm/archive/v0.11.0.tar.gz",
)

load(
    "@cgrindel_rules_spm//spm:defs.bzl",
    "spm_pkg",
    "spm_repositories",
    )

load(
    "@cgrindel_rules_spm//spm:deps.bzl",
    "spm_rules_dependencies",
)
spm_rules_dependencies()

load(
    "@com_github_buildbuddy_io_rules_xcodeproj//xcodeproj:repositories.bzl",
    "xcodeproj_rules_dependencies",
)
xcodeproj_rules_dependencies()

load(
    "@build_bazel_rules_apple//apple:repositories.bzl",
    "apple_rules_dependencies",
)
apple_rules_dependencies()

load(
    "@build_bazel_rules_swift//swift:repositories.bzl",
    "swift_rules_dependencies",
    )
swift_rules_dependencies()

load(
    "@build_bazel_rules_swift//swift:extras.bzl",
    "swift_rules_extra_dependencies",
)
swift_rules_extra_dependencies()

load(
    "@build_bazel_apple_support//lib:repositories.bzl",
    "apple_support_dependencies",
)
apple_support_dependencies()

spm_repositories(
    name = "swift_pkgs",
    platforms = [
        ".macOS(.v10_15)",
        ".iOS(.v15)"
    ],
    dependencies = [
        spm_pkg(
            url = "https://github.com/apple/swift-log.git",
            exact_version = "1.4.2",
            products = ["Logging"],
        ),
        spm_pkg(
            url = "https://github.com/pointfreeco/swift-composable-architecture.git",
            exact_version = "0.43.0",
            products = ["ComposableArchitecture"],
        ),
        spm_pkg(
            name = "Bow",
            url = "https://github.com/bow-swift/bow.git",
            exact_version = "0.8.0",
            products = ["Bow", "BowEffects", "BowOptics"],
        ),
        spm_pkg(
            url = "https://github.com/grpc/grpc-swift.git",
            exact_version = "1.7.3",
            products = ["GRPC"],
        ),
        spm_pkg(
            url = "https://github.com/hyperoslo/Cache",
            exact_version = "6.0.0",
            products = ["Cache"],
        ),
    ],
)

[cgrindel]

@mattdornfeld Unfortunately, I think you are running into some issues described here. In short, building and deploying to iOS devices is not working.

I am working on a Gazelle plugin that resolves dependencies and defines build files that use rules_swift rules to build. (Today, external packages are built using Swift package manager.) This will allow applications for any platform supported by rules_swift and rules_apple to work properly. I will post information about the plugin on this repo and in Slack. (I think that I am a couple of weeks away from an initial release.)

[jberkel]

Having had a lot of pain with Xcode recently, I'm exploring some alternatives. Looks like bazel (in combination with SPM) is not quite there yet. I am not really familiar with bazel, how will this work? So you're going to generate bazel build files from Package.swift and reproduce some of spm's functionality directly in bazel? Will you have to re-generate the bazel files every time you change your dependencies, or will this be automatic?

[cgrindel]

There will be two ways to use the Gazelle plugin. They differ in whether your project is going to be a Swift package that you are going to publish.

Common Setup

Configure your WORKSPACE file to load some prerequisites (e.g., rules_swift, rules_apple, bazel-gazelle, and cgrindel_swift_bazel (the repo with with my Gazelle plugin)). It will look like the following:

workspace(name = "pkg_manifest_example")

# NOTE: This will be an http_archive in your project.
local_repository(
    name = "cgrindel_swift_bazel",
    path = "../..",
)

load("@cgrindel_swift_bazel//:deps.bzl", "swift_bazel_dependencies")

swift_bazel_dependencies()

# MARK: - Gazelle

# gazelle:repo bazel_gazelle

load("@bazel_gazelle//:deps.bzl", "gazelle_dependencies")
load("@cgrindel_swift_bazel//:go_deps.bzl", "swift_bazel_go_dependencies")
load("@io_bazel_rules_go//go:deps.bzl", "go_register_toolchains", "go_rules_dependencies")

# Declare Go dependencies before calling go_rules_dependencies.
swift_bazel_go_dependencies()

go_rules_dependencies()

go_register_toolchains(version = "1.19.1")

gazelle_dependencies()

# MARK: - Swift Toolchain

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

http_archive(
    name = "build_bazel_rules_swift",
    sha256 = "51efdaf85e04e51174de76ef563f255451d5a5cd24c61ad902feeadafc7046d9",
    url = "https://github.com/bazelbuild/rules_swift/releases/download/1.2.0/rules_swift.1.2.0.tar.gz",
)

load(
    "@build_bazel_rules_swift//swift:repositories.bzl",
    "swift_rules_dependencies",
)
load("//:swift_deps.bzl", "swift_dependencies")

# gazelle:repository_macro swift_deps.bzl%swift_dependencies
swift_dependencies()

swift_rules_dependencies()

load(
    "@build_bazel_rules_swift//swift:extras.bzl",
    "swift_rules_extra_dependencies",
)

swift_rules_extra_dependencies()

Add a BUILD.bazel file to the root of your repository. You will add a few targets that will help you keep your Bazel files up-to-date:

load("@bazel_gazelle//:def.bzl", "gazelle", "gazelle_binary")

# MARK: - Gazelle

gazelle_binary(
    name = "gazelle_bin",
    languages = [
        "@bazel_skylib//gazelle/bzl",
        "@cgrindel_swift_bazel//gazelle",
    ],
)

# gazelle:prefix github.com/cgrindel/swift_bazel/examples/simple
gazelle(
    name = "gazelle",
    gazelle = ":gazelle_bin",
)

Option 1: Your Project is a Swift Package (i.e., will be used by other Swift projects)

Set up your Package.swift. This needs to have all of the configuration needed for your clients to build/use your package.

Add a target to the BUILD.bazel file at the root of your workspace that will generate repository rules to load your external dependencies from your Package.resolved file.

gazelle(
    name = "swift_update_repos",
    args = [
        "-from_file=Package.resolved",
        "-to_macro=swift_deps.bzl%swift_dependencies",
        "-prune",
    ],
    command = "update-repos",
    gazelle = ":gazelle_bin",
)

You will then execute the following to update your dependencies, generate build files for your project and build your code.

# Generate repository rules for your external dependencies.
$ bazel run //:swift_update_repos

# Generate build files for your project
$ bazel run //:gazelle

# Build your project
$ bazel build //...

Option 2: Your Project is not a Swift package (i.e., build applications for deployment)

In your WORKSPACE file, you will define your external dependencies like you do with rules_spm.

swift_repositories(
    name = "swift_pkgs",
    platforms = [
        ".macOS(.v10_15)",
        ".iOS(.v15)"
    ],
    dependencies = [
        swift_pkg(
            url = "https://github.com/apple/swift-log.git",
            exact_version = "1.4.2",
            products = ["Logging"],
        ),
        swift_pkg(
            url = "https://github.com/pointfreeco/swift-composable-architecture.git",
            exact_version = "0.43.0",
            products = ["ComposableArchitecture"],
        ),
        swift_pkg(
            name = "Bow",
            url = "https://github.com/bow-swift/bow.git",
            exact_version = "0.8.0",
            products = ["Bow", "BowEffects", "BowOptics"],
        ),
        swift_pkg(
            url = "https://github.com/grpc/grpc-swift.git",
            exact_version = "1.7.3",
            products = ["GRPC"],
        ),
        swift_pkg(
            url = "https://github.com/hyperoslo/Cache",
            exact_version = "6.0.0",
            products = ["Cache"],
        ),
    ],
)

Add a target to the BUILD.bazel file at the root of your workspace that will generate repository rules to load your external dependencies.

# NOTE: I am still working on the design for generating the Package.resolved from the WORKSPACE declarations.
gazelle(
    name = "swift_update_repos",
    args = [
        "-from_file=Package.resolved",
        "-to_macro=swift_deps.bzl%swift_dependencies",
        "-prune",
    ],
    command = "update-repos",
    gazelle = ":gazelle_bin",
)

You will then execute the following to update your dependencies, generate build files for your project and build your code.

# Generate repository rules for your external dependencies.
$ bazel run //:swift_update_repos

# Generate build files for your project
$ bazel run //:gazelle

# Build your project
$ bazel build //...

[cgrindel]

Also, if you want to continue using Xcode as an IDE, but build with Bazel, check out https://github.com/buildbuddy-io/rules_xcodeproj.

[mattdornfeld]

Thanks @cgrindel I'm probably going to handle dependencies manually using new_git_repository and swift_library for now. I'll check back in when rules_spm supports the features you described above. It was working pretty great until I ran into this issue!

[jberkel]

@cgrindel ok, thanks for the explanation. My use case is an application, so perhaps a bit simpler. It uses SPM to separate out related code into modules. The pain point with Xcode is that some parts of the projects seem to get build with SPM, and some of it is Xcode's internal build system, which is poorly documented, and debugging is a nightmare.

If instead the whole project could be built with just one system, I imagine it would make things easier, or at least easier to diagnose problems. I'd still like to keep SPM around, so that these modules can be build standalone, without bazel.

[cgrindel]

@jberkel Building everything with one system is the final goal of the Gazelle plugin. Bazel will build everything.