Setting up custom derived data

[adityadaniel]

I try to use Injection in our project which is quite large and build using Bazel. When rebuilding, I got this message in console

💉  ⚠️ Could not locate containing project or it's logs.
For a macOS app you need to turn off the App Sandbox.
Have you customised the DerivedData path?

How can I set up custom derived data path on Injection?

[johnno1962]

Hi, I'm interested in seeing if Bazel can be supported but only the standard and relative derived data paths are supported when using injection. Perhaps you could try using the https://github.com/johnno1962/HotReloading project which finds DerivedData wherever you put it.

[adityadaniel]

Let me take a look at https://github.com/johnno1962/HotReloading.

Also I'm curious, since in https://github.com/johnno1962/HotReloading we could use custom DerivedData, in theory, if we swap the implementation in Injection III, with HotReloading ones to find DerivedData folder, it would work, isn't it? If so, which file I should look into?

[johnno1962]

Start with HotReloading as it always finds the DerivedData folder using the package SwiftTrace and is easier to debug and iterate over than the app. I'm considering changing the error message you first encountered which almost reads like an invitation to use a custom derived data path which is difficult to support for the app.

[johnno1962]

There, I changed the message, 0107904c2a7b96f91f50da4f126d8a0b43888865, 4.3.7

[xiaohanyu]

Hello @johnno1962,

First thanks very much for this awesome project.

Recently we are trying out this project to boost our productivity and we met the same issue with bazel. After some digging we found that the root cause for why InjectionIII is not working with bazel is not really the path change of DerivedData but actually the logs format change.

Bazel do not follow the legacy Xcode *.xcactivitylog format (namely, log every compilation commands for each objective-c or swift file), but instead, bazel has its own log format, so when InjectionIII was trying to parse out the compilation commands from DerivedData with bazel content, it would fail.

One more thing, bazel by default do not log the compilation commands in its log.

So to fix this, we need to do two things:

1. Make bazel to output the compilation commands in its logs
2. Make InjectionIII to be able to recognize and parse the command from bazel logs.

For step 1, we could pass a --subcommands flag to bazel build, and then bazel would log the exact compiler commands in its log, it looks something like this:

# Configuration: 4a6a1902fd5e0fc309d088d733bbc2d662605957f039fcd8d04ebc66f89eb8db
# Execution platform: @local_config_platform//:host
SUBCOMMAND: # //XXX:XXX [action 'Compiling Swift module XXX', configuration: b0f34f8841808ad1e79b24f6eebcb1f31626982262b2f00142d0fb1f63c69e3e, execution platform: @local_config_platform//:host]
(cd /private/var/tmp/_bazel_hanyu/9bec08d6b37aa2133077091cdc6e152d/execroot/__main__ && \
  exec env - \
    APPLE_SDK_PLATFORM=iPhoneOS \
    APPLE_SDK_VERSION_OVERRIDE=15.4 \
    XCODE_VERSION_OVERRIDE=13.3.0.13E113 \
  bazel-out/darwin-opt-exec-2B5CBBC6-ST-8bd7c7db9774/bin/external/build_bazel_rules_swift/tools/worker/worker \
    swiftc \
    @bazel-out/ios-arm64-min12.0-applebin_ios-ios_arm64-dbg-ST-6b6216040413/bin/XXX/XXX.swiftmodule-0.params)
# Configuration: b0f34f8841808ad1e79b24f6eebcb1f31626982262b2f00142d0fb1f63c69e3e
# Execution platform: @local_config_platform//:host
SUBCOMMAND: # //YYY:ZZZ [action 'Compiling Swift module ZZZ', configuration: b0f34f8841808ad1e79b24f6eebcb1f31626982262b2f00142d0fb1f63c69e3e, execution platform: @local_config_platform//:host]
(cd /private/var/tmp/_bazel_hanyu/9bec08d6b37aa2133077091cdc6e152d/execroot/__main__ && \
  exec env - \
    APPLE_SDK_PLATFORM=iPhoneOS \
    APPLE_SDK_VERSION_OVERRIDE=15.4 \
    XCODE_VERSION_OVERRIDE=13.3.0.13E113 \
  bazel-out/darwin-opt-exec-2B5CBBC6-ST-8bd7c7db9774/bin/external/build_bazel_rules_swift/tools/worker/worker \
    swiftc \
    @bazel-out/ios-arm64-min12.0-applebin_ios-ios_arm64-dbg-ST-6b6216040413/bin/YYY/ZZZ.swiftmodule-0.params)

I would like to ask for your thoughts for this and it is OK for you, I may kick the contribution for InjectionIII in order for it to work with bazel.

Thank you!

cc @adityadaniel

[johnno1962]

Thanks @xiaohanyu! contributions are always welcome and support for Bazel is something a few people have asked about. It looks like you'd need to make some very delicate changes to an important piece of code (written in Perl) that greps out the logs in SwiftEval.swift. If you want to work on InjectionIII, I recommend switching to the HotReloading version instead which is the same source in a package you add to your project and can iterate over quickly by cloning the project and dragging it into the client project so it takes the place of the configured version. If you find a solution and raise a PR I can look at how dramatic the changes are and asses whether there is a risk of regression for other users of the project.

[xiaohanyu]

Hey @johnno1962,

Thanks for the fast reply. Yeah I've already read some code of HotReloading project and I know the combination of shell script and perl thing (and to be honest, also experienced some pain for this, considering whether I can make the contribution to avoid generated shell/perl script and make it pure swift, off topic though, LOL).

Would check bazel to see whether bazel has a native method to return the compilation commands. It has bazel query and bazel aquery command but I'm not very familiar with these.

• https://docs.bazel.build/versions/main/query-how-to.html
• https://docs.bazel.build/versions/main/aquery.html

Will surely keep you updated once I have any new discoveries.

[johnno1962]

Good Luck! I'm afraid I wouldn't accept a PR that moved away from the Perl code (Sorry!) to Swift as it would likely be much slower and there is tricky code in there to support >256 files in a project that might be difficult to replicate. This is a part of the code that seems to be performing well and I'd be very reluctant to revisit it. Apart from that, see how you go. Perhaps I (or you) could release a separate branch of the project supporting Bazel till it is proven.

[johnno1962]

Hey @xiaohanyu, did you make any headway looking at Bazel support? Seems like the very largest projects in the community are using it so it would be nice to be able to offer a solution. Sorry I wasn't very receptive to your offer to rewrite the Perl log searcher but it is something that is currently working quite well. I had a look at Bazel and while I found the small Objective-C example project but I came up against various errors trying follow the QuickStart. Is there any chance you would have the time to prepare a small example Swift project I could have a look at and see if we can move this forward? I imagine the log search could be replaced completely by the Bazel command you mention to extract the recompilation command for a source if I could just get to a working starting point!

[johnno1962]

Hey @keith, you seem to have be pretty involved in using Bazel for large Swift iOS projects, can you think of a bazel query command that could be used to extract the arguments for a compile command for a particular source file or some other way it could be extracted?

[saragiotto]

Hi @johnno1962 , I think I could help you with this small project with bazel. There some examples at this repo https://github.com/bazelbuild/rules_apple/tree/master/examples/ios.

I'm looking forward to this integration. Let me know if this samples is enough to make some tests.

[johnno1962]

Thanks for stepping up @saragiotto, I've been working on a solution for bazel which already works and should be ready in a couple of days. One thing that did crop up which perhaps you could help me with? For injection to work properly the app needs to be linked with "Other Linker Flags" -Xlinker -interposable. Can you think of a way users could specify that for a Debug build only when using bazel?

[keith]

Hey @keith, you seem to have be pretty involved in using Bazel for large Swift iOS projects, can you think of a bazel query command that could be used to extract the arguments for a compile command for a particular source file or some other way it could be extracted?

I think the ideal solution for this would be to fix https://github.com/grailbio/bazel-compilation-database/issues/71 or adapt https://github.com/hedronvision/bazel-compile-commands-extractor to support generating compile commands from the build.

Without one of those solutions, you can pretty easily get the flags for swift targets by doing something like bazel aquery 'deps(some app)' --output=proto and parsing the protobuf output from that (maybe other formats would be easier, but that's the general idea). Using aquery takes into account given configurations so you can ensure that the flags for the targets match what you expect, and I think a production solution would allow users to pass custom flags like --config=debug or something that might correspond to some internal configuration users have done. This output requires a small amount of post processing for some specific things bazel abstracts to keep build command lines hermetic, like replacing the __BAZEL* placeholders with the applicable absolute paths on the user's machine.

One gotcha you might hit with this, and why I used some app in the example above, is that raw swift_library targets do not have knowledge about what platform they will be built for, which allows a single swift_library to be used across macOS, iOS, Linux, etc. Because of this assuming you actually want specific platform info, it's important to start the aquery with the correct top level targets, such as ios_application targets, which encode the platform info on all their transitive dependencies.

[keith]

Thanks for stepping up @saragiotto, I've been working on a solution for bazel which already works and should be ready in a couple of days. One thing that did crop up which perhaps you could help me with? For injection to work properly the app needs to be linked with "Other Linker Flags" -Xlinker -interposable. Can you think of a way users could specify that for a Debug build only when using bazel?

There are a few ways you can do this, in general I think it's common for users to have a custom debug configuration setup in their .bazelrc file which allows you to pass something like --config=debug to their bazel invocation, at which point any lines in their .bazelrc prefixed with build:debug will be applied. So in this example you would add build:debug --linkopt=-Wl,-interposable to have this apply only in debug mode. If this doesn't work for folks it's also possible to use a dbg specific select() statement that you applied to a specific top level target, like your ios_application target. To do this you'd add something like:

# some target...
linkopts = select({
    "//:dbg": ["-Wl,-interposable"],
    "//conditions:default": [],
}),

As an aside we have SwiftUI previews working with our bazel project which uses similar infra, so doing something like this is definitely feasible.

[johnno1962]

Thanks for the tip! I'm still at sea with bazel -- would it be possible to tell me how/where to apply your suggestion to this BUILD file?

load("@rules_cc//cc:defs.bzl", "objc_library")
load("@build_bazel_rules_apple//apple:ios.bzl", "ios_application")
load(
    "@com_github_buildbuddy_io_rules_xcodeproj//xcodeproj:defs.bzl",
    "top_level_target",
    "xcodeproj",
)

objc_library(
    name = "UrlGetClasses",
    srcs = [
        "UrlGet/AppDelegate.m",
        "UrlGet/UrlGetViewController.m",
        "UrlGet/main.m",
    ],
    hdrs = glob(["UrlGet/*.h"]),
)

load("@build_bazel_rules_swift//swift:swift.bzl", "swift_library")

swift_library(
    name = "Foo",
    srcs = glob(["UrlGet/*.swift"]),
    module_name = "Foo",
    visibility = ["//:__subpackages__"],
    deps = [
#        "@swift_pkgs//swift-nio:NIO",
    ],
)

#load("@cgrindel_rules_spm//spm:spm.bzl", "spm_pkg", "spm_repositories")
#load("@rules_cc//cc:defs.bzl", "cc_library")

#spm_repositories(
#    name = "swift_pkgs",
#    dependencies = [
#        spm_pkg(
#            "https://github.com/johnno1962/HotReloading.git",
#            from_version = "4.6.3",
#            products = ["HotReloading"],
#        ),
#    ],
#)

ios_application(
    name = "ios-app",
    bundle_id = "Google.UrlGet",
    families = [
        "iphone",
        "ipad",
    ],
    infoplists = [":UrlGet/UrlGet-Info.plist"],
    launch_storyboard = "UrlGet/UrlGetViewController.xib",
    minimum_os_version = "15.0",
    # provisioning_profile = "<your_profile_name>.mobileprovision", # Uncomment and set your own profile.
    visibility = ["//visibility:public"],
    deps = [":UrlGetClasses", ":Foo"],
)

xcodeproj(
    name = "xcodeproj",
    build_mode = "bazel",
    project_name = "ios-app",
    tags = ["manual"],
    top_level_targets = [
        top_level_target(":ios-app", target_environments = ["device", "simulator"]),
    ],
)

[keith]

In an isolated BUILD file like this, using the linkopts approach I posted above is probably easiest for testing. You can add that directly on the ios_application rule. The only change you need is to define a custom config_setting to match when you're doing a debug build. In this same BUILD file you can probably just add:

config_setting(
    name = "debug_build",
    values = {
        "compilation_mode": "dbg",
    },
)

Then in the select you can use :debug_build. And on your bazel command line build with --compilation_mode=dbg and that should match. To verify you can also pass --linkopt=-v on the command line to see the final ld64 invocation.

[johnno1962]

Thanks, I seem to have found a way adding --linkopt=-Wl,-interposable to the bazel build inside the .xcodeproj I'm using. For some reason, interposing isn't working though the linker flag is being used. Will have to dig a bit deeper tomorrow.

[johnno1962]

OK, so it was working after all. It was reporting it was failing because there was in fact nothing to interpose. More testing to come from here to see if there are any other rough edges and we'll see if I can post a new release candidate tomorrow some time.

[johnno1962]

I've made available a preliminary release that should go a long way towards supporting injection in a project that is using the Bazel build system: https://github.com/johnno1962/InjectionIII/releases/tag/4.5.0RC1

It assumes the source you are injecting is under a directory containing a WORKSPACE file and makes a small patch to https://github.com/bazelbuild/rules_swift to create a link from the parameters files passed by bazel to swiftc to compile a module to a file named after that module in /tmp i.s. /tmp/bazel_\<ModuleName>.resp. When you inject a file under the directory contains a WORKSPACE file it scans these files looking for how to recompile the modified source and calls that command, links the object file into a dylib and injects it in the way InjectionIII has worked up to now.

To use download the pre-prelease I mention and add the following to.run when your application starts:

 Bundle(path: "/Applications/InjectionIII.app/Contents/Resources/iOSInjection.bundle")!.load()

It is no longer necessary to run the app itself and this fallback to "standalone" mode will watch for source file changes anywhere in your home directory. In addition, for injection to work fully you need to arrange for the options --compilation_mode=dbg --linkopt="-Wl,-interpossable" to be passed to your bazel build, either on the command line or in the tulsi generated .xcodeproj you are using for development.

If anybody would like to try it out, let me know how you get on 👍

[adityadaniel]

Hi @johnno1962, thank you for taking time and looking into this. I try to take it for spin in our project but unfortunately couldn't make it work. Here are the steps that I do:

• Download Injection III app from release https://github.com/johnno1962/InjectionIII/releases/tag/4.5.0RC1
• Add --linkopt="-Wl,interposable" in our .bazelrc

We already compile in "dbg" mode in our mini app so I don't need to add it.

Here's what printed in console after changing the code:

💉 ⚠️ Locating response file failed (see: /tmp/command.sh)
ls: /tmp/bazel_*.resp: No such file or directory

I assume I need to patch our rules_swift to write into /tmp. Can you elaborate more on patching rules_swift?

[johnno1962]

HI @adityadaniel, thanks for taking it for a spin. You didn't see a message ⚠️ bazel patched, restart app? If you look at the file /tmp/command.sh that it mentions you can see what the new version is doing and if you try $ bash -x /tmp/command.sh, perhaps it will give you more info you can report back. The question is why didn't the patch succeeded? I'm using a slightly old bazel 5.3.0 from homebrew, so something might be different. You should have seen this the first time you save a file after loading the injection bundle:

💉 ⚠️ Locating response file failed (see: /tmp/command.sh)
Checking patch tools/worker/swift_runner.cc...
Checking patch tools/worker/swift_runner.h...
Applied patch tools/worker/swift_runner.cc cleanly.
Applied patch tools/worker/swift_runner.h cleanly.
⚠️ bazel patched, restart app

[adityadaniel]

Yes, I did see this message

💉 ⚠️ Locating response file failed (see: /tmp/command.sh)
Checking patch tools/worker/swift_runner.cc...
Checking patch tools/worker/swift_runner.h...
Applied patch tools/worker/swift_runner.cc cleanly.
Applied patch tools/worker/swift_runner.h cleanly.
⚠️ bazel patched, restart app

and after restarting the app, Injection seems connected to the app because I see this in console:

💉 InjectionIII connected /Users/daniel.istyana/Developer/gw-xcodeproj/tulsiprojects/tokopedia-tulsi.xcodeproj
💉 Watching files under the directory /Users/daniel.istyana/Developer/gw-xcodeproj

However, when I try to change a file and save it, nothing happens. Is this expected?

Here's the output of bash -x /tmp/command.sh:

+ cd /Users/daniel.istyana/Developer/gw-injection/bazel-out/../external/build_bazel_rules_swift
+ grep module_name_ tools/worker/swift_runner.h
+ cd /Users/daniel.istyana/Developer/gw-injection
++ ls -t '/tmp/bazel_*.resp'
+ exit 1

If I understand correctly, you patch bazel so it output /tmp/bazel_*.resp in order to Injection could know which files to compile and updated in the runtime, however I don't see any file with bazel_*.resp in my tmp folder. Could that be the problem?

[johnno1962]

Seems like the patching is working. Did you modify a Swift source? So far I've only been using a swift_library entry in the BUILD file. If swiftc has been called, it should have created the /tmp/*.resp link. Did you edit the source file i.e actually change its contents and save after the patching and before the restart? Perhaps this is needed to force the recompile which should create the link.

[johnno1962]

For the curious I've pushed the changes required to the HotReloading repo https://github.com/johnno1962/HotReloading/pull/63/files.

[johnno1962]

It seems in RC1, you could get into a state where the links in /tmp are not created and no amount of resaving and rebuilding would create them. I've uploaded a new release with a minor change that should make this less likely if someone wants to try it out. https://github.com/johnno1962/InjectionIII/releases/tag/4.5.0RC2 The new version will need to re-patch the bazel worker code to take effect. To force this type: rm -rf bazel-out/../external in your workspace directory.

[saragiotto]

Thank you @johnno1962 so much in getting deep in this matter. We are looking forward this support.

[keith]

I definitely think a solution using aquery or something else would be preferred to hacking the code in rules_swift, which is pretty fragile and also invalidates the bazel cache, since it changes the inputs to all swift compilation actions. Note that params files are written to disk in bazel-out/, not the absolutely final ones, but you could replicate the argument manipulation if needed

[johnno1962]

Thanks for the input. If had had more experience with bazel and was convinced people are hungry for a solution I might be able to take that on. Can you tell me about "invalidates the bazel cache, since it changes the inputs to all swift compilation actions"? All the patching does is make a link to the params files to /tmp at the very last stage inside the worker so they don't get deleted and are available at a known path. Doing it this late also has the advantage everything configuration related has already been expanded. It seemed the simplest thing to do.

[keith]

Yea so the issue I'm referring to has 2 downsides for this:

1. if users use bazel's remote cache to reduce unnecessary build work, these actions won't be executed locally, and therefore these files won't be produced at all, since bazel only downloads declared outputs by virtue of it requiring you to perform hermetic actions only
2. even if you're not using the remote cache, because you change the worker with this patch, bazel treats the entire build graph as a merkle tree where all inputs are used to determine whether or not an action needs to be re-run, or it can pull from a previous build / the remote cache, since this changes the worker, bazel will rebuild the worker, which will then be a different binary, affecting all downstream actions that rely on the worker, which is all swift compiles. theoretically this might only happen once because once you patch it once, subsequent builds will reuse the patched version, but that's the issue. also note that the files in bazel-out are meant to be immutable, so bazel might replace those with the real ones at any point.

also of course the patch contents could break with rules_swift updates, as the worker changes pretty regularly.

[johnno1962]

Ah, I didn't know there were remote builds going on. That might be difficult for injection to support without a lot more work :(. Perhaps these parameter files could be defined as a build product in their own right to have them move between hosts. The patch will be applied again when you inject if rules_swift updates or it is replaced and it doesn't change the contents of the parameters files, it just makes them visible/retains them using a hard link. I've changed the code so injection writes its own copy of the parameters file to make sure other compiles are not affected though they shouldn't have been as these files are normally deleted; The new files are differentiated by the module name anyway.

https://github.com/johnno1962/InjectionIII/releases/tag/4.5.0RC3

At this point I'd like to see if we can get a result on the local-only patching version so it can be evaluated to see if it's worth putting in the extra effort for a more comprehensive version "done properly" by more extensive changes involving bazel. I could work on this if I can find someone working on a large app that is more familiar with bazel to work alongside. The results I've seen this end are very promising in that injection "simply works" as normal provided you add the -interposable linker flag. It could be a real time saver as there is always the extra time of restarting the app, getting back to where you were working/testing etc.

[johnno1962]

I'm looking at another approach that just runs the unpatched bazel build then injects whatever object files have changed. Far simpler but currently tussling with the find command to get the list of object files recompiled.

[saragiotto]

Just test the RC3 and everything works perfectly here. Thank you again @johnno1962 🚀 🎉

As pointed out by @keith, remote cache will be a problem on huge projects.

[keith]

Perhaps these parameter files could be defined as a build product in their own right to have them move between hosts.

The fully processed params files can't be because they contain absolute paths, but if you grab the ones from bazel-out/ and process them yourself that should be mostly ok. Stepping back, are there specific arguments do you need from these? or do you need the full invocation?

[johnno1962]

Excellent! New version is proceeding well even if it's a little slower when injecting. @Keith, quick question, when performing remote builds are the object files copied back onto the machine running the Xcode build? Re: arguments the preference is for fully expanded param/response files that can be simply passed to an xcrun swiftc.

[keith]

when performing remote builds are the object files copied back onto the machine running the Xcode build

Technically you can't rely on that, because bazel has a setting download the minimum set of things, in which case you could just download the .a archive that contains the objects, instead of also downloading the object files that are an input to that, but in practice you might be able to rely on it because that setting has some practical issues, or you could make that a requirement if needed, and users would have to manage that themselves.

arguments the preference is for fully expanded param/response files that can be simply passed to an xcrun swiftc.

makes sense, unfortunately since those args will never be hermetic given the absolute paths, bazel won't ever be able to produce that file in a way that satisfies the remote caching issues above. second level question: do the args have to match the compilation that was actually done 100%, or just semantically match it? for example a lot of argument processing in the worker is pretty trivial find / replace that would be safe to reproduce, but some things like the incremental logic would be much more annoying to have to reproduce. but theoretically you could just drop some arguments and still have a valid invocation, just not a matching one, would that be ok?

[johnno1962]

Right, I've pushed https://github.com/johnno1962/InjectionIII/releases/tag/4.5.0RC4 which takes the simpler and perhaps more flexible approach of using the actual bazel build taken from the Xcode logs to incrementally build during injection rather than approach patching "swift worker" which only works for non-remote builds. If the object files are copied from a remote build into the local filesystem they can be linked into a dynamic library and loaded. The code looks for object files in basel-out/* under a directory _swift_incremental, modified since the modification time of the file being injected.

This version only works if you are running the InjectionIII app and using the bundle in non-standalone mode. If you want the previous RC3 behaviour quit the InjectionIII app and it will fallback to the "standalone" code which runs inside the simulator. This injects a little faster as it is only recompiling rather running the bulk of the bazel build (minus linking which is made to fail quickly by passing an invalid argument). The code is getting pretty messy now as all these building/running variations get catered for but, for now we have what could be a proof of concept.

[keith]

Note that _swift_incremental won't exist in the case builds are built with -wmo

[johnno1962]

aha, good point. WMO on a debug build and injection aren't a good combination or I can negate the "grep" to exclude rather than include this directory. Give me five minutes and I'll re-rollup the RC4 release.

[johnno1962]

Well.. it wasn't so simple as adding -v to the grep of modified object file paths as you loose the object files only being updated if the source file had changed. For now, if it's possible, avoid using WMO during a Debug build if you want to use injection and your incremental compile will be faster for it. I'll see if there is something I can do about this tomorrow though I suspect it will be difficult.

[johnno1962]

I've released https://github.com/johnno1962/InjectionIII/releases/tag/4.5.0RC5. While this new version will inject Objective-C and a "remote" project built with whole module optimizaton enabled (--swiftcopt=-wmo) it's not a happy combination and it only works as it injects all the object files that have changed (which for WMO is all of them in a module).

I tried to optimise to inject only the object file related to the source edited but it seems, under WMO but the object files have shared symbols with "hidden" visibility which can prevent them from dynamic loading independently. Even still, it works reasonably quickly and should faster than relinking and re-running an app but I recommend not using WMO for development builds if you want to use injection. WMO is only really faster if you are doing a clean build for release.

[adityadaniel]

hi @saragiotto I'm curious on how you integrate Injection into your Bazel project because it seems I can make it to work properly (which most likely I don't know how to configure it).

I've setup small sample project here https://github.com/adityadaniel/SimpleBazel, it use rules_xcodeproj to generate Xcode project https://github.com/buildbuddy-io/rules_xcodeproj. Here's my steps

1. Download Injection RC5 https://github.com/johnno1962/InjectionIII/releases/tag/4.5.0RC5
2. Install it on Applications folder, and open it
3. Clone the repository and generate Xcode project
4. Build in Xcode and after it done, Injection will be connected (menu bar icon turns orange)
5. Change the background of ViewController.swift to something else. Injection will try to replace changes but stuck with this error message

   
   💉 ⚠️ Could not locate compile command for /Users/daniel.istyana/Developer/SimpleBazel/Sources/ViewController.swift.
   This could be due to one of the following:

6. Injection does not work with Whole Module Optimization.
7. There are restrictions on characters allowed in paths.
8. File paths in the simulator are case sensitive.
9. The modified source file is not in the current project.
10. The source file is an XCTest that has not been run yet.
11. Xcode has removed the build logs. Edit a file and re-run. Try a build clean then rebuild to make logs available or consult: "/Users/daniel.istyana/Library/Developer/CoreSimulator/Devices/10C2F706-08C3-4EFA-90A6-1CB34ECE5C7E/data/Containers/Data/Application/23151449-86C1-453E-A7A9-A3A64EFF0263/tmp/command.sh".

[johnno1962]

HI @adityadaniel, I was able to get your example project (thanks!) injecting using a .xcproject created using tulsi (bazel run @tulsi//:tulsi). The message you are seeing is because injection is not seeing a particular line in the build logs which seems to be specific to tulsi which I've been using during development. The line needs to start with Running " to switch into "bazel mode" in the RC4-5 versions. What is the command line to generate the .xcodeproj using rules_xcodeproj? I tried bazel build @//Sources:xcodeproj but could not find the project file.

To use tulsi, you need this in your WORKSPACE file:

TULSI_COMMIT_HASH = "518f18da4948192c72074e07fa1dfe15858d40f4"

http_archive(
    name = "tulsi",
    url = "https://github.com/bazelbuild/tulsi/archive/{0}.tar.gz".format(TULSI_COMMIT_HASH),
    strip_prefix = "tulsi-{0}".format(TULSI_COMMIT_HASH),
    sha256 = "92c89fcabfefc313dafea1cbc96c9f68d6f2025f2436ee11f7a4e4eb640fa151",
)

Note: in your ViewController.swift you need an injected method to refresh the display if you want to see anything happening when you inject.

    @objc func injected() {
        viewDidLoad()
    }

[johnno1962]

Update: I was able to generate and find a .xcodeproj file in the end with bazel run //Sources:xcodeproj (D'oh) but it found an interesting new way to fail in that it built the iOS app for Intel on an M1 Mac so the object files were not of the right architecture when you came to rebuild to inject. So, for now, see if you can get tulsi generated project's running where you can also add the required --linkopt="-Wl,interposable" to its build phase. I also suspect rules_xcodeproj defaults to building with whole module optimisation by default which wouldn't be ideal.

Johns-Mac-mini SimpleBazel % git diff
diff --git a/Sources/BUILD b/Sources/BUILD
index fcdd152..9684be2 100644
--- a/Sources/BUILD
+++ b/Sources/BUILD
@@ -14,7 +14,7 @@ swift_library(

 ios_application(
     name = "SampleApp",
-    minimum_os_version = "13.0",
+    minimum_os_version = "14.0",
     deps = [":AppSource"],
     infoplists = ["Info.plist"],
     families = ["iphone"],
diff --git a/Sources/ViewController.swift b/Sources/ViewController.swift
index f489a57..a384941 100644
--- a/Sources/ViewController.swift
+++ b/Sources/ViewController.swift
@@ -9,6 +9,10 @@ import UIKit

 class ViewController: UIViewController {

+    @objc func injected() {
+        viewDidLoad()
+    }
+
     override func viewDidLoad() {
         super.viewDidLoad()
         // Do any additional setup after loading the view.
diff --git a/WORKSPACE b/WORKSPACE
index 30865b5..be2bb3d 100644
--- a/WORKSPACE
+++ b/WORKSPACE
@@ -47,4 +47,12 @@ load(
 )

 apple_support_dependencies()
+TULSI_COMMIT_HASH = "518f18da4948192c72074e07fa1dfe15858d40f4"
+
+http_archive(
+    name = "tulsi",
+    url = "https://github.com/bazelbuild/tulsi/archive/{0}.tar.gz".format(TULSI_COMMIT_HASH),
+    strip_prefix = "tulsi-{0}".format(TULSI_COMMIT_HASH),
+    sha256 = "92c89fcabfefc313dafea1cbc96c9f68d6f2025f2436ee11f7a4e4eb640fa151",
+)

[johnno1962]

I've spent a little more time looking at the possibility of supporting bazel and released https://github.com/johnno1962/InjectionIII/releases/tag/4.5.0RC6.

It works largely as before but has some code detecting when rebuilds produce arm64 when the main build is for x86_64 as can be the case if you use rules_xcodeproj rather than the linker failing silently. It also has a "fix" to force the architecture on the rebuild when it detects this problem.

The situation when using "Whole module Optimisation" has been improved where it no longer injects absolutely everything to resolve hidden symbols shared between object files (when you compile with WMO) but links with the static library bazel produces and the linker will pull in only what it needs.

As before there are two options, the more conservative and slower version when using the InjectionIII app which needs only a line starting with "Running" in your build logs to know how to invoke a a full bazel build (as should be the case if you use tusli) to generate your .xcodeproj. If you quit the app, loading the bundles of the binary InjectionIII falls fall back to use the original implementation I put forward up to RC3 where bazel is slightly patched and the params files made available in /tmp are used to invoke swiftc directly for an incremental build.

There is an extended writeup here. Let me know how you get on.

[Kiesco08]

HI @adityadaniel, I was able to get your example project (thanks!) injecting using a .xcproject created using tulsi (bazel run @tulsi//:tulsi). The message you are seeing is because injection is not seeing a particular line in the build logs which seems to be specific to tulsi which I've been using during development. The line needs to start with Running " to switch into "bazel mode" in the RC4-5 versions. What is the command line to generate the .xcodeproj using rules_xcodeproj? I tried bazel build @//Sources:xcodeproj but could not find the project file.

To use tulsi, you need this in your WORKSPACE file:

TULSI_COMMIT_HASH = "518f18da4948192c72074e07fa1dfe15858d40f4"

http_archive(
    name = "tulsi",
    url = "https://github.com/bazelbuild/tulsi/archive/{0}.tar.gz".format(TULSI_COMMIT_HASH),
    strip_prefix = "tulsi-{0}".format(TULSI_COMMIT_HASH),
    sha256 = "92c89fcabfefc313dafea1cbc96c9f68d6f2025f2436ee11f7a4e4eb640fa151",
)

Note: in your ViewController.swift you need an injected method to refresh the display if you want to see anything happening when you inject.

    @objc func injected() {
        viewDidLoad()
    }

Hi @johnno1962, I tried running this Sample project with your instructions but I keep getting the following error:

could not get the user's cache directory: $HOME is not defined

Any idea what's causing it?

[Kiesco08]

We also use bazelisk to generate our Xcode projects. I wasn't able to get it working unfortunately. This is what I run into:

💉 Using logs: /Users/franck/Library/Developer/Xcode/DerivedData/SampleApp-atheuorjgmdxxecobbfszezgubxe/Logs/Build.
💉 ⚠️ Locating response file failed (see: /Users/franck/Library/Developer/CoreSimulator/Devices/1EE75099-19C5-4735-8C74-7E432E123451/data/Containers/Data/Application/9D69714B-5590-46B8-BC8F-6A3C46A89753/tmp/command.sh)
Could not open filemap '' at /Users/franck/Library/Developer/CoreSimulator/Devices/1EE75099-19C5-4735-8C74-7E432E123451/data/Containers/Data/Application/9D69714B-5590-46B8-BC8F-6A3C46A89753/tmp//bazel.pl line 9.

This is my command.sh:

# search through bazel args, most recent first
cd "/Users/franck/Development/SimpleBazel/bazel-out/../external/build_bazel_rules_swift" 2>"/Users/franck/Library/Developer/CoreSimulator/Devices/1EE75099-19C5-4735-8C74-7E432E123451/data/Containers/Data/Application/9D69714B-5590-46B8-BC8F-6A3C46A89753/tmp/eval101.err" &&
grep module_name_ tools/worker/swift_runner.h >/dev/null 2>>"/Users/franck/Library/Developer/CoreSimulator/Devices/1EE75099-19C5-4735-8C74-7E432E123451/data/Containers/Data/Application/9D69714B-5590-46B8-BC8F-6A3C46A89753/tmp/eval101.err" ||
(git apply -v <<'BAZEL_PATCH' 2>>"/Users/franck/Library/Developer/CoreSimulator/Devices/1EE75099-19C5-4735-8C74-7E432E123451/data/Containers/Data/Application/9D69714B-5590-46B8-BC8F-6A3C46A89753/tmp/eval101.err" && echo "⚠️ bazel patched, restart app" >>"/Users/franck/Library/Developer/CoreSimulator/Devices/1EE75099-19C5-4735-8C74-7E432E123451/data/Containers/Data/Application/9D69714B-5590-46B8-BC8F-6A3C46A89753/tmp/eval101.err" && exit 1) &&
diff --git a/tools/worker/swift_runner.cc b/tools/worker/swift_runner.cc
index 535dad0..3ae653d 100644
--- a/tools/worker/swift_runner.cc
+++ b/tools/worker/swift_runner.cc
@@ -369,6 +369,11 @@ std::vector<std::string> SwiftRunner::ParseArguments(Iterator itr) {
         arg = *it;
         output_file_map_path_ = arg;
         out_args.push_back(arg);
+      } else if (arg == "-module-name") {
+        ++it;
+        arg = *it;
+        module_name_ = arg;
+        out_args.push_back(arg);
       } else if (arg == "-index-store-path") {
         ++it;
         arg = *it;
@@ -410,11 +415,15 @@ std::vector<std::string> SwiftRunner::ProcessArguments(
     ++it;
   }

-  if (force_response_file_) {
+  if (force_response_file_ || 1) {
     // Write the processed args to the response file, and push the path to that
     // file (preceded by '@') onto the arg list being returned.
     auto new_file = WriteResponseFile(response_file_args);
     new_args.push_back("@" + new_file->GetPath());
+    // patch to retain swiftc arguments file
+    auto copy = "/tmp/bazel_"+module_name_+".params";
+    unlink(copy.c_str());
+    link(new_file->GetPath().c_str(), copy.c_str());
     temp_files_.push_back(std::move(new_file));
   }

diff --git a/tools/worker/swift_runner.h b/tools/worker/swift_runner.h
index 952c593..35cf055 100644
--- a/tools/worker/swift_runner.h
+++ b/tools/worker/swift_runner.h
@@ -153,6 +153,9 @@ class SwiftRunner {
   // The index store path argument passed to the runner
   std::string index_store_path_;

+  // Swift modue name from -module-name
+  std::string module_name_ = "Unknown";
+
   // The path of the global index store  when using
   // swift.use_global_index_store. When set, this is passed to `swiftc` as the
   // `-index-store-path`. After running `swiftc` `index-import` copies relevant
BAZEL_PATCH

cd "/Users/franck/Development/SimpleBazel" 2>>"/Users/franck/Library/Developer/CoreSimulator/Devices/1EE75099-19C5-4735-8C74-7E432E123451/data/Containers/Data/Application/9D69714B-5590-46B8-BC8F-6A3C46A89753/tmp/eval101.err" &&
for params in `ls -t /tmp/bazel_*.params 2>>"/Users/franck/Library/Developer/CoreSimulator/Devices/1EE75099-19C5-4735-8C74-7E432E123451/data/Containers/Data/Application/9D69714B-5590-46B8-BC8F-6A3C46A89753/tmp/eval101.err"`; do
    #echo "Scanning $params"
    /usr/bin/env perl "/Users/franck/Library/Developer/CoreSimulator/Devices/1EE75099-19C5-4735-8C74-7E432E123451/data/Containers/Data/Application/9D69714B-5590-46B8-BC8F-6A3C46A89753/tmp//bazel.pl" "$params" "Sources/ViewController.swift"     >"/Users/franck/Library/Developer/CoreSimulator/Devices/1EE75099-19C5-4735-8C74-7E432E123451/data/Containers/Data/Application/9D69714B-5590-46B8-BC8F-6A3C46A89753/tmp/eval101.sh" 2>>"/Users/franck/Library/Developer/CoreSimulator/Devices/1EE75099-19C5-4735-8C74-7E432E123451/data/Containers/Data/Application/9D69714B-5590-46B8-BC8F-6A3C46A89753/tmp/eval101.err" && exit 0
done
exit 1;

Line 9 of bazel.pl is:

my $file_handle = IO::File->new( "< $filemap" )

[johnno1962]

Hi, It's early days for bazel support, thanks for trying it out. Any chance you could try using the InjectionIII app and an .xcodeproj generated using tusli? This should be the more reliable combination. If you're available to put in the time and give me some feedback I can work with, the $HOME problem might be fixable if I roll a new release.

[johnno1962]

I've pushed a new release candidate https://github.com/johnno1962/InjectionIII/releases/tag/4.5.1RC1 which should work better in "bazelLight" mode (selected by loading the bundle but not using the InjectionIII app) with the example project mentioned above https://github.com/adityadaniel/SimpleBazel as a starting point. The "light" version is the one I'd like to see working in the long run as it is faster, far simpler and doesn't require you to use tusli to generate your project ᠆ There were a couple of things I had to fix. If you could give it a try @Kiesco08 with the example project and then perhaps something more ambitious so we can move this forward I'd appreciate it. As before, it creates links and working files in /tmp. As the small patch to bazel has been changed with this release, delete or move the bazel-out/../external directory to reset it.

[Kiesco08]

Hi @johnno1962, thanks for the prompt response! After retrying with the latest build, I'm still getting the following error:

💉 ⚠️ Locating response file failed (see: /Users/franck/Library/Developer/CoreSimulator/Devices/F14DAFD1-A688-4393-8C0A-9AA1ED78C429/data/Containers/Data/Application/CAEBFFD9-E9FE-4ED0-9B35-8E09F421DB05/tmp/command.sh)
Could not open filemap '' at /Users/franck/Library/Developer/CoreSimulator/Devices/F14DAFD1-A688-4393-8C0A-9AA1ED78C429/data/Containers/Data/Application/CAEBFFD9-E9FE-4ED0-9B35-8E09F421DB05/tmp//bazel.pl line 9.

[johnno1962]

Thanks for trying it out. Did you remove the bazel-out/../external directory and see it re-patch the bazel source? Can you tell me the contents of the /tmp/bazel_Sources_AppSource.params file?