Adding "Embed Without Signing" and "Embed & Sign" options for frameworks to xcode

[thomashope]

I am using premake to generate an xcode project and have almost everything working, except I can't figure out how to set third party frameworks to be embedded into the final application bundle. Reading the wiki I found copylocal which appears to describe what I want to do, but is only for C# projects.

Possible Workarounds

1. Emulate "Embed Without Signing" by adding a postbuildcommand to copy the framework into MyApp.app/Contents/Frameworks (I haven't tried this).
2. Emulate "Embed & Sign" by adding a postbuildcommand to copy the framework and then sign it with codesign? (Also haven't tried this).
3. Open Xcode and change the drop down under general > Frameworks, Libraries, and Embedded Content, this is what i'm currently doing...

My current premake5.lua for reference: https://gist.github.com/thomashope/9616970abefa3e3ff55fb7d7316aa907

If there is already support for doing this and I just haven't found it then great! Otherwise I can start putting together a pull request to add this feature if someone can give me guidance on how best to integrate it.

Adding to Premake

If i'm going to work on a pull request my current plan for this would be to have copylocal, when used as in my gist, set the framework to "Embed & Sign" as the (untested) workaround for that seems like more of a PITA than the (also untested) workaround for "Embed Without Signing". I've been poking around the premake source and generated xcode projects and think I have a fair idea of how to do this, but will take some time anyway.

Suggestions welcome :) Thanks for all your work so far!

[samsinsane]

The XCode generator is pretty awful to override using premake.override but you can always try to do that as a fourth option.

I'm not 100% certain if reusing copylocal is the right option, if you need to be able to swap between signing and not signing, copylocal doesn't provide any flexibility there. @starkos any ideas on how this could or should be handled?

[ghost]

It's not terribly complicated in the project file but it's mostly about figuring out how to cleanly specify it in Premake scripts. I've looked at this a couple of times but I found it really tricky to do with the current way that specifying libraries to link. I felt like it should actually be part of links but currently links just adds linker flags which isn't quite right for how you'd normally setup an embedded library for linking. Additionally you wouldn't have options for whether you just want it to copy vs copy and sign. It might just end up needing to be a custom xcodeembed function or similar.

Since I couldn't figure out how to modify Premake to do this, I ended up writing a Ruby script that I run after Premake that uses the xcodeproj Gem to modify my project to add my embedded library. Here's that source for reference in case others want to use it or in case it helps add this functionality to Premake itself.

require "xcodeproj"

# Paths relative to build folder!
LIBS = [
  "../vendor/sdl2-2.0.12/macos/lib/libSDL2-2.0.0.dylib",
]

project_path = File.join(__dir__, "../build/BF05Client.xcodeproj")
project = Xcodeproj::Project.open(project_path)

target = project.targets.first

frameworks_group = project.main_group['Frameworks']

embed_libraries = target.new_copy_files_build_phase("Embed Libraries")
embed_libraries.symbol_dst_subfolder_spec = :frameworks

LIBS.each do |lib|
  ref = frameworks_group.new_file(lib)
  target.frameworks_build_phase.add_file_reference(ref)

  build_file = project.new(Xcodeproj::Project::Object::PBXBuildFile)
  build_file.file_ref = ref
  build_file.settings = {
    ATTRIBUTES: ['CodeSignOnCopy']
  }
  embed_libraries.files << build_file
end

project.save

[ghost]

One more thing I remembered from playing with this is that it should work not just for externally built dylibs and frameworks but also for any other projects in the workspace. If one project builds A.dylib and project B.app references it, I should be able to have A.dylib copied/signed into B.app. That's why I first tried looking at modifying the links behavior because that's currently handling both project and external references.

[samsinsane]

@nickgravelyn Sorry, to clarify, you're saying that there's no need to have a difference between "Don't Copy", "Copy" and "Copy and Sign"? That it should always "Copy and Sign"? If that's the case, why is it optional in XCode?

The Ruby code doesn't really help since I haven't got a clue what that will output, I'd need to see where in the XCode file that was stored, and what it stored. You can find all the XCode generator code here but it is a bit of a mess, the project code is split between project and common.

[ghost]

@nickgravelyn Sorry, to clarify, you're saying that there's no need to have a difference between "Don't Copy", "Copy" and "Copy and Sign"? That it should always "Copy and Sign"? If that's the case, why is it optional in Xcode?

Oh sorry I meant if you tried to use links to infer whether to embed (i.e. if the link is a relative path) then you would lack the ability to specify the behavior. There should be an option for it, like in Xcode, because I believe there is a way to sign a .framework separately in which case you don't need to sign it as part of the app bundle.

The Ruby code doesn't really help since I haven't got a clue what that will output, I'd need to see where in the XCode file that was stored, and what it stored. You can find all the XCode generator code here but it is a bit of a mess, the project code is split between project and common.

Yes I've looked at modifying Premake before but as you say it's a bit of a mess. I haven't tried to go back and add a new command for this yet, though.

I just went and made a new Xcode project and committed it to Git, then added an embedded library. The changes to the project look like this:

1. In the PBXBuildFile section you two entries, one for the library in the Frameworks build phase to link and one in the Embed Libraries build phase to copy:

013BB35F250FA38100A36650 /* libSDL2-2.0.0.dylib in Frameworks */ = {isa = PBXBuildFile; fileRef = 013BB35E250FA38100A36650 /* libSDL2-2.0.0.dylib */; };
013BB360250FA38700A36650 /* libSDL2-2.0.0.dylib in Embed Libraries */ = {isa = PBXBuildFile; fileRef = 013BB35E250FA38100A36650 /* libSDL2-2.0.0.dylib */; settings = {ATTRIBUTES = (CodeSignOnCopy, ); }; };

The key in the second entry is the settings. If you have it present with the ATTRIBUTES set to CodeSignOnCopy you get the "Copy and Sign" behavior. If you have the second entry without the settings then you get the "Embed Without Signing" behavior. The "Do Not Embed" behavior of the dropdown simply omits the second entry (and the Embed Libraries build phase) entirely since you're not doing any embedding.

2. The Embed Libraries build phase is just a PBXCopyFilesBuildPhase that references the file. dstSubfolderSpec of 10 is to specify the Frameworks directory in the app bundle.

/* Begin PBXCopyFilesBuildPhase section */
     013BB365250FA47600A36650 /* Embed Libraries */ = {
         isa = PBXCopyFilesBuildPhase;
         buildActionMask = 2147483647;
         dstPath = "";
         dstSubfolderSpec = 10;
         files = (
             013BB364250FA47600A36650 /* libSDL2-2.0.0.dylib in Embed Libraries */,
         );
         name = "Embed Libraries";
         runOnlyForDeploymentPostprocessing = 0;
     };
/* End PBXCopyFilesBuildPhase section */

3. In the PBXFileReferences section you have, like any other file, just the reference to the actual file on disk (in the case of local references; it's likely different here for project references):

013BB35E250FA38100A36650 /* libSDL2-2.0.0.dylib */ = {isa = PBXFileReference; lastKnownFileType = "compiled.mach-o.dylib"; name = "libSDL2-2.0.0.dylib"; path = "../../Documents/bf05/vendor/sdl2-2.0.12/macos/lib/libSDL2-2.0.0.dylib"; sourceTree = "<group>"; };

After that it's some normal wiring. The file reference is added to the files for the Frameworks build phase for linking, it's added to the proper group to show up in the hierarchy correctly (like any other file), and the Embed Frameworks build phase is added to the native target.

Hopefully that helps a little bit more.

[samsinsane]

Hopefully that helps a little bit more.

That is immensely helpful, thank you! We still need a way to specify this, but I'm wondering if perhaps we can hijack the :static system to specify :copy and :signandcopy. I think @starkos should definitely weigh in on this as he might have a better idea on how we can handle all the additional link settings without the string magic.

I think we'll likely need something like xcodebuildresources but maintains folder structures and allows you to specify which folder. A project at work creates a Framework and copies the header files across which isn't supported by the XCode generator either. Unfortunately, the best I can think of is something like:

xcodecopyphase {
  ["Frameworks"] = {
    ["SignAndCopy"] = { "SDL2.dylib" },
    ["Copy"] = { "Signed.Framwork" }
  },
  ["Resources"] = {
    ["virtual path"] = { "physical path" },
    -- Maybe also supporting this as an alternative way, however using one excludes the use of the other.
    { "physical path" } -- maintains the structure relative to the files specified
  },
  ["Headers"] = {} -- Probably same as Resources
}

Though, maybe having individual APIs for each folder is better? 🤷‍♂️

[thomashope]

Looks like @nickgravelyn beat me to it but here is a diff of my test project with the framework set to "Do Not Embed", "Embed Without Signing" and "Embed & Sign" for what it's worth. Seems to be almost the same except the PBXCopyFilesBuildPhase section is labeled /* Embed Frameworks */ instead of /* Embed Libraries */.

[starkos]

Maybe a stupid question, but how often would you ever not want to embed the framework into your final .app? Could/should it just be the default behavior?

[ghost]

@starkos That's a good question. For me, anything that isn't a built-in system library I want to have it be embedded and code-signed. I don't know a reason not to embed. I did some quick research this morning (not necessarily conclusive or anything) and I also don't know a reason not to code sign on embed. From what I've seen one can code sign a framework/library separately, but you can also just sign them as part of the app which seems to be totally fine. I'm guessing signing a framework/library directly is only necessary when distributing that binary (e.g. through Homebrew or some other distribution mechanism).

So yeah, if links would just automatically configure any non-system libraries/frameworks as embedded frameworks that are embedded and signed I think that'd be exactly what I'd want Premake to be doing.

[thomashope]

Reasons I can think of for not embedding a library include if it's a system library as you mentioned, i suppose premake could keep a list of system libraries and check against them but that list would change over time and be version dependant which feels like a fragile solution.

Another reason is if for example you were developing a suite of applications that come with an installer. Your installer might place any frameworks used in /Library/Frameworks rather than bundling them with each application individually. This means all the apps can be upgraded at once and I think it also means the framework only gets loaded into memory once if multiple apps are running. The apple docs here give some more details.

As for "Embed Without Signing"? Not sure exactly... But this path is supported by apple for distributed apps with a checkbox called Disable Library Validation found under the Hardened Runtime capability in xcode.

IMHO it makes the most sense to support all three options, though it get it's tricky to figure out exactly how.

A heavy handed solution would be just to add xcodeembed and xcodeembedandsign options? Not the most elegant but at least it would be obvious to the end user what's going on?

[ghost]

Reasons I can think of for not embedding a library include if it's a system library as you mentioned, i suppose premake could keep a list of system libraries and check against them but that list would change over time and be version dependant which feels like a fragile solution.

In this case the solution could be that anything in links that is a relative path is considered a non-system library. I believe this is already how local frameworks are required to be specified but currently you can have .a and .dylib files in links as just names that then use the libdirs to locate so it's not as easy to determine.

A heavy handed solution would be just to add xcodeembed and xcodeembedandsign options? Not the most elegant but at least it would be obvious to the end user what's going on?

This seems reasonable to me but I'm not sure how they'd work with links. Currently if you use links with a .dylib Premake just adds -lSDL2 as an additional linker flag rather than "properly" adding it to the libraries list in Xcode. I think that would need to be fixed first and then each of these new commands could just take in the names of links entries to embed. So my Premake file could contain something like:

links { "SDL2" }
filter "action:xcode4"
  xcodeembedandsign { "SDL2" }

Personally I'm fine having more responsibility in my Premake file if it means proper support for embedding libraries so I can remove my second pass Ruby script. Especially with something that's so clearly Xcode specific.

[ghost]

This ended up getting fixed in https://github.com/premake/premake-core/pull/1619 which merged to master. This issue can probably be closed now.

[thomashope]

Thanks everyone for resolving this.

Took a while for me to figure out the correct combination of settings, but here's a snippet of my premake5.lua in case it helps anyone else. The created projects makes an .app that supports notarisation for distribution without triggering gatekeeper.

-- mac specific settings
filter "action:xcode4"
    files {
        "source/mac/Info.plist", -- add your own your .plist and .entitlements so you can customise them
        "source/mac/app.entitlements",
    }

    links {
        "third_party/sdl2/macos/SDL2.framework",    -- relative path to third party frameworks
        "CoreFoundation.framework",                 -- no path needed for system frameworks
        "OpenGL.framework",
    }

    sysincludedirs {
        "third_party/sdl2/macos/SDL2.framework/Headers", -- need to explicitly add path to framework headers
    }

    frameworkdirs {
        "third_party/sdl2/macos/", -- path to search for third party frameworks
    }

    embedAndSign {
        "SDL2.framework" -- bundle the framework into the built .app and sign with your certificate
    }

    xcodebuildsettings {
        ["MACOSX_DEPLOYMENT_TARGET"] = "10.11",
        ["PRODUCT_BUNDLE_IDENTIFIER"] = config.appleBundleId,
        ["CODE_SIGN_STYLE"] = "Automatic",
        ["DEVELOPMENT_TEAM"] = config.appleDevelopmentTeam,
        ["INFOPLIST_FILE"] = "../../source/mac/Info.plist",                     -- path is relative to the generated project file
        ["CODE_SIGN_ENTITLEMENTS"] = ("../../source/mac/app.entitlements"),     -- ^
        ["ENABLE_HARDENED_RUNTIME"] = "YES",                                    -- hardened runtime is required for notarisation
        ["CODE_SIGN_IDENTITY"] = "Apple Development",                           -- sets 'Signing Certificate' to 'Development'. Defaults to 'Sign to Run Locally'. not doing this will crash your app if you upgrade the project when prompted by Xcode
        ["LD_RUNPATH_SEARCH_PATHS"] = "$(inherited) @executable_path/../Frameworks", -- tell the executable where to find the frameworks. Path is relative to executable location inside .app bundle
    }

[starkos]

If you have the time, it would be great to have documentation for these new settings, along with this example. I suspect people will have trouble finding it here! If you're up for it, you can just copy from any of the existing pages in website/docs, and then add any new pages to website/sidebars.js.