Build CoreKiwix.xcframework

[rgaudin]

Fixes https://github.com/kiwix/kiwix-build/issues/455 Fixes https://github.com/kiwix/kiwix-build/issues/590

CoreKiwix.xcframework is the packaging format required by Kiwix apple (macOS/iOS) reader for libkiwix. Naming will hopefully be revisited later (libkiwix.xcframework?)

This folder is a combination of all static archive (*.a): dependencies and libkiwix itself for all apple-supported platforms: macOS x86_64, macOS arm64, iOS arm64 and iOS x86_64. It also includes the headers. Note: while iOS archs are separated in the framework, the macOS archs are bundled as a fat binary.

This thus adds:

• A new apple_all_static target that lists mentioned sub-targets. Is name OK?
• A new AppleXCFramework virtual dependency and builder
  • requires libkiwix
  • merges the archives into one per target
  • creates the fat binary for macOS
  • creates the xcframework
• Special handling in github/common to produce the archive from diff. files archive
  • it's a libkiwix export afterall
  • requires a new xcframework (is name appropriate?) platform_name in builddef

Ref:

https://developer.apple.com/documentation/xcode/creating-a-multi-platform-binary-framework-bundle

⚠️ DO NOT MERGE YET I want to investigate #650 first

[mgautierfr]

How xcframework (and new dependency AppleXCFramework) compare the iOS_multi (and the ios_fat_lib dependency) ?

Is iOS_multi still used ? (@automactic) xcframework use fatlib for macos but not iOS. Why we use fatlib for iOS elsewhere ?

[rgaudin]

I've thoroughly described what the xcframework includes. iOS_multi is not used anymore by kiwix. I suggest, given our low throughput on apple platform to only keep what we use until there's a request for something else. But that's out of the scope of this PR IMO

[kelson42]

@rgaudin Great but what was ios_multi?

[rgaudin]

a fat binary of both arch for iOS

[rgaudin]

dylib I believe

[rgaudin]

@kelson42 you edited the PR comment to indicate it fixes #590, that's not the case… but the ticket is not precise, so to be clear:

• this does not concern libzim
• libzim for macOS is already published (as a shared library)
• we don't plan to distribute libzim for iOS (because we don't need it ourselves)
• this archive contains static objects for both libzim and libkiwix (and deps) so devs could use it to retrieve zim symbols transparently

So I'm not sure what to do with that ticket 🤷‍♂️

[kelson42]

@rgaudin Your PR "does the job" and ticket can be closed. I will put a link to your last comment to be even more clear.

[rgaudin]

OK so as I suspected, there's a CI bug that makes a build using base_deps2 from succeeding:

[BUILD]
build pugixml (macOS_x86_64):
WARNING: qmake command not found.
  configure pugixml : ERROR
Stopping build due to errors
run command 'meson . /Users/runner/./BUILD_macOS_x86_64/pugixml-1.2 --buildtype=release  --default-library=static  --prefix=/Users/runner/./BUILD_macOS_x86_64/INSTALL --libdir=lib --cross-file /Users/runner/./BUILD_macOS_x86_64/meson_cross_file.txt'
current directory is '/Users/runner/SOURCE/pugixml-1.2'
ERROR: Neither source directory '.' nor build directory '/Users/runner/./BUILD_macOS_x86_64/pugixml-1.2' contain a build file meson.build.
WARNING: Running the setup command as `meson [options]` instead of `meson setup [options]` is ambiguous and deprecated.

If I remove that file, then the CI properly builds its deps, (uploads them) and compiles fine.

It means that for some reason, the uploaded+downloaded deps differ from just having them built in the same job. @mgautierfr does that ring a bell?

[mgautierfr]

@mgautierfr does that ring a bell?

Yes.

You probably need a files_to_archive += HOME.glob("BUILD_macos*/**/.*_ok") just after files_to_archive += HOME.glob("BUILD_android*/**/.*_ok")

The .*_ok files are use to trace if the command has been run. If not present kiwix-build tries to do the command, else it skip it (search for try_skip). So we must keep theses files else we want to run the configure/compile even if we have saved the INSTALL directory (and even if we don't have saved the sources)

[rgaudin]

Ah ok I wondered what those were

[rgaudin]

@mgautierfr I eventually fixed the base_deps but what I had to do might not be the best practice. Please take a look at the last commits.

[mgautierfr]

Hum, it seems that it is because we skip the configure. (This should be broken since a long time, but as we don't launch android (only) build, we never found it).

I think it would be better to remove this line to configure all our projects (even if we have a .*_ok file). (As we do for build and install).

It means that kiwix-build run localy will reconfigure the project everytime it is run but I can live with it (or patch it to a real solution)

[rgaudin]

@mgautierfr you mean to remove that line instead of 15fed1c right? Or would it be in addition to it?

[mgautierfr]

Instead of (if I am right about the issue here. Don't lose the commit)

[rgaudin]

@mgautierfr OK we're now green (appveyor finishin). Should we keep the last commit (libzim version bump) or not ? Let me know if I can rewrite history

[mgautierfr]

Yes you can rewrite the history.

Please check with a change of base_deps_meta_version:

• Be sure that base_deps2_*_<base_deps_meta_version>.tar.xz don't exist on server
• Launch a CI build
• Launch a second CI build ("Rerun all jobs") to check it passes with deps archives.

It is a pity as this double run take 2h long but it the price to check everything is ok.

Should we keep the last commit (libzim version bump) or not ?

We may keep it in the PR to check CI is passing but better to remove it to not mix things.

[rgaudin]

Please check with a change of base_deps_meta_version:

Indeed ; I forgot this affected all builds. It's now OK.

Let me know if that's OK to rewrite/rebase

[mgautierfr]

Let me know if that's OK to rewrite/rebase

Yes. That's OK.

[kelson42]

FYI @automactic has made the xcf with a .zip file, and we do .tar.gz, not sure what is the best, I just noticed.

[rgaudin]

We discussed it during the meeting (!). It's just a transport format and supported on macOS so no impact but it prevents additional code for a special case on kiwix-build