Native Library - File Size Reduction?

[rac146]

So it looks like the sizes of the Skia library come to about 20 MB for Android and 13 MB for IOS. Is there any way to reduce this by creating custom builds or by another method?

Thanks, Robert

[djcparker]

same question from me. 20MB for Android is pretty heavy. btw; Matt - loving SkiaSharp - awesome job!

thanks Dave

[mattleibow]

Just to note the average sizes per platform architecture:

• iOS/tvOS: ~8MB
• Android: ~14MB
• macOS: ~21MB
• Windows: ~5MB
• UWP: ~4MB

[mattleibow]

I think, after looking at this, the Android is a bit large. I must have a look at what gyp is doing when it generates the project. It might be the case that the build is wrong as with Windows: https://github.com/mono/skia/pull/25

[mattleibow]

I tried a few things, but I can't seem to reduce the size at all. Right now, it IS building release with optimizations, yet it is 15MB vs Windows's 5MB and iOS's 8MB

[djcparker]

it is a big difference.

The realm.io guys had an interesting post on native sizes: reducing apk size, not sure if it is relevant as I haven't pulled the apk apart..

For me, between skia and a few other packages my app is a 40MB download.

[mattleibow]

@djcparker Thanks for those links, trying some options now. I'll see if I can get a doc together to help with ideas for reducing apk sizes

[laarmen]

While researching the subject, I saw this: https://groups.google.com/forum/#!topic/skia-discuss/5hNRcmERVSI

[mattleibow]

@laarmen, thanks for that link. It was very helpful in confirming some not-so-great news. The NDK build already strips the output 😞

The actual size before stripping is ~35-40 MB and this gets stripped to ~13-16. I tried re-stripping the builds, and stripping all debug and other unused symbols, but was unable to reduce the size.

[kekekeks]

@mattleibow Try to compile with -ffunction-sections -fdata-sections -fno-rtti, link with -gc-sections.

[mattleibow]

I changed the build scripts, and managed to get an overall reduction of about 4MB on Android, 2.5MB on iOS, 2.5MB on macOS and 1MB for tvOS.

Not a big drop, but the new sizes are:

• Android: 59.9MB (4 arch)
• iOS: 45.5MB (5 arch)
• macOS: 44.2MB (2 arch)
• tvOS: 19.4MB (2 arch)
• Windows 7: 13.4MB (2 arch)
• Windows 10: 15.4 (3 arch)

@kekekeks have you been able to get smaller files? I am building and linking almost everything in: core, pdf, svg, codecs, intrinsics.

[kekekeks]

The smallest of my buillds was 1.7MB for arm32 Android. I was not explicitly excluding anything from build, just linked unused stuff away.

[mattleibow]

Let me look some more... i think i found that if I run strip after the build, then i can get an iOS arch from 11.5MB to 7.5MB. I need check that all works still, and nothing is getting stripped away.

[mattleibow]

I have managed to drop iOS/tvOS quite a bit to ~5MB per arch - which is the same as Windows/UWP. macOS is a bit smaller at ~15MB per arch and Android is ~12MB per arch. As I am looking at Linux as well (#90) I notice that it is also ~16MB per arch. I still want to see why Linux/Android/macOS are so much larger, even with the same strict compile/link flags. I even made everything private and am just keeping the C API public (size did drop a bit).

Basically, everything is ~5MB, except the macOS/Android/Linux builds which are ~15MB.

@kekekeks were you linking in the GPU bits skgpu=1. And, are you able to see what size your build is when you link this stuff in: native-builds/libSkiaSharp_android/jni/SkiaSharp.mk (also not that those flags in the makefile only apply to the final library, not all the others)

[kekekeks]

My ogirinal Linux build with GPU support was 2.6MB.

[kekekeks]

I'm not sure that PDF was linked in. You see, I wasn't actually exposing Skia API itself, but implementing our own drawing API using Skia. So I'm sure that rendering, GPU support and font/image loading was included, but at least PDF wasn't included. On Linux I was also linking to external ZLib and libpng instead of including static ones.

[rocuh]

I'm no expert on elf format by any means, but i did take a quick look at the .so file for android and noticed that the .rodata section was the largest at around 7MB and code (.text) was around 4.8MB .

  [Nr] Name              Type            Addr     Off    Size   ES Flg Lk Inf Al
  [ 8] .text             PROGBITS        00191ed0 191ed0 497748 00  AX  0   0 16
  [11] .rodata           PROGBITS        00639c70 639c70 6a4df0 00   A  0   0 16

That seems a lot of constant data, it only looked like a few KB of text at the start of the .rodata section, no idea what the rest of it is.

[kekekeks]

Could you guys upload an unstripped android build somewhere? I don't have a mac machine nearby.

[rocuh]

.rodata 0x000000000057b480 0x642520 jni/../../../externals/skia/out/config/android-x86_64/Release/obj/gyp/libicuuc.a(jni/../../../externals/skia/out/config/android-x86_64/Release/obj/gyp/../third_party/externals/icu/android/icuuc.icudtl_dat.o)

It looks like on linux, mac and android there is a file linked in which is 6.5MB. I assume it is something do with with unicode formatting ("externals/skia/third_party/externals/icu/android/icudtl.dat ").

[rocuh]

Maybe this can be used to create a more compact icu data file:

http://apps.icu-project.org/datacustom/

[rocuh]

Actually i think that online tool is for ICU57 and skia uses 56.1 currently, so this one is the correct one:

http://apps.icu-project.org/datacustom/ICUData56.html

There is also a project that can be used:

http://download.icu-project.org/files/icu4c/56.1/icu4c-56_1-data.zip

[mattleibow]

@roceh Thanks for this info, I also determined that it was the icu bits. My method was: link nothing and then add bits until is jumps :) Real scientific.

Anyway, I am looking at Linux right now and I think I can get a nice small build there if I use the system version.

What I might do is to not include the font subsetters on the mobile devices (skia_pdf_use_sfntly=0). This is only used by the PDF bits, and may not be too essential. The UWP actually doesn't have the subsetters due to there being an issue during compilation (I don't think the icu builds for UWP yet).

[mattleibow]

@roceh I am having a look and I think the size of the icu data is about 10MB (give or take). The reason I was able to shrink the Windows/UWP/iOS/tvOS to ~5MB was because they don't include the subsetter. The macOS/Android/Linux builds do.

I can't use the system icu on linux as the version is not available via apt-get, so i will have to link it in. However, I don't think the size matters here since both macOS and linux are desktops, thus an extra 10MB might not be a problem.

[rocuh]

I think like you said it is better to remove the font subsetter, the large apk size is a large price to pay for slightly smaller PDFs, and if the iOS builds are not doing it anyway probably best to be consistent as well.

[kekekeks]

thus an extra 10MB might not be a problem.

.NET core's footprint is only 50MB, so additional 10MB might be huge.

It would be nice to introduce some kind of pay-for-play system where PDF support lives in a separate package, but I'm not sure that it's even possible to build libskia that way.

But linking system libecu is probably a bad idea from portability point of view anyway.

[mattleibow]

After all the changes and linking, I have a result of:

• removed the public C++ API (each arch dropped about 1MB) origionally both the C and C++ was visible, now just C
• stricter stripping of Android, iOS, tvOS and macOS strip all, instead of just debug symbols
• removed the PDF font subsetter for Android and macOS the other platforms don't have it, except for Windows which has only an extra 1MB overhead

The new sizes are:

• Android (total: 22.4MB for 4 arch => ~5.6MB per arch)
• iOS (total: 24.8MB for 5 arch => ~4.96MB per arch)
• macOS (total: 10.4MB for 2 arch => ~5.2MB per arch)
• tvOS (total: 10.4MB for 2 arch => ~5.2MB per arch)
• Windows (total: ~13.4MB~ 12.9MB for 2 arch => ~~6.7MB~ 6.4MB per arch)
• UWP (total: ~15.4MB~ 14.2MB for 3 arch => ~~5.1MB~ 4.7MB per arch)
• Linux (total: 5.4MB for 1 arch => ~5.4MB per arch) [in progress] https://github.com/mono/SkiaSharp/issues/90

All the platforms include the core, the codecs, the GPU and the PDF bits.

These should be the sizes of the native bits in the next release. And, the NuGet should be smaller even more since I am avoiding native files. I added the mac bits twice to reduce configuration, now this is no longer the case.

I'll see if I can get a beta out soon so that you can give it a go before it goes stable. Just in case you are using something that somehow got linked out when it shouldn't. But I think it is all good.

[kentcb]

It would be nice to introduce some kind of pay-for-play system

Agreed, but wouldn't it be 💯 if the platform tooling supported linking both managed and unmanaged code? For example, Xamarin's linker is totally OK linking out the unused managed SkiaSharp code for me, but imagine if it spelunked into the native code and did the same thing? Since I only call a handful of APIs, I could imagine my APK dropping from 40MB to maybe 15MB!

[mattleibow]

I am closing this as I think this is "solved" for now. We should probably open a new issue for things like builds without GPU/PDF.

[kentcb]

@mattleibow awesome! Looking forward to this.

Any idea when 1.56.1 will be shipping?

[mattleibow]

@kentcb Sometime real soon, like when CI finishes. :) Well actually, it is going to take 2 hours, but I need to sleep (for 6 hours). So, if all goes well, I'll wake up to a green build and then release.

It is going to be marked beta, but this is just because I want to make sure that the changes to the packaging for .net core and .net standard didn't break anyone (it shouldn't) and the the stricter linking for smaller sizes didn't exclude anything (it shouldn't have). If no-one complains in a day or so, I'll release stable - this'll also give me time to finish up the API docs for intellisense and all that.

Here are some of the big and cool things from the release notes:

• Support for .NET Standard 1.3 project types
• Support for .NET Core (win7, win10, osx)
• Reduced native library sizes to ~5MB per platform architecture (#174)
• Partial work to get a .NET Core Linux build. (#90)

[mattleibow]

@kentcb Should be usable right now!

Release Notes: https://github.com/mono/SkiaSharp/releases/tag/v1.56.1-beta NuGet: https://www.nuget.org/packages/SkiaSharp/1.56.1-beta

[kentcb]

FYI, my APK has bloated again by ~10MB changing nothing but Xamarin tooling from pre 15.2 (cycle 9?) to 15.2.2:

Chasing up with Xamarin, but thought I'd mention here too.

[kentcb]

With 15.2.2, the native bits are being embedded as a resource in the SkiaSharp assembly:

// 0x0000224C: __AndroidNativeLibraries__.zip‎ (9526790 bytes, Embedded, Public)

[djcparker]

@kentcb - the size bloat is mentioned in #259 and links to https://bugzilla.xamarin.com/show_bug.cgi?id=53250 which is still showing as "confirmed" suggesting it hasn't been included yet... which is a right pain as my apps have bloated and I can't get the linker workaround to work properly yet...

[mattleibow]

I can't say for sure right now, but I think this was fixed in 15.3. The regression came in due to a bug in Mono.Cecil, which was then updated for 15.3.

You can try the alpha channel and test it out (and switch back if need be).

[djcparker]

@mattleibow nice one! Will do that when I get some time.

[kentcb]

Ah, thanks @djcparker @mattleibow! I've made Jon Douglas (Xamarin) aware of this because I was liaising with him yesterday on this issue. I'll report back if I can get a confirmation it's fixed in alpha.

[kentcb]

OK, tried switching Bitrise to alpha channel and rebuilding. Unfortunately, it still produces a bloated APK :(

[djcparker]

@kentcb :( not good.

[mattleibow]

I am sure this was fixed. You should try archiving your app. It may not reduce the app size if you are running in Debug to reduce compile times, but I am able to get my app reduced if I Archive, even when I just set the linker to "Link SDK assemblies only".

This is my test app: https://github.com/mattleibow/SkiaSharpDemo

[djcparker]

Hmmm, ok. Will try the latest alpha :)

[kentcb]

@mattleibow I am building with Bitrise. Prior to 15.2 my APK was ~30MB. With 15.2 and no code changes, it's now ~40MB. Cracking open the APK and looking inside shows that the native code is in two places: inside lib and also embedded as a resource inside SkiaSharp.dll. This is true of both stable Xamarin and alpha Xamarin. The bug report is CONFIRMED with no mention of a fix being in place in any channel yet.

[mattleibow]

@kentcb If you build my sample app (https://github.com/mattleibow/SkiaSharpDemo) does the same thing happen? I can't seem to reproduce when running "Archive for Publishing".

What is your IDE and your version numbers?

Mine are: https://gist.github.com/mattleibow/bff95014b9cee90c828a67f1ed335fc7

[kentcb]

@mattleibow I work locally with VS, but I don't use that to produce productions APKs/IPAs - they always come through Bitrise. So I'd be comparing apples to oranges if I looked at my VS output.

Hmmm, I wonder whether it has to do with the Bitrise images. They've upgraded to 15.2.2, but they do not yet include VS4Mac, only XS. There's a small chance that's a factor. Bitrise plan to include VS4Mac in their images sometime in the next week or two. Once they've done that, I'll report back here as to whether that rectifies the problem.

All the same, Xamarin have confirmed it is an issue, so I assume it still is.

[mattleibow]

@kentcb I just built Release of my sample app using the stable VS and everything appears to be fine. No need for anything fancy, I just did a plain old Release build on Any CPU. I will point out that the SkiaSharp.dll in the bin folder STILL has the embedded native libraries (9MB), but the one in the .apk does not (200KB).

Does my sample work for you locally in VS? And in your Bitrise build, could you send (maleib@microsoft.com) the build log? I will have a look and see what is up.

Here is my VS config: https://gist.github.com/mattleibow/8395177ffec5f0ef61d95ccf0eb0dfe3

[kentcb]

@mattleibow interestingly, my APK dropped from 45MB to 35MB again today when I merged my branch to switch us to netstandard and SDK-style csproj 🤔

[EmilAlipiev]

Hello, this is crazy. using version 1.60.0, it ads 3,8 mb size while 1.60.1 adds 9mb.. what has changed in between? you didnt even increase major version number. it cant be such a big difference. see this issue

https://github.com/luberda-molinet/FFImageLoading/issues/893

[mattleibow]

@EmilAlipiev thanks for reporting this. The build system changed a fair bit and the stripping may have fallen back to the original settings. I opened a new issue #573 and I will see if I can get this resolved ASAP.

[mattleibow]

Could all new comments be made on the new issue so we can track them with the next release.

[EmilAlipiev]

I think that size was increased also from 3mb to 5mb between 1.6x version to 2.8x version. it is not a big number but i want to just inform you :)