Build libSkiaSharp.so and libHarfBuzzSharp.so entirely in make

[mattleibow]

As mentioned by @hurricanehrndz (https://github.com/mono/SkiaSharp/issues/90#issuecomment-307112384), there is not that much needed to build on linux, and the dependency on mono, dotnet and cake is a bit overkill just to run make:

Reasons for current state (https://github.com/mono/SkiaSharp/issues/90#issuecomment-307124367):

• I didn't know make or linux at the time of starting
• building native skia was a bit herder when I started (few extra steps)

This is the previous linux issue: https://github.com/mono/SkiaSharp/issues/90

[hurricanehrndz]

@mattleibow, My only concern is getting libSkiaSharp.so built. I have been able to do that with a minimal make file. I will try and write a small bash script that ensures all the build dependencies are installed and that make is called. I will post something early next week.

[andrew-boyarshin]

Why not just use Skia's GN build file? GN is a robust solution used in Chromium project as well as some other projects, such as Skia. It's not that complicated to setup even on Windows. It also builds with harfbuzz.

[mattleibow]

@andrew-boyarshin the reason you may want libSkiaSharp is that it comes with extras, as well as less :) The basic skia is OK, if you are using C++, but for libSkiaSharp, we have a largish C API, and link out the rest. As a result, you get a tiny little, but comprehensive, package that can do CPU and GPU drawing. In addition, we have some extra APIs that help out with managed wrappers. Finally, we also provide a managed wrapper that can talk to it.

[mattleibow]

@hurricanehrndz I may have something that might be good... I have just creating a makefile that will pop out the native bits. It is awful, should be taken out back and shot, but it works :)

I WILL be accepting ANY PRs, lol.

I will try get it up ASAP.

[andrew-boyarshin]

@mattleibow I am afraid I still don't see your point or I just misunderstood the goal of this issue.

mono/skia has BUILD.gn (of course with modifications to take winrt platform addition and C source set expansions in account). There is also plenty of skia_use_* to control what is being built in. Why make the Makefile that would duplicate the GN build file just for one platform?

I also see that now Cake is used to call GN and then call Ninja. The original build overhead discussed in #90 is about the usage of Cake and therefore the .NET platform to do that. That can indeed be removed, but in my opinion a premade args.gn is sufficient. Why make? Why even shell script? gclient sync && gn gen out && ninja -C out or sth like that is not that hard but provides better understanding what is done on each step.

The addition of alternative build file would definitely slow down the contribution back to upstream (as I can see from BINDING.md it was planned, or at least a year ago).

[mattleibow]

@andrew-boyarshin I see what you mean... simple reason - I am no Linux guy, and I barely know what I am supposed to do for this platform. (I just recently learned about sonames and stuff that make Linux tick) If someone can help out then I will gladly accept. There is also a few bits of code in the SkiaSharp repo that handle the API for managed interop.

I just committed my dodgy Makefile. This does all the work and pops out a native binary that can be used: https://github.com/mono/SkiaSharp/blob/linux-make/native-builds/libSkiaSharp_linux/Makefile. I now need to get into what I need to do, and how I need to do it. I am totally a Windows/MSBuild guy, but I am learning :)

The reason to split libSkiaSharp and libHarfBuzz is that I want to keep them separate as (hopefully) I will replace it with ICU for better text support. Skia doesn't do text shaping, and adding harfbuzz to it helps no one (I think it is just for the tests).

So right now, there are 3 Makefiles:

• build all Linux: https://github.com/mono/SkiaSharp/blob/linux-make/Makefile
• build libSkiaSharp: https://github.com/mono/SkiaSharp/blob/linux-make/native-builds/libSkiaSharp_linux/Makefile
• build libHarfBuzzSharp: https://github.com/mono/SkiaSharp/blob/linux-make/native-builds/libHarfBuzzSharp_linux/Makefile

Right now, libSkiaSharp is basically a linked, C interface designed for managed P/Invoke. As skia does not support text shaping, this has no shaping engines (such as harfbuzz or icu). libHarfBuzzSharp is a "temporary" text shaping engine that can do basic shaping. It is helpful for many simple cases (a single direction and typeface) I want to replace this soonish with icu, which is a much fuller system that can to multi-direction, multi-typeface, wrapping, etc.

I need to spend some time to understand how gn works and to create a build for Linux, and maybe get this to produce the binaries for all the platforms. This way we don't need to spin up cake just to call ninja.

[mattleibow]

This bit is how I start make from cake: https://github.com/mono/SkiaSharp/blob/linux-make/cake/BuildExternals.cake#L720-L732

    RunProcess ("make", new ProcessSettings {
        Arguments = "externals-linux" +
            " SKIA_PATH=" + SKIA_PATH.FullPath + 
            " DEPOT_PATH=" + DEPOT_PATH.FullPath + 
            " HARFBUZZ_PATH=" + HARFBUZZ_PATH.FullPath + 
            " NATIVEBUILDS_PATH=" + ROOT_PATH.Combine("native-builds").FullPath + 
            " ARCH=" + arch + 
            " SKIA_VERSION=" + VERSION_FILE + 
            " HARFBUZZ_VERSION=" + HARFBUZZ_VERSION_FILE + 
            " HARFBUZZ_VERSION_SOURCE=" + HARFBUZZ_VERSION_SOURCE + 
            " SUPPORT_GPU=" + SUPPORT_GPU,
        WorkingDirectory = ROOT_PATH.FullPath,
    });

[mattleibow]

Also want to mention that I currently have to store some of the C/C++ code in the SkiaSharp repo as that will never be merged into upstream skia. I have been trying to upstream the C API - it doesn't do anything extra, just exposes still to C, but Google is not that interested in this at the moment. Basically, what gets into upstream is what Android/Chrome need. The C API is more of a nice to have, but they haven't gotten round to it yet.

We can, or course, have our own BUILD.gn in SkiaSharp and that pulls in the skia bits - sort of what Google is doing to pull in third party libraries. So it is totally feasible, but I think that we have to keep in mind that all changes in the mono/skia repo need to be, at least considering, possible to upstream. So, the SkManagedStream (in SkiaSharp) will never get upstreamed, but the C API for SkStream (in skia) will.

Hope this makes things a bit clearer...

[mattleibow]

@kekekeks @galvesribeiro I thought I would add you in case you were interested... You seem to have done some Linux work before 😉

This is just a Makefile (https://github.com/mono/SkiaSharp/issues/323) that should pop out a native file - it really just download/syncs the native code and then calls the individual Makefiles. I am not sure exactly how you do your own builds, but maybe you can comment. Obviously, I am the worst Linux/make guy, but hopefully it is useful in some way :skeptical-emoji:

[galvesribeiro]

Thanks @mattleibow! I agree with @andrew-boyarshin that if we could use GN as it is the standard tooling from Google it would be great.

However, as you know (and spent days and nights helping me on that), Google tooling is great if you are doing what they want, the way they want and to use how they want. If you need anything different than that, you are doomed (as we were).

In my case, we weren't able to build Skia for ARM Linux just because their build config files (by that time, gyp files) were so tricky and hardcoded that we weren't able to make it build for ARM. If you remember Matthew, we had to hack several gyp files in order to make it build properly and I could finally use it on our ARM device.

So, if using a makefile will enable more scenarios to run SkiaSharp, I'm all up for it! What I don't like is to be caged by Google tooling (believe me, Skia is not my first adventure with Google's code) and have 0 flexibility.

I'll have a look on the new changes over the weekend.

Thank you Matthew!

[mattleibow]

Any comments/suggestions will be very welcome!

[andrew-boyarshin]

@mattleibow ICU LayoutEngine is deprecated and has been removed since ICU 58. ICU encourages users to switch to HarfBuzz which is now more robust and bug-free text shaping engine. So, probably no point in trying ICU LE.

[andrew-boyarshin]

@galvesribeiro GN buildconfigs for Skia indeed seem too small and inflexible compared to Chromium ones for my taste. I was negatively surprised by the lack of real VC tools and Windows SDK search in Skia configs. GCC and MSVC toolchains in one main BUILD.gn file. Well, no wonder it is so given GN was built specifically to suit Chromium needs and Skia only adopted it later for the sake of ecosystem.

[mattleibow]

@andrew-boyarshin the ICU LayoutEngine is deprecated in favour of HarfBuzz as a shaping engine, but not the rest. For example, HarfBuzz cannot handle mixed cultures or typefaces. ICU will first process the text, split it into text chunks and the HarfBuzz will actually do the layout/shaping

[kekekeks]

The problem with ICU is that it's huge (~30MB) so it's super inconvenient to ship as a part of the app. Is it possible to somehow build only "text shaping bits"? Pango, for example, doesn't seem to have a dependency on ICU and only takes around 1MB of disk space.

[kekekeks]

http://apps.icu-project.org/datacustom/ - seems to provide a way to exclude unneeded data

[andrew-boyarshin]

@mattleibow I might not have put emphasis correctly in my comment. ICU LayoutEngine was deprecated and removed, all of it. LayoutEngine itself used to contain the same set of features as HarfBuzz (as far as I can tell from docs, sources, samples, and ICU-LE-HB compatibility layer). LE itself couldn't handle mixed font properties and scripts too, but there is mostly complete (as an extra) scrptrun module with implementation of text split by script algo (sample) without LE dependency. Of course the whole power of the rest of the ICU remains (break iterators, bidi algorithm, scrptrun and so on) and can be used, it's just that HarfBuzz now has little alternative as a cross-platform text shaping library (only Graphite, but its a whole different ecosystem).

[mattleibow]

Just adding this comment from another issue:

This is all extra work, and I am not that up to standard with make. Obviously we accepts PRs and help. Most of the time, you should be able to install the NuGet from nuget.org, and then just include your own native build of the libSkiaSharp.so for whatever distro/architecture you are running.

[mattleibow]

This is no longer relevant as the build is now done with GN and ninja directly: https://github.com/mono/SkiaSharp/wiki/Building-on-Linux