WIP: Android meson integration

[rsglobal]

Require additional dependencies:

# Mainline libyaml with Android.mk file
git clone --branch android https://github.com/rsglobal/libyaml.git external/libyaml

# Chromium libyuv (as libyuv_chromium target)
git clone --branch android https://github.com/rsglobal/libyuv.git external/libyuv_chromium

How to use:

1. Enable libcamera, add pipelines and IPAs into your board.mk file:

   board.mk:
   BOARD_LIBCAMERA_USES_MESON_BUILD := true
   BOARD_LIBCAMERA_IPAS := rkisp1
   BOARD_LIBCAMERA_PIPELINES := simple rkisp1

2. Add the following package configuration into your device.mk file:

device.mk:
    PRODUCT_PACKAGES += \
        camera.libcamera
    PRODUCT_PROPERTY_OVERRIDES += \
        ro.hardware.camera=libcamera

Example of /vendor/etc/libcamera/camera_hal.yaml: https://chromium-review.googlesource.com/c/chromiumos/overlays/board-overlays/+/2866850/2/overlay-soraka-libcamera/media-libs/libcamera-configs/files/camera_hal.yaml

[rsglobal]

CC: @johnstultz-work

[rsglobal]

@kbingham ,

Hello Kieran,

Sorry for creating this PR in your personal repository. Hope you don't mind we do some preliminary review here until it will be ready for submission via mailing list.

[kbingham]

If we can resolve the workaround/hacks, I'd say the meson_cross.mk should be something we could integrate into libcamera! Very interesting work. Needs posting through the libcamera mailing list of course though.

[rsglobal]

Sounds good.

Do you know if raspberry pi guys have any plans to drop boost dependency?

[kbingham]

Sounds good.

Do you know if raspberry pi guys have any plans to drop boost dependency?

It's used for json parsing and property trees. We're trying to find a way to not need boost but it hasn't been a high priority yet. You can disable the boost requirement by not enabling the RPi IPA.

[kbingham]

Just been takling to Laurent about this - he's concerned that the android.mk files support will be deprecated and replaced by .bp files, which are only declarative. Is there a plan to manage this with Mesa?

[rsglobal]

Just been takling to Laurent about this - he's concerned that the android.mk files support will be deprecated and replaced by .bp files, which are only declarative. Is there a plan to manage this with Mesa?

We know that someday Google plan to stop supporting Android.mk's . Also Google announced that it will switch to Bazel as a replacement of Soong. But since date for both is unknown, and Android.mk is the only option known ATM, we have no plans to manage this.

[rsglobal]

@pinchartl,

Hi, )

I just saw https://github.com/kbingham/libcamera/commit/969da31894394db4f64c4a0e96ec2252fb13142b which is causing conflicts. It is very similar to my ("libcamera: Make issetugid() optional").

I can assume someone else is also working on Android integration. Otherwise please don't hesitate to at least keep Suggested-by tag. Or push me to up-stream it next time.

[kbingham]

Hi @rsglobal

There is another team which has been working on android for quite some time as well. Though they have taken a different approach to you, and they are manually creating blue prints, so I don't think they overlap with the same goals.

[pinchartl]

Commit 969da31 has been developed independently from this pull request. I'm sorry for not having noticed your patch first, the change came from a hack I noticed in a parallel development, as mentioned by Kieran. I'm also afraid I can't add a Suggested-by tag as the commit has been merged already.

I'd recommend submitting patches to the libcamera-devel mailing list, as that's what we use for development. Kieran is kind enough to follow issues and pull requests on github, but he can't scale by himself :-)

[rsglobal]

Updated:

• Fixed missing libcamera.so and libcamera-base.so deps
• Small fixes in makefiles
• First successful run. I can see the logs probing for camera device.

[rsglobal]

Latest new: Looks like meson build within AOSP script can build shared libraries only, but can't build executables. Executables require different set of linker arguments, so it require additional handling. And libcamera uses proxy executable for IPA (is it really necessary?....). So I'm currently stuck on bringing-up camera for rpi4.

[kbingham]

The proxy executable isn't necessary as long as you don't run isolated IPAs. For RPi this is fine, as the IPA can be built from source and signed accordingly. Except - you stated above that you don't have gnutls - which is the mechanism that it decides whether to trust the IPA or not I think. So you'll need to do some hacking to disable IPA isolation.

[rsglobal]

@kbingham ,

Thanks, It worked!!!

Got rpi4 pipeline initialized on Android-12 with libcamera. Camera app now visible, but no preview image ATM..

[pinchartl]

One issue with the Android.mk approach is that Android is moving away from make. The current build system, Soong (https://android.googlesource.com/platform/build/soong/+/refs/heads/master/README.md), uses blueprint (Android.bp) files instead, which don't allow running arbitrary commands. They are moving to yet another build system, Bazel, as detailed in https://cs.android.com/android/platform/superproject/+/master:build/bazel/docs/concepts.md, but it will still take some time.

[rsglobal]

@pinchartl ,

So, what's the plan?

[rsglobal]

@kbingham

Looks like raspberrypi pipeline is incomplete. It doesn't set bufferscount during validation as ipu3 does, which disapoints the Android.

[kbingham]

Interesting, the RPi hasn't been tested much with the Android layer, but I'd like to see it tested more. I'd need either a working Chrome or AOSP image for that though.

I see that there was some work to refactor bufferCount recently, but it has stalled. The latest version is https://lists.libcamera.org/pipermail/libcamera-devel/2021-August/023773.html

This might be a path towards solving this - or otherwise, initialising a default bufferCount of say 4 in the RPi pipeline handler might help.

Can you post this on the mailing list perhaps?

[kbingham]

I keep meaning to try: https://github.com/FydeOS/chromium_os-raspberry_pi/releases/tag/r92 but I haven't had any opportunity or time to do so.

[kbingham]

Sounds good.

Do you know if raspberry pi guys have any plans to drop boost dependency?

This is dropped now by the way. I wonder if it helps any development here.

[rsglobal]

This is dropped now by the way. I wonder if it helps any development here.

I'll check it soon, thanks

[rothn]

Hey! @rsglobal @kbingham , I'm interested in this too and can offer help, a second pair of eyes, etc. if that would help get this merged to git.libcamera.org. Let me know!

Additional context: https://github.com/waydroid/waydroid/issues/519

[kbingham]

Can you rebase to the latest libcamera, and make sure everything compiles successfully, then post to the libcamera mailing list please?

[rsglobal]

This is dropped now by the way. I wonder if it helps any development here.

I'll check it soon, thanks

It works.

Can you rebase to the latest libcamera, and make sure everything compiles successfully, then post to the libcamera mailing list please?

Yes, but it would be nice to have code owner for these files from libcamera side. Otherwise it won't work for a long period.

[pinchartl]

The first two commits can be reviewed on the list and merged. The third one is more problematic, as make-based build are deprecated in Android (see https://source.android.com/docs/setup/build).

[rsglobal]

The first two commits can be reviewed on the list and merged. The third one is more problematic, as make-based build are deprecated in Android (see https://source.android.com/docs/setup/build).

It was deprecated long time ago nevertheless it is still supported and currently works and used by mainline mesa3d and everybody is happy :) . Building using NDK is an alternative, but it may have a different set of challenges, and NDK doesn't allow to build it within AOSP.

[rothn]

@rsglobal, My understanding is that @pinchartl's comment refers to Android's new Soong build system, based on Bazel. So, Android.mk would be replaced by Android.bp.

The main challenge here is that libcamera uses the Soong build system. We have at least a few options if we want to use an Android.bp instead of the deprecated Android.mk:

• Transition libcamera to bazel and build intuitively (best, simplest, most consistent, but potentially disruptive)
• Invoke Soong from bazel with a genrule, akin to what meson_cross.mk does (complex-- I would be happy to do this but I'll need help understanding how meson_cross.mk works; probably harder than writing meson_cross.mk was)
• Parallel build systems-- Bazel for ChromeOS and Android, and meson for any other users. This would be less complex and less error-prone than invoking Soong from bazel, and less disruptive than moving to Bazel exclusively, but would incur an increased maintenance burden.

If there's appetite for either of these options, I'm happy to help out where I can. I use Bazel regularly at work, but I'll need help understanding these Makefiles.

For example, we'd have something like this:

cc_library_shared {
    name: "camera.libcamera",
    relative_install_path: "hw",
    vendor: true,
    srcs: [
        "foo",
        "bar",
    ],
    shared_libs: [
        "libc",
        "libexif",
        "libjpeg",
        "libyuv_chromium",
        "libdl",
        "libyaml",
    ],
}

cc_library_shared {
    name: "libcamera",
    vendor: true,
    srcs: [
        "foo",
        "bar",
    ],
    shared_libs: [
        "libc",
        "libexif",
        "libjpeg",
        "libyuv_chromium",
        "libdl",
        "libyaml",
    ],
}

cc_library_shared {
    name: "libcamera-base",
    vendor: true,
    srcs: [
        "foo",
        "bar",
    ],
    shared_libs: [
        "libc",
        "libexif",
        "libjpeg",
        "libyuv_chromium",
        "libdl",
        "libyaml",
    ],
}

...Instead of this:

# SPDX-License-Identifier: Apache-2.0
#
# Copyright (C) 2021, GlobalLogic Ukraine
# Copyright (C) 2021, Roman Stratiienko (r.stratiienko@gmail.com)
#
# Android.mk - Android makefile
#

ifneq ($(filter true, $(BOARD_LIBCAMERA_USES_MESON_BUILD)),)

LOCAL_PATH := $(call my-dir)
LIBCAMERA_TOP := $(dir $(LOCAL_PATH))

include $(CLEAR_VARS)

LOCAL_SHARED_LIBRARIES := libc libexif libjpeg libyuv_chromium libdl libyaml
MESON_GEN_PKGCONFIGS := libexif libjpeg yaml-0.1 libyuv dl

ifeq ($(TARGET_IS_64_BIT),true)
LOCAL_MULTILIB := 64
else
LOCAL_MULTILIB := 32
endif
include $(LOCAL_PATH)/meson_cross.mk

ifdef TARGET_2ND_ARCH
LOCAL_MULTILIB := 32
include $(LOCAL_PATH)/meson_cross.mk
endif

#-------------------------------------------------------------------------------

define libcamera-lib
LOCAL_MODULE_CLASS := SHARED_LIBRARIES
LOCAL_MODULE := $1
LOCAL_VENDOR_MODULE := true
LOCAL_MODULE_RELATIVE_PATH := $2
ifdef TARGET_2ND_ARCH
LOCAL_SRC_FILES_$(TARGET_ARCH) := $(call relative_top_path,$(LOCAL_PATH))$($3)
LOCAL_SRC_FILES_$(TARGET_2ND_ARCH) := $(call relative_top_path,$(LOCAL_PATH))$(2ND_$3)
LOCAL_MULTILIB := both
else
LOCAL_SRC_FILES := $(call relative_top_path,$(LOCAL_PATH))$($3)
endif
LOCAL_CHECK_ELF_FILES := false
LOCAL_MODULE_SUFFIX := .so
include $(BUILD_PREBUILT)
include $(CLEAR_VARS)
endef

__MY_SHARED_LIBRARIES := $(LOCAL_SHARED_LIBRARIES)
include $(CLEAR_VARS)
LOCAL_SHARED_LIBRARIES := $(__MY_SHARED_LIBRARIES)

# Modules 'libcamera', produces '/vendor/lib{64}/libcamera.so'
$(eval $(call libcamera-lib,libcamera,,LIBCAMERA_BIN))
# Modules 'libcamera-base', produces '/vendor/lib{64}/libcamera-base.so'
$(eval $(call libcamera-lib,libcamera-base,,LIBCAMERA_BASE_BIN))

LOCAL_SHARED_LIBRARIES += libcamera libcamera-base
# Modules 'camera.libcamera', produces '/vendor/lib{64}/hw/camera.libcamera.so' HAL
$(eval $(call libcamera-lib,camera.libcamera,hw,LIBCAMERA_HAL_BIN))

#-------------------------------------------------------------------------------

endif

[rothn]

I'll add that a lot of thought has been put into steering people away from using genrule()s like the one described, so I fully expect this to be more challenging than writing meson_cross.mk if not completely infeasible. By far the best option would seem to be moving libcamera to Bazel if we need a .bp, parallel build systems, or accepting a deprecated .mk if the libcamera team doesn't want to use Bazel. Considering that 2/3 use-cases (ChromeOS, Android) for libcamera are Google ecosystems, Bazel might even be the best way to go for the whole project. It's a solid build system too, and I don't immediately see a good reason not to use it.

Interested to read your thoughts on this too, @kbingham!

[rsglobal]

@rothn ,

Each solution has its pros and cons, maintainers should decide which way to go.

@kbingham @pinchartl ,

Since libcamera advertises Android support , could you tell us more how and on what hardware it's used now? It would be very nice to have some android-related how-to docs.

[kbingham]

I would very much like it to be easier to use libcamera with AOSP, so this deserves a discussion on the libcamera mailing list (https://lists.libcamera.org/listinfo/libcamera-devel) not here.

I can't see us changing the build system drastically though, we've chosen meson already - so we'd need to see real clear benefits for the change.

We already support ChromeOS with meson.

Keeping two builds in parallel will be a pain as well though, but if a build for android exists, I think perhaps it's better to live in libcamera.git than outside. At least then it's centralised - but we'd probably want a way to easily automate building against AOSP/NDK so we can track it.

It seems I have a lot of questions about how we can handle this - but I'd really prefer we discuss the options on the actual development list so others can be involved, not just me.

[kbingham]

If we merge the Android.mk file, how do we build test it?

[kbingham]

Since libcamera advertises Android support , could you tell us more how and on what hardware it's used now? It would be very nice to have some android-related how-to docs.

The Android HAL is used by ChromeOS. We currently use this with Intel IPU3 Chromebooks, and I believe it can be used already by a Rockchip RK3399 ChromeTab too, though I don't think that one has been used in quite a while. Most of our development has been with the IPU3.

[rsglobal]

If we merge the Android.mk file, how do we build test it?

Since there're a lot of issues that require resolution, I would suggest to start from something small, like build using Android NDK for raspberry-pi4 and starting semi-official support for it. Then you could extend Android coverage.

And finally adapt Android.mk to build within AOSP.

I think a lot of rpi4-based distributions already adapted meson/Android.mk and use libcamera, but I would still suggest using glodroid/rpi4 images because in this case I could support you.

After successful NDK build you should be able to push binaries onto the board using adb push command.

[kbingham]

@rsglobal Is this something that you are willing to do ? I.e. will you post the support to the libcamera mailing list, and provide documentation on how to build/use it ?

[rsglobal]

@rsglobal Is this something that you are willing to do ? I.e. will you post the support to the libcamera mailing list, and provide documentation on how to build/use it ?

Unfortunately I don't have enough free time for that right now. Also @pinchartl statements that Android.mk is obsolete makes me feel that I'm risking to waste a lot of time without eventually get things up-streamed.

[kbingham]

I'm happy to merge an Android.mk if we can build test it to catch breakage. If Android.mk options become obsolete by Android, then it won't work past that version, but it would still work for earlier versions that the deprecation - so I still think there's value.

But ... the patches need to be posted to the libcamera-devel mailing list - They won't get merged here.

[rsglobal]

@kbingham ,

To many effort has to be done to modify common code to support building within AOSP. I feel like review process will take forever. Also I hate mailing lists.

If you can assign some smart trainee from libcamera team I could share all my experience and support him outside mailing list.

[kbingham]

We're a small team and resource constrained. I don't think we have anyone that could do that right now.

@rothn , as you resumed this conversation here, is this something you want to take on?

[rothn]

Yes, I'm happy to help, not least because this happens to support my phone's camera :). I'd like to take a stab at creating a parallel Android.bp which would be continuously tested with "blaze test ..." or similar. Let me know if that would work for you.

I'll try posting this to the mailing list as well later this evening along with some context, and you may prefer to reply there.

On Tue, Oct 11, 2022, 1:10 PM Kieran Bingham @.***> wrote:

We're a small team and resource constrained. I don't think we have anyone that could do that right now.

@rothn https://github.com/rothn , as you resumed this conversation here, is this something you want to take on?

— Reply to this email directly, view it on GitHub https://github.com/kbingham/libcamera/pull/26#issuecomment-1275087711, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDTDN6LB3OGYQN5PPINQD3WCWUSJANCNFSM46J7WCKA . You are receiving this because you were mentioned.Message ID: @.***>

[rsglobal]

Yes, I'm happy to help, not least because this happens to support my phone's camera :). I'd like to take a stab at creating a parallel Android.bp

Mesa3d was living with similar approach for a long time. Once a week it required tuning and therefore Android build was usually broken. With current Android.mk approach we have just a 1 fix per 6 month which is a way better.

Building using android NDK should similar to building for pure linux.

[rsglobal]

https://docs.mesa3d.org/android.html

[kbingham]

https://docs.mesa3d.org/android.html

Thanks, if we merge support to build android in libcamera, then some equivalent documentation for this should be added to libcamera too so it can be referenced for how people can use it.

[kbingham]

Yes, I'm happy to help, not least because this happens to support my phone's camera :). I'd like to take a stab at creating a parallel Android.bp

Mesa3d was living with similar approach for a long time. Once a week it required tuning and therefore Android build was usually broken. With current Android.mk approach we have just a 1 fix per 6 month which is a way better.

Wrapping the existing meson build sounds more supportable than a parallel build that needs constant maintenance everytime we make a change / add a new object / compile flags etc... But lets follow that discussion on the list.

Building using android NDK should similar to building for pure linux.

Good, that makes it easy to incorporate into the build tests.