Integrate Pixel Backend into Tech Press App

[dylanzika-google]

Please integrate Pixel Specific Changes / Backend into the unified tech press app.

Pixel used standard model set and tflite NNAPI so these changes should be minor?

Related - https://github.com/mlcommons/mobile_app_open/pull/77

Chun-Yen Ho (@ptt2panda) from Google

[d-uzlov]

Our Flutter already has options to build with Pixel backend. Command make WITH_PIXEL=1 docker/flutter/android/apk should give you an APK with pixel backend support.

There is also a build of the Flutter app on the Releases page: https://github.com/mlcommons/mobile_app_open/releases/tag/v3.0.0-flutter-android-alpha-3 You can grab it and test if your backend loads and if there are any issues. You can grep cpp/flutter/dart_backend_match.cc in the adb logcat output to check if the app actually picked up your backend. There is a known issue with offline benchmark, fix for which is not yet merged, but other benchmarks should theoretically work as intended.

[dylanzika-google]

@d-uzlov / @relja128 are we going to build the historical v1.1 application with which this was tested on and verify that first?

Is the request to test this new flutter app on our device to ensure we have reasonable functionality etc?

For 2.0 are we moving to the flutter app for submissions? If so I need to inform my team :) @ptt2panda

[jwookiehong]

Dylan, This build will be 1.1 with the latest backends integrated.

2.0 will be on the old android app. The alpha can be found here: link https://github.com/mlcommons/mobile_app_open/releases/tag/v2.0.1-alpha-mosaic

Please test the 2.0 alpha and let us know if anything breaks. If you have someone that can attend the regular engineering meeting, that would be wonderful.

v/r, -wookie

On Fri, Jan 14, 2022 at 5:15 dylanzika-google @.***> wrote:

@d-uzlov https://github.com/d-uzlov / @relja128 https://github.com/relja128 are we going to build the historical v1.1 application with which this was tested on and verify that first?

Is the request to test this new flutter app on our device to ensure we have reasonable functionality etc?

For 2.0 are we moving to the flutter app for submissions? If so I need to inform my team :) @ptt2panda https://github.com/ptt2panda

— Reply to this email directly, view it on GitHub https://github.com/mlcommons/mobile_app_open/issues/214#issuecomment-1012481958, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQSJ5SJQG45S2LNVY2B6OLTUV4XFZANCNFSM5L2IUHUQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you are subscribed to this thread.Message ID: @.***>

[dylanzika-google]

When is the regular engineering meeting scheduled for? Can you send me and invite and I can see if the TW team can make it?

On Thu, Jan 13, 2022 at 4:32 PM jwookiehong @.***> wrote:

Dylan, This build will be 1.1 with the latest backends integrated.

2.0 will be on the old android app. The alpha can be found here: link < https://github.com/mlcommons/mobile_app_open/releases/tag/v2.0.1-alpha-mosaic

Please test the 2.0 alpha and let us know if anything breaks. If you have someone that can attend the regular engineering meeting, that would be wonderful.

v/r, -wookie

On Fri, Jan 14, 2022 at 5:15 dylanzika-google @.***> wrote:

@d-uzlov https://github.com/d-uzlov / @relja128 https://github.com/relja128 are we going to build the historical v1.1 application with which this was tested on and verify that first?

Is the request to test this new flutter app on our device to ensure we have reasonable functionality etc?

For 2.0 are we moving to the flutter app for submissions? If so I need to inform my team :) @ptt2panda https://github.com/ptt2panda

— Reply to this email directly, view it on GitHub < https://github.com/mlcommons/mobile_app_open/issues/214#issuecomment-1012481958 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/AQSJ5SJQG45S2LNVY2B6OLTUV4XFZANCNFSM5L2IUHUQ

. Triage notifications on the go with GitHub Mobile for iOS < https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675

or Android < https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub .

You are receiving this because you are subscribed to this thread.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/mlcommons/mobile_app_open/issues/214#issuecomment-1012578402, or unsubscribe https://github.com/notifications/unsubscribe-auth/AUNPDJ7YNCAZCK3LUAUKYY3UV5HI3ANCNFSM5L2IUHUQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you authored the thread.Message ID: @.***>

-- [image: Google Logo] Dylan Zika AI Product Manager G-Chips, Pixel, & Edge TPU Group (+1) 832-260-6134

[jwookiehong]

Sent meeting info to your email account.

On Fri, Jan 14, 2022 at 8:04 dylanzika-google @.***> wrote:

When is the regular engineering meeting scheduled for? Can you send me and invite and I can see if the TW team can make it?

On Thu, Jan 13, 2022 at 4:32 PM jwookiehong @.***> wrote:

Dylan, This build will be 1.1 with the latest backends integrated.

2.0 will be on the old android app. The alpha can be found here: link <

https://github.com/mlcommons/mobile_app_open/releases/tag/v2.0.1-alpha-mosaic

Please test the 2.0 alpha and let us know if anything breaks. If you have someone that can attend the regular engineering meeting, that would be wonderful.

v/r, -wookie

On Fri, Jan 14, 2022 at 5:15 dylanzika-google @.***> wrote:

@d-uzlov https://github.com/d-uzlov / @relja128 https://github.com/relja128 are we going to build the historical v1.1 application with which this was tested on and verify that first?

Is the request to test this new flutter app on our device to ensure we have reasonable functionality etc?

For 2.0 are we moving to the flutter app for submissions? If so I need to inform my team :) @ptt2panda https://github.com/ptt2panda

— Reply to this email directly, view it on GitHub <

https://github.com/mlcommons/mobile_app_open/issues/214#issuecomment-1012481958

, or unsubscribe <

https://github.com/notifications/unsubscribe-auth/AQSJ5SJQG45S2LNVY2B6OLTUV4XFZANCNFSM5L2IUHUQ

. Triage notifications on the go with GitHub Mobile for iOS <

https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675

or Android <

https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub

.

You are receiving this because you are subscribed to this thread.Message ID: @.***>

— Reply to this email directly, view it on GitHub < https://github.com/mlcommons/mobile_app_open/issues/214#issuecomment-1012578402 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/AUNPDJ7YNCAZCK3LUAUKYY3UV5HI3ANCNFSM5L2IUHUQ

. Triage notifications on the go with GitHub Mobile for iOS < https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675

or Android < https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub .

You are receiving this because you authored the thread.Message ID: @.***>

-- [image: Google Logo] Dylan Zika AI Product Manager G-Chips, Pixel, & Edge TPU Group (+1) 832-260-6134

— Reply to this email directly, view it on GitHub https://github.com/mlcommons/mobile_app_open/issues/214#issuecomment-1012597406, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQSJ5SOKISVQEMVVBVV3IH3UV5K7FANCNFSM5L2IUHUQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you commented.Message ID: @.***>

[relja128]

This should be the old Android app, built on the 1.1 branch that we released, with all the backends.

[anhappdev]

Here is the Android app with all backends integrated: mlperf_app_qti+samsung+tflite+pixel.apk.zip

[freedomtan]

The one is for 1.1-based. Do we need to test this integrated app? @Mostelk and @freedomtan to test if it works as expected

[dylanzika-google]

Chun-Yen (@ptt2panda) Can you ensure that this V1.1 APK is works on pixel as expected.

ML Perf wants to ensure that it works before the can give it to tech press.

[ptt2panda]

@freedomtan Which tensorflow version did we use for building this apk? One of the optimization for Object Detection would only work if we use tensorflow 2.7, and from the results, it seems that this apk is using earlier version.

[freedomtan]

@ptt2panda I think this based on an earlier version of TensorFlow since this is a v1.1-based one. @anhappdev Could you confirm it?

It seems to me there two possible solutions for the APK,

1. use 2.7 and ask other backends to check if it affects other backends (technically it should be fine. but updating an old branch with new feature always looks problematic :-))
2. check-pick related patches and apply it the Pixel backend.

@Mostelk, @mohitmundhragithub

[ptt2panda]

https://github.com/mlcommons/mobile_app_open/pull/234/files @freedomtan is it OK if we merge this PR?

[anhappdev]

@ptt2panda I think this based on an earlier version of TensorFlow since this is a v1.1-based one. @anhappdev Could you confirm it?

The app was built using https://github.com/mlcommons/mobile_app_open/tree/release-1.1-Sept-2021, so the TensorFlow version used was 2.6.0.

[mohitmundhragithub]

https://github.com/mlcommons/mobile_app_open/pull/234/files @freedomtan is it OK if we merge this PR?

Is the patch already part of v2.7TF?

@ptt2panda I think this based on an earlier version of TensorFlow since this is a v1.1-based one. @anhappdev Could you confirm it?

It seems to me there two possible solutions for the APK,

1. use 2.7 and ask other backends to check if it affects other backends (technically it should be fine. but updating an old branch with new feature always looks problematic :-))
2. check-pick related patches and apply it the Pixel backend.

@Mostelk, @mohitmundhragithub

Generally, TF upgrade shouldn't impact our performance, as we use SNPE backend. But we would still want to try with the upgraded patch.

[anhappdev]

Is there any change to be merged, patched into this branch release-1.1-Jan-2022? Else I will build a new app from that branch.

[ptt2panda]

@mohitmundhragithub I merged the patch to release-1.1-Jan-2022. And yes, that is already part of 2.7 TF.

[anhappdev]

Here is a new build from release-1.1-Jan-2022

2022-01-29_mlperf_app_tflite_qti_samsung_pixel_r1.1_7fd5b8d.apk.zip

For Samsung backend I used snpe-1.54.2.2899 and lib files from the private repo mobile_back_samsung

[ptt2panda]

Hi @anhappdev, I tested the build, and found that this branch doesn't seem to have the pixel backend. I think we might need to patch more changes.

Could you confirm if you used https://github.com/mlcommons/mobile_app_open/tree/release-1.1-Sept-2021 to build the previous apk (mlperf_app_qti+samsung+tflite+pixel.apk.zip)? Because I didn't see pixel backend in this tree either.

[anhappdev]

@ptt2panda I don't have a Pixel device to test, so I could not validate if Pixel backend is correctly recognized. As far as I remember, no one has confirmed whether the Pixel backend worked correctly or not in the release 1.1.

BTW, other backend vendors use a flag in Makefile to specify if a certain backend should be included in the build. For example, I use make app WITH_QTI=1 WITH_SAMSUNG=1 WITH_MEDIATEK=1 to build the app for those 3 backends (TFLite is the default backend and always be included). There is no such flag WITH_PIXEL available in the code. I assume that Pixel use the default TFLite backend and therefore no need for this flag. Is it correct?

[anhappdev]

To be specific, I mean there is no such flag WITH_PIXEL in the branch release-1.1-Sept-2021 and also release-1.1-Jan-2022

[anhappdev]

@ptt2panda I did some research, the Pixel backend was added in this commit 45be2ea, but it's not included in the branch release-1.1-Sept-2021. So please create a PR to add the Pixel backend to the new branch release-1.1-Jan-2022. I will build a new APK after that.

[ptt2panda]

https://github.com/mlcommons/mobile_app_open/pull/240 This version should work. Really thanks for your help!

[freedomtan]

240 is merged. @anhappdev will build a new APK and share the link.

[anhappdev]

Here is the new build from release-1.1-Jan-2022 (commit b957f96): 2022-02-01_mlperf_app_tflite_qti_samsung_pixel_r1.1_b957f96.apk

For Samsung backend I used snpe-1.54.2.2899 and lib files from the private repo mobile_back_samsung

Per log the app did check for QTI, Samsung, MTK and Pixel backend.

App's log

```text 02-01 10:12:12.621 22863 22863 I MLPerfTasks: Set tasks config path: https://github.com/mlcommons/mobile_models/raw/main/v1_0/assets/tasks_v2.pbtxt 02-01 10:12:12.630 22863 22863 I MiddleInterface: Checking support for backend: qti 02-01 10:12:12.632 22863 22863 E native : cpp/backends/external.cc:68 Unable to load: libqtibackend.so: dlopen failed: library "libcdsprpc.so" not found: needed by /data/app/~~WPd7joUXkdxexN8OiCPDGQ==/org.mlperf.inference-i3a5rkWOZKCeU1sJzTXjjA==/lib/arm64/libhta.so in namespace classloader-namespace 02-01 10:12:12.632 22863 22863 I MiddleInterface: Soc Not supported. Trying next lib. 02-01 10:12:12.632 22863 22863 I MiddleInterface: Checking support for backend: samsung 02-01 10:12:12.634 22863 22863 E native : cpp/backends/external.cc:68 Unable to load: libsamsungbackend.so: dlopen failed: library "libofi_rt_framework_user_vendor.so" not found: needed by /data/app/~~WPd7joUXkdxexN8OiCPDGQ==/org.mlperf.inference-i3a5rkWOZKCeU1sJzTXjjA==/lib/arm64/libsamsungbackend.so in namespace classloader-namespace 02-01 10:12:12.634 22863 22863 I MiddleInterface: Soc Not supported. Trying next lib. 02-01 10:12:12.634 22863 22863 I MiddleInterface: Checking support for backend: tfliteneuron 02-01 10:12:12.636 22863 22863 I native : cpp/backends/external.cc:119 Using default allocator 02-01 10:12:12.637 22863 22863 I native : cpp/backend_tflite/tflite_c.cc:133 mtk_neuron_backend backend, Soc Not supported. Trying next backend 02-01 10:12:12.638 22863 22863 I MiddleInterface: Soc Not supported. Trying next lib. 02-01 10:12:12.638 22863 22863 I MiddleInterface: Checking support for backend: tflitepixel 02-01 10:12:12.639 22863 22863 I native : cpp/backends/external.cc:119 Using default allocator 02-01 10:12:12.639 22863 22863 I native : cpp/backend_tflite/tflite_pixel.cc:94 Pixel HW supported check: model: SM-M315F, manufacturer: samsung 02-01 10:12:12.640 22863 22863 I MiddleInterface: Soc Not supported. Trying next lib. 02-01 10:12:12.640 22863 22863 I MiddleInterface: Checking support for backend: tflite 02-01 10:12:12.641 22863 22863 I native : cpp/backends/external.cc:119 Using default allocator 02-01 10:12:12.641 22863 22863 I native : cpp/backend_tflite/tflite_c.cc:149 TFLite backend matches hardware ```

[ptt2panda]

Verified that this version works for Pixel, really thanks for your help.

[freedomtan]

Confirmed that the MTK TfLite Neuron Delegate backend is used on Oppo Reno6 Pro 5G (ColorOS V11.3, Android 11, Dimensity 1200).

[freedomtan]

@fatihcakirs and @mohitmundhragithub also verified this for Samsung and Qualcomn devices.

[freedomtan]

@anhappdev please add the apk to github released page

[anhappdev]

The build from this comment is released here: https://github.com/mlcommons/mobile_app_open/releases/tag/v1.1-android