Closed freedomtan closed 1 year ago
@freedomtan add most descriptions for what was proposed for vendor backend and the "fallback" tflite backend.
Current status: platforms / devices supported by current app is quit limited. Our vendors backends support most recently high-end or middle-end devices only. And vendor backends probably do support new high-end devices with high-end SoCs (because SoC vendors have not submitted newest patches / settings yet. To mitigate this problem, we suggest to add
To be: Suppose we have a phone with Samsung Exynos 9999,
false
when mlperf_backend_matches_hardware()
is calledfalse
now. We probably want change it to return true
and add a warning or alter message saying "hey, it's a Samsung platform, but we are gonna run on CPU with TFLite interpreter for now". The user could select to continue or not. If there is a new from Oppo with Oppo's SDK, we go to the "fallback TFLite backend".
If there is a MediaTek SoC, but the MediaTek backend just refuses to support it. Then we go to the "fallback TFLite backend", too.
I would propose an alternative flow to choose the backend
:
Let say an Android phone with supported Samsung hardware.
mobile_back_samsung
and mobile_back_tflite
are available)mobile_back_samsung
or ask the user which backend he wants to use.Regarding choosing a delegate
after choosing a backend
:
I know that TFLite
has multiple delegates
we can choose from, but what about other backends? We will probably need an API to ask the backend
if it has multiple delegates
and an API to switch to that selected delegate
.
a case: suppose we have a new Exynos 9000 SoC: the choices to run TFLite or any other fallback stuff should be decided or say controlled by the Samsung backend.
@freedomtan to check if we can ask a Google TFLite guy to talk to us (about delegate related stuff)
@Mostelk @freedomtan @AhmedTElthakeb @mohitmundhragithub The flow we discussed in the meeting today:
vendor_backend
can decide either:
2.1 the phone is supported, continue with vendor_backend
2.2 the phone is not supported, show an error to user that this phone is not supported
2.3 the phone is not supported, continue with fallback mobile_back_tflite
@anhappdev: make the choice of TFLite delegates (NNAPI delegate, GPU delegate, and CPU only) available to users.
2.2 is not preferred. 2.2.5: tflite interpreter + whatever options you want to add in the vendor backend as the latest line of defense, then you don't need to go to 2.3.
outline/define the interface for users to choose as @mohitmundhragithub mentioned. Hopefully, we can use the same component for both vendor backend and the mobile_back_tflite
.
Currently, we check if the NNAPI Delegate works first. If yes, we use it. If no, falling back to CPU (TFLite interpreter). It would be more convenient if uses could check if she/he wants to use