jiuqiant
commented
3 years ago How about the following?
eglManager = new EglManager(null);
processor =
new FrameProcessor(
this,
eglManager.getNativeContext(),
BINARY_GRAPH_NAME,
INPUT_VIDEO_STREAM_NAME,
OUTPUT_VIDEO_STREAM_NAME);
processor.getVideoSurfaceOutput().setFlipY(FLIP_FRAMES_VERTICALLY);
float focalLength = cameraHelper.getFocalLengthPixels();
if (focalLength != Float.MIN_VALUE) {
Packet focalLengthSidePacket = processor.getPacketCreator().createFloat32(focalLength);
Map<String, Packet> inputSidePackets = new HashMap<>();
inputSidePackets.put(FOCAL_LENGTH_STREAM_NAME, focalLengthSidePacket);
processor.setInputSidePackets(inputSidePackets);
}
b4017816
sgowroji
mcclanahoochie
rahulsuxsena1
fadi-yas
Hi, I'm trying to integrate Iris into an android app using an .aar, I was able to set up and build the app for face detection using the .aar tutorial, but I'm trying when trying to modify the example to use an Iris .aar I'm running into issues. The app builds and deploys, starts to run then crashes giving exception "focal_length_pixel" is required but not provided, which is expected but I don't know how to add it to the main activity correctly.
Does the .aar I built contain the calculators for the subgraphs as well as the main graphs or do I need to add something else to my AAR Build File? AAR Build File: load("//mediapipe/java/com/google/mediapipe:mediapipe_aar.bzl", "mediapipe_aar") mediapipe_aar( name = "mp_iris_tracking_aar", calculators = ["//mediapipe/graphs/iris_tracking :iris_tracking_gpu_deps"], )
Android Studio Assets: iris_tracking_gpu.binarypb face_landmark.tflite iris_landmark.tflite face_detection_front.tflite
I've tried to add code for the focal length to onCreate based on the Iris example but I don't know how to modify this to work with an Iris .aar, Is there any further docs I can read to point me in the right direction? I'm trying to create an accessibility app that can utilise Iris for infererring gaze direction for use in controlling a text to speech keyboard, I Would really appreciate any help or further reading materials, Thanks :)
I need to integrate this snippet (I think) into the modified code for the facedetection example but not sure how.
if (!haveAddedSidePackets) { float focalLength = cameraHelper.getFocalLengthPixels(); if (focalLength != Float.MIN_VALUE) { Packet focalLengthSidePacket = processor.getPacketCreator().createFloat32(focalLength); Map<String, Packet> inputSidePackets = new HashMap<>(); inputSidePackets.put(FOCAL_LENGTH_STREAM_NAME, focalLengthSidePacket); processor.setInputSidePackets(inputSidePackets); } haveAddedSidePackets = true;
Modified Face Tracking AAR example: package com.example.iristracking;
// Copyright 2019 The MediaPipe Authors. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License.
import android.graphics.SurfaceTexture; import android.os.Bundle; import android.util.Log; import java.util.HashMap; import java.util.Map; import androidx.appcompat.app.AppCompatActivity; import android.util.Size; import android.view.SurfaceHolder; import android.view.SurfaceView; import android.view.View; import android.view.ViewGroup; import com.google.mediapipe.components.CameraHelper; import com.google.mediapipe.components.CameraXPreviewHelper; import com.google.mediapipe.components.ExternalTextureConverter; import com.google.mediapipe.components.FrameProcessor; import com.google.mediapipe.components.PermissionHelper; import com.google.mediapipe.framework.AndroidAssetUtil; import com.google.mediapipe.framework.Packet; import com.google.mediapipe.glutil.EglManager;
/* Main activity of MediaPipe example apps. / public class MainActivity extends AppCompatActivity { private static final String TAG = "MainActivity"; private boolean haveAddedSidePackets = false;
} `