Bridging to React Native

naimroslan commented 10 months ago

Before opening, please confirm:

[X] I have searched for duplicate or closed issues.

Which UI component is this feature-request for?

Liveness

Please include any relevant guides or documentation you're referencing

No response

Please describe your feature-request in detail.

I'm trying to bridge the FaceLiveness from Android to React Native. I have make it work successfully on Android. I had an issue before where the camera turns off immediately once it began scanning someone's face and it was Auth issue, and I fixed it. But now, it happens again on React Native.

Does anyone ever encounter the same issue as mine when bridging from Android to React Native?

Thank you in advance :)

Initialization steps (if applicable)

No response

Code Snippet

App.tsx

This is my React Native where I called startFaceLiveness with onPress.


const ToastService = NativeModules.FaceLivenessModule;

function App(): JSX.Element {

  return (
    <View style={styles.container}>
      <View>
          <TouchableOpacity 
            onPress={() => ToastService.startFaceLiveness() }
            style={styles.button}  
          >
            <Text style={styles.buttonText}>Start FaceLiveness</Text>
          </TouchableOpacity>
        </View>
    </View>
  );
}

ToastService.kt

This is Kotlin class in Android folder where I call LivenessActivity


class ToastService(private val reactContext:ReactApplicationContext):ReactContextBaseJavaModule(reactContext) {

    override fun getName(): String {
        return "FaceLivenessModule"
    }

    @ReactMethod
    fun startFaceLiveness() {
        val currentActivity = currentActivity

        if (currentActivity != null) {
            val intent = Intent(currentActivity, LivenessActivity::class.java)
            currentActivity.runOnUiThread {
                currentActivity.startActivity(intent)
            }
        } else {
            Log.e(TAG, "Current activity is null")
        }
    }

    companion object {
        private const val TAG = "RNAndroidFaceLiveness"
    }
}

amplifyconfiguration.json


{
    "UserAgent": "aws-amplify-cli/2.0",
    "Version": "1.0",
    "auth": {
        "plugins": {
            "awsCognitoAuthPlugin": {
                "UserAgent": "aws-amplify-cli/0.1.0",
                "Version": "0.1.0",
                "IdentityManager": {
                    "Default": {}
                },
                "CredentialsProvider": {
                    "CognitoIdentity": {
                        "Default": {
                            "PoolId": "ap-northeast-1:xxxxxxxx",
                            "Region": "ap-northeast-1"
                        }
                    }
                },
                "CognitoUserPool": {
                    "Default": {
                        "PoolId": "ap-northeast-1_1Oxxxxxx",
                        "AppClientId": "5sjncfxxxxxx",
                        "Region": "ap-northeast-1"
                    }
                },
                "Auth": {
                    "Default": {
                        "authenticationFlowType": "USER_SRP_AUTH",
                        "socialProviders": [],
                        "usernameAttributes": [
                            "EMAIL"
                        ],
                        "signupAttributes": [
                            "EMAIL"
                        ],
                        "passwordProtectionSettings": {
                            "passwordPolicyMinLength": 8,
                            "passwordPolicyCharacters": []
                        },
                        "mfaConfiguration": "OFF",
                        "mfaTypes": [
                            "SMS"
                        ],
                        "verificationMechanisms": [
                            "EMAIL"
                        ]
                    }
                }
            }
        }
    }
}

Additional information and screenshots

No response

tylerjroach commented 10 months ago

Please ensure you are still authenticated. Update to the latest version of liveness and let me know what error you receive in the onError callback. It looks like you are launching into a LivenessActivity so this is going directly into native and would not be any different from the native only experience.

naimroslan commented 10 months ago

I've tried run amplify init and amplify add auth in react native's android folder, failed. I've tried run amplify init and amplify add auth in react native root project, same, failed. The error just shows that it failed to start the face liveness session.

Test case: Setup amplify in RN root project

Run amplify init in react native root project

Project information
| Name: Northlight
| Environment: dev
| Default editor: Visual Studio Code
| App type: javascript
| Javascript framework: react-native
| Source Directory Path: src
| Distribution Directory Path: /
| Build Command: npm.cmd run-script build
| Start Command: npm.cmd run-script start

Run amplify add auth
Run amplify push

Test case: Setup amplify in RN's android project

Run amplify init

Project information
| Name: android
| Environment: dev
| Default editor: Visual Studio Code
| App type: android
| Res directory: app/src/main/res

Run amplify add auth
Run amplify push

Error log: log-22nov2023.txt

tylerjroach commented 10 months ago

What you are showing is not enough. While you have added the configuration file to the project, you haven't actually configured Amplify, or the Auth plugin in your code.

You need to figure out if you want to use Amplify Android and Amplify Swift, or if you want to use Amplify JS React library. Whichever option you choose, you need to not just add the amplify configuration to your project, but you need to add code to configure the Auth plugin. Please read up here: https://docs.amplify.aws/android/build-a-backend/auth/set-up-auth/

If you do not pass a custom credentials provider into the FaceLivenessDetector, it will attempt to call Amplify.Auth.fetchAuthSession(). If you choose to use the JS library, you will need to implement a custom credentials provider to pass tokens into the liveness component, since the default Amplify Android Auth plugin will not be available.

naimroslan commented 10 months ago

I see. I used Amplify Android. I called Amplify.configure in Amplify.kt and then I called Amplify() function in onCreate in LivenessActivity.kt

Amplify.kt

class Amplify: Application() {

    override fun onCreate() {
        super.onCreate()

        Amplify.addPlugin(AWSCognitoAuthPlugin())
        Amplify.configure(applicationContext)

        Amplify.Auth.fetchAuthSession(
            { Log.i("AmplifyQuickstart", "Auth session = $it") },
            { error -> Log.e("AmplifyQuickstart","Failed to fetch auth session", error)}
        )

    }

}

LivenessActivity.kt

class LivenessActivity : AppCompatActivity() {

    private val CAMERA_PERMISSION_REQUEST_CODE = 101
    private val region = "ap-northeast-1"
    private val client = OkHttpClient()

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_liveness)

        Amplify()

// rest of the code

naimroslan commented 10 months ago

Hi @tylerjroach

I was wondering if there have been any updates on this issue. Your insights would be much appreciated. Here, I include the steps to reproduce this issue:

Clone this repository git clone https://github.com/naimroslan/React-Native-Face-Liveness.git
Run npm install
Run npm run start then yarn android
Once the app is installed, you can begin the Face Liveness by clicking Start Liveness button

tylerjroach commented 9 months ago

You have Amplify configuration in https://github.com/naimroslan/React-Native-Face-Liveness/blob/initial-commit/android/app/src/main/java/com/northlight/Amplify.kt but you already have a MainApplication class. The MainApplication class is the one defined in the manifest. You can't have 2 Application classes. You need to move Amplify configuration to you MainApplication.

naimroslan commented 9 months ago

I have deleted Amplify.kt and call Amplify.configure from onCreate inside MainApplication instead. However, it still shows the same error

Error during Face Liveness flow
Failed to start the face liveness session.
com.amplifyframework.ui.liveness.model.FaceLivenessDetectionException@ba2a537

and this error

OpenGLRendererJni
Failed to draw frame due to OpenGL error: GL_OUT_OF_MEMORY`

and this is my MainApplication.java

@Override
  public void onCreate() {
    super.onCreate();

    try {
        // Add this line, to include the Auth plugin.
        Amplify.addPlugin(new AWSCognitoAuthPlugin());
        Amplify.configure(getApplicationContext());
        Log.i("Northlight", "Initialized Amplify");

        SoLoader.init(this, /* native exopackage */ false);
        if (BuildConfig.IS_NEW_ARCHITECTURE_ENABLED) {
            // If you opted-in for the New Architecture, we load the native entry point for this app.
            DefaultNewArchitectureEntryPoint.load();
        }
        ReactNativeFlipper.initializeFlipper(this, getReactNativeHost().getReactInstanceManager());
    } catch (AmplifyException error) {
        Log.e("Northlight", "Could not initialize Amplify", error);
    }

  }

and it did log out Initialized Amplify inside the log. I have also tried putting SoLoader.init.. outside of the try-catch block but I still get the same result.

LivenessActivity.kt still the same, now without Amplify()

override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_liveness)

        // camera permission
        if (ContextCompat.checkSelfPermission(this, Manifest.permission.CAMERA) != PackageManager.PERMISSION_GRANTED) {
            ActivityCompat.requestPermissions(this, arrayOf(Manifest.permission.CAMERA), CAMERA_PERMISSION_REQUEST_CODE)
        } else {

            runLambdaFunction("https://o7tx3ew2q4lkz2xg5ccydatc3m0sqypp.lambda-url.ap-northeast-1.on.aws/") { responseBody ->
                Log.i(TAG, "Response: $responseBody")

// rest of the code

I have also pushed and merged a branch where I included res/raw folder so it'd be easier for you (if you would like to give it a try). https://github.com/naimroslan/React-Native-Face-Liveness

Thank you.

naimroslan commented 9 months ago

Its working now! I changed the version of implementation "com.amplifyframework.ui:liveness" from 1.1.3 to 1.1.0.

Thank you for your help!

aws-amplify / amplify-ui-android