Error: failure(FaceLiveness.FaceLivenessDetectionError(code: 0, message: "An unknown error occurred.", recoverySuggestion: "Please open an issue...."))

[lendk-dua]

Describe the bug

Hi Team,

We are using the Amplify SDK (AmplifyUiLiveness) version 1.2.16 and Amplify version 2.35.4. However we have encountered an issue where once the camera is initialized it does show the correct message - Move face in front of camera (when no face is in camera). As soon as I start scanning my face an error is returned as following.

failure(FaceLiveness.FaceLivenessDetectionError(code: 0, message: "An unknown error occurred.", recoverySuggestion: "Please open an issue...."))

I have attached the code sample as a screenshot.

Note: We are using custom credentials. We have tried to create a React sample project using the SDK and it seems to run perfectly fine with the same API endpoints used in our iOS App.

We believe that there might be an issue with the SDK. I have also tried to use an older version (1.2.9) but the issue still persists

Swift

React

Steps To Reproduce

Steps to reproduce the behavior:
1. Create new swift UI CustomView 
2. Add FaceLivenessDetectorView in body
3. Add session Id
4. Add credential provider, including accessKey, secretAccessKey, expiration and sessionToken
5. Build app
6. Scan Face

Expected behavior

The camera feed should show all steps to scan face. Instead it is failing as soon as face is position within the camera frame.

Swift Liveness Version

1.2.16

Xcode version

15.4

Relevant log output

failure(FaceLiveness.FaceLivenessDetectionError(code: 0, message: "An unknown error occurred.", recoverySuggestion: "Please open an issue...."))

Is this a regression?

Yes

Regression additional context

No response

OS Version

iOS 17.5.1

Device

iPhone 14 Plus

Specific to simulators

No response

Additional context

No response

[harsh62]

@lendk-dua Would you be able to provide verbose logs when this issue is happening? You can enable verbose logging to the console by doing the following before initializing the liveness view:

Amplify.Logging.logLevel = .verbose

[lendk-dua]

@harsh62 This is the only extra log printed with logLevel in verbose:

doUnloadModel:options:qos:error:: model=_ANEModel: { modelURL=file:///var/containers/Bundle/Application/22D35D3E-C5B9-410F-90B2-CF5343A50424/dua.app/AmplifyUILiveness_FaceLiveness.bundle/face_detection_short_range.mlmodelc/ : sourceURL= (null) : key={"isegment":1,"inputs":{"input.44_ctx_tx_default__0":{"shape":[8,8,1,96,1]},"input.40_ctx_tx_default__0":{"shape":[16,16,1,88,1]}},"outputs":{"1477":{"shape":[16,896,1,1,1]},"1011":{"shape":[1,896,1,1,1]}}} : identifierSource=0 : cacheURLIdentifier=E04A8B21D3B627656102DCC848F99119A0293BC2749FE59BF32E5F055A75690E_67072AFF8E9C1CF25FB32FD4D148D1CFD9A9EAE4E5B2DB2DD62BC5D9DEC8528B : string_id=0x00000000 : program=_ANEProgramForEvaluation: { programHandle=14709728524660 : intermediateBufferHandle=0 : queueDepth=127 } : state=3 : programHandle=14709728524660 : intermediateBufferHandle=0 : queueDepth=127 : attr={ ANEFModelDescription = { ANEFModelInput16KAlignmentArray = ( ); ANEFModelOutput16KAlignmentArray = ( ); ANEFModelProcedures = ( { ANEFModelInputSymbolIndexArray = ( 0, 1 ); ANEFModelOutputSymbolIndexArray = ( 0, 1 ); ANEFModelProcedureID = 0; } ); kANEFModelInputSymbolsArrayKey = ( "input.40_ctx_tx_default__0", "input.44_ctx_tx_default__0" ); kANEFModelOutputSymbolsArrayKey = ( "1011@output", "1477@output" ); kANEFModelProcedureNameToIDMapKey = { "net_1" = 0; }; }; NetworkStatusList = ( { LiveInputList = ( { BatchStride = 90112; Batches = 1; Channels = 88; Depth = 1; DepthStride = 90112; Height = 16; Interleave = 1; Name = "input.40_ctx_tx_default__0"; PlaneCount = 88; PlaneStride = 1024; RowStride = 64; Symbol = "input.40_ctx_tx_default__0"; Type = Float16; Width = 16; }, { BatchStride = 49152; Batches = 1; Channels = 96; Depth = 1; DepthStride = 49152; Height = 8; Interleave = 1; Name = "input.44_ctx_tx_default__0"; PlaneCount = 96; PlaneStride = 512; RowStride = 64; Symbol = "input.44_ctx_tx_default__0"; Type = Float16; Width = 8; } ); LiveOutputList = ( { BatchStride = 57344; Batches = 1; Channels = 1; Depth = 1; DepthStride = 57344; Height = 896; Interleave = 1; Name = "1011@output"; PlaneCount = 1; PlaneStride = 57344; RowStride = 64; Symbol = "1011@output"; Type = Float16; Width = 1; }, { BatchStride = 57344; Batches = 1; Channels = 1; Depth = 1; DepthStride = 57344; Height = 896; Interleave = 1; Name = "1477@output"; PlaneCount = 1; PlaneStride = 57344; RowStride = 64; Symbol = "1477@output"; Type = Float16; Width = 16; } ); Name = "net_1"; } ); } : perfStatsMask=0} was not loaded by the client. doUnloadModel:options:qos:error:: model=_ANEModel: { modelURL=file:///var/containers/Bundle/Application/22D35D3E-C5B9-410F-90B2-CF5343A50424/dua.app/AmplifyUILiveness_FaceLiveness.bundle/face_detection_short_range.mlmodelc/ : sourceURL= (null) : key={"isegment":0,"inputs":{"image":{"shape":[128,128,1,3,1]}},"outputs":{"input.6":{"shape":[64,64,1,24,1]}}} : identifierSource=0 : cacheURLIdentifier=E04A8B21D3B627656102DCC848F99119A0293BC2749FE59BF32E5F055A75690E_8A291746C8801B05A789F3BB8952C06985CE3DDF0823BA3740A385A08019BFC9 : string_id=0x00000000 : program=_ANEProgramForEvaluation: { programHandle=14709722325900 : intermediateBufferHandle=0 : queueDepth=127 } : state=3 : programHandle=14709722325900 : intermediateBufferHandle=0 : queueDepth=127 : attr={ ANEFModelDescription = { ANEFModelInput16KAlignmentArray = ( ); ANEFModelOutput16KAlignmentArray = ( ); ANEFModelProcedures = ( { ANEFModelInputSymbolIndexArray = ( 0 ); ANEFModelOutputSymbolIndexArray = ( 0 ); ANEFModelProcedureID = 0; } ); kANEFModelInputSymbolsArrayKey = ( image ); kANEFModelOutputSymbolsArrayKey = ( "input.6@output" ); kANEFModelProcedureNameToIDMapKey = { net = 0; }; }; NetworkStatusList = ( { LiveInputList = ( { BatchStride = 98304; Batches = 1; Channels = 3; Depth = 1; DepthStride = 98304; Height = 128; Interleave = 1; Name = image; PlaneCount = 3; PlaneStride = 32768; RowStride = 256; Symbol = image; Type = Float16; Width = 128; } ); LiveOutputList = ( { BatchStride = 196608; Batches = 1; Channels = 24; Depth = 1; DepthStride = 196608; Height = 64; Interleave = 1; Name = "input.6@output"; PlaneCount = 24; PlaneStride = 8192; RowStride = 128; Symbol = "input.6@output"; Type = Float16; Width = 64; } ); Name = net; } ); } : perfStatsMask=0} was not loaded by the client.

Here is a screen-recording in action -> https://we.tl/t-tx4DZqdgf6

[harsh62]

@lendk-dua

The verbose logs were not useful to debug what is happening, would you be able to share some additional details?

• Provide the session ids with region and time associated with when they're created
• verify that the backend permissions are configured correctly for for the IAM user/role to the action "rekognition:StartFaceLivenessSession". https://ui.docs.amplify.aws/swift/connected-components/liveness#update-iam-role-permissions

[lendk-dua]

@harsh62

I have created a quick video so you can see the full projects (both in React and Swift) -> https://youtu.be/oB2nq-EMXdo

Note: After reviewing the video I noticed that I didn't properly paste the sessionToken. This is just for demonstration purposes as we have tried it with multiple different sessionTokens and the same issue persists

Here is the sessionId, region and time sent:

{
    "sessionId": "6be0fe48-adb3-4dde-b302-77376889e7c4",
    "region": "eu-west-1"
}
{
    "accessKeyId": "******",
    "secretAccessKey": "*****",
    "sessionToken": "*****",
    "expiration": "2024-07-12T08:51:29.000Z"
}

[harsh62]

@lendk-dua Thanks for sharing the video, I observed in your video that the session token was copied incorrectly (minute 2:06 to minute 2:08).

I will also share the sessions with my service team to figure out what is happening..

[lendk-dua]

@harsh62

Any update on this?

[github-actions[bot]]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see. If you need more assistance, please open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.

[lendk-dua]

@harsh62

After further investigation we have found that the difference between react and swift is that in react when the websocket is called X-Amz-Security-Token is sent as a parameters, meanwhile this is not sent at all in Swift. We have tested this with Fiddler in our mobile side and here are the results:

This is is iOS

This is in React

Here is the struct I have created for setting my credentials when initializing FaceLivenessDetectorView

[lendk-dua]

I will close this issue as the fix was quiet simple. AWSCredentials protocol does not contain sessionToken, I had to use AWSTemporaryCredentials when creating my Credentials struct and everything seems to work now.

[github-actions[bot]]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see. If you need more assistance, please open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.