When I say 'What's one plus one?', the GUI page is displayed but the key is no response

[longjinxiang]

IMPORTANT: Before you create an issue, please take a look at our Issue Reporting Guide.

Briefly summarize your issue:

On my TV, after I say "What's one plus one", it shows a notebook UI. I can't use the Remote control to exit, and there is no response before the page disappear automatically. I use linux's 'top' command, and I find it costs 200% CPU usage. Other pages (likes 'What's the weather in New York' ) no problem

After I update to v2.7.1(https://github.com/alexa/alexa-smart-screen-sdk/tree/v2.7.1/modules/Alexa/APLClientLibrary/apl-client-js), when I say "what's one plus one? ", I can't get any response of remote control.

I return to v2.3.0(https://github.com/alexa/alexa-smart-screen-sdk/blob/v2.3.0/modules/Alexa/APLClientLibrary/apl-client-js/index.js), it's ok!

I think some error updated before v2.7.1

What is the expected behavior?

I say "What's one plus one", I can't use the Remote control to exit.

What behavior are you observing?

I say "What's one plus one", I can use the Remote control to exit.

Provide the steps to reproduce the issue, if applicable:

Say 'What's one plus one?' to Alexa, after UI updated, press any key to test

Tell us about your environment:

What version of the AVS Device SDK are you using?

  <2.7.1>

Tell us what hardware you're using:

• [ ] Desktop / Laptop
• [ ] Raspberry Pi
• [x] Other - tell us more:

Tell us about your OS (Type & version):

• [x] Linux
• [ ] MacOS
• [ ] Raspbian Stretch
• [ ] Raspbian Jessy
• [ ] Other - tell us more:

Have you tried the same use case with AVS Device SDK SampleApp?

• [x] The issue is not applicable to the AVS Device SDK SampleApp.
• [x] The issue is not reproducible on the AVS Device SDK SampleApp.

[longjinxiang]

hope your answer

[longjinxiang]

This error come from bc5c086f3e035b72bc9eef636c806bd5cd427089, but I don't know how to fix it. My project base on v2.7.1

[1cengizk]

Hi @longjinxiang

Thanks for reaching out. If possible, can you upload a GUI log file for the failure similar to the one here?

[longjinxiang]

Yes, log and the video.

alexablockLog.log

https://user-images.githubusercontent.com/35523756/147798157-ad64e65a-c119-4037-87e7-5df3fdb0c793.mp4

This problem seems to block the alexa certification.

[longjinxiang]

GUI responds to key presses after this page disappears.

[1cengizk]

@longjinxiang Ideally, pressing the EXIT key on your remote should generate a navigation event in GUI that should be then propagated down to the SDK. I can see the EXIT key request in the GUI logs you provided; but, still need more info to understand where the problem is.

1) Can you collect GUI logs with all log levels enabled in console?

2) Can you also collect DEBUG9 logs from the SmartScreenSDK. I am wondering if the EXIT key press triggers the navigation event at all within the SDK. To collect these DEBUG9 logs; start the SmartScreenSDK as follows:

modules/Alexa/SampleApp/src/SampleApp -C <path-to-AlexaClientSDKConfig.json-file> -C <path-to-SmartScreenSDKConfig.json-file> -L DEBUG9

[longjinxiang]

Log with DEBUG9 whats one plus one Log.txt

[1cengizk]

Log with DEBUG9 whats one plus one Log.txt

@longjinxiang Could you describe what is the test case here? Are you hitting the EXIT key on the remote immediately after asking for 1+1 or wait for a while?

I can see the following in the logs:

2022-01-04 03:51:24.514 [ 3] 9 GUIClient:onMessageInExector:payload={"type"\:"aplEvent"\,"windowId"\:"tvFullscreen"\,"payload"\:{"type"\:"localeMethod"\,"seqno"\:3\,"payload"\:{"value"\:"WHAT'S ONE PLUS ONE"}}}

Then about 30 seconds later:

sh-3.2# exit key Following this, I can see the following functions in the log:

2022-01-04 03:51:56.364 [ 1e] 9 GUIClient:onMessageInExector:payload={"type"\:"navigationEvent"\,"event"\:"EXIT"}
...
2022-01-04 03:51:56.432 [ 2a] 5 SmartScreenClient:forceExit::Force Exit
...
2022-01-04 03:51:56.432 [ 2a] 5 AlexaPresentation:clearCard
...
2022-01-04 03:51:56.767 [ 14] 5 AlexaPresentation:executeClearCardEvent:sourceId=0

So, the exit functions are invoked; but, 30 seconds after the initial request. If you are pressing the EXIT key on the remote within that 30-second period, it doesn't seem to reach the SDK.

Could you add the following line here and see if that appears in the browser console logs at all?

this.logger.info('EXIT button pressed');

[longjinxiang]

test case： what's one plus one? After UI render, EXIT key is not response，I can't exit page, and it costs too much CPU usage.

Is add there?

[1cengizk]

test case： what's one plus one?

After UI render, EXIT key is not response，I can't exit page, and it costs too much CPU usage.

Is add there?

Yes, you can add the log statement there.

[longjinxiang]

The method (logger.info) does not seem to be implemented. I use "console.log('[long] EXIT button pressed')" instead alexaSDKconsole.log

[longjinxiang]

Could you reduce the CPU usage?

[1cengizk]

@longjinxiang Did a little bit more triage on this issue.

I am not able to reproduce this issue locally using SmartScreenSDK-2.7.1 with TvFullScreen as the reference device. The notebook appears in the overlay; not in fullscreen. Which reference device profile are you using with SmartScreenSDK, is it HubLandscape maybe?

[longjinxiang]

This problem seems to be related to my TV browser, which is based on the chrome kernel version 73, but it can only run on the TV. Could you tell me how to build ‘what's one plus one’ page and /modules/Alexa/APLClientLibrary/apl-client-js/index.js?

and ‘what's the price of BTC’ page has the same problem, do they use the same page?

[longjinxiang]

Composite Layers costs most time https://drive.google.com/file/d/1IIZP1N-s6Mqq_hMeztzHq_MQBEFlhcs1/view?usp=sharing

[mdaix-az]

Built SmartScreenSDK 2.7.1 locally with Sample GUI Config GuiConfigSample_SmartScreenLargeLandscape, was able to see a full screen "one plus one" page and EXIT navigation event was able to dismiss it.

According to the logs you shared earlier, the "one plue one" page was trying to present on your device in a window with ID "tvFullscreen". Could you also share your GUI config file for further investigation?

[longjinxiang]

Is this? SmartScreenSDKConfig.zip

[longjinxiang]

When I opened the file on the TV, the problem reappeared. But I don't know how to modify this html structure since it is from the compressed index.js file.

testIndex.zip

Related websocket information（After this message， react get a DOM element） { token: "amzn1.as-tt.v1.Domain:Application:Knowledge#TID#9ba60136-1d40-4189-94d5-261168606625", type: "aplRender", windowId: "smartScreenLandscape" }

[mdaix-az]

The GUI config file looks correct to me. Since we are unable to reproduce this issue using the same SDK version and configuration your device is using, we are suspecting the issue is caused by the exit navigation event is not invoked correctly or is not handled correctly.

When a navigation event is initiated from the GUI side, if it has reached the SDK, following logs will be printed in order:

1. GUIManager.cpp#L554
2. SmartScreenClient.cpp#L1400

According to the DEBUG9 log you shared earlier, above logs are not printed until 30+ seconds after receiving the "One plus one" message.

Could you help us confirm: If you press the exit button as soon as you see the "one plus one" page, do you see the above two logs being printed immediately? If not maybe something went wrong in the process then we can go from there.

[longjinxiang]

I have something urgent today, I'll look at it tomorrow

[longjinxiang]

I found that the source of the problem was caused by this sentence, which seems redundant and not compatible with my browser.

[longjinxiang]

This is the log of pressing exit when the page comes up, and doesn't have the print you need. I found out that this is due to a GUI HTML element, could you help me to modify the html structure or tell me how to modify it?

exit2.log

[rpalkar-amzn]

Hi @longjinxiang

With Smart Screen SDK v2.8, the apl-client-js has been moved to an independent repository: https://github.com/alexa/apl-client-library. This repository would be apt for requesting any future improvements related to APL Client. For context, the HTML structure for APL experiences is generated by apl-client-js.

1. Regarding the exit button issue, is there a way to identify the exit key press in the system log? Is the key press not working only because of the CPU usage and system isn't able to respond to any key press?
2. Also, are you planning on moving forward with Smart Screen SDK v2.7 or able to upgrade to a later version?
3. Would it possible to check if you observe same issue with apl-client-js v1.7 or later available here: https://github.com/alexa/apl-client-library/tags?

[longjinxiang]

My TV will not be upgraded to a later version.

These prints indicate that the system received the Exit key and sent it to the UI, but the UI did not respond

[RW_WEBUI_AGENT]</home/raid/yixiaoyan/MLR53/TEST/apollo53/apollo/linux_mts/rpc/broadcast/rpc_wrapper/mtktvapi/rw_mtktvapi_webui_agent.c: 144> <_hndlr_a_mtktvapi_webui_agent_browser_cmd_notify: 144>

[rpalkar-amzn]

Hi @longjinxiang

Thank you for the reply.

Can you please add another event handler on keydown event in the UI application?

document.addEventListener(
    'keydown', 
    (event) => { 
        console.log('key pressed: ' + event.KeyCode); 
        this.logger.info('key pressed: ' + event.KeyCode);
    }
)

Please ensure that the logger is initialized with log level INFO as done here. Using this.logger.info will enable getting UI app logs on the native side.

Do these logs get printed when you press key for the "what's one plus one" experience? Please share the log file as well. If they aren't printed, could it be the case that the UI application platform is not responding as expected due to high CPU usage?

[longjinxiang]

log.txt

[rpalkar-amzn]

Below are observations from the attached log file:

1. The first keydown event log is printed only after the Dismissed event is sent. Dismissed event is sent when the APL UI is cleared. This happens nearly 35 seconds after the message to render APL aplRender is sent from the C++ App to the UI App.

[SRAF_LOG][1350:1350:20220114/081308.098:console.log:INFO:sraf_render_frame_observer.cc(94):DetailedConsoleMessageAdded()] [file:///basic/webui/alexaGUI/main.bundle.js(2)]: "[jinkexin debug] message WW: {
        "type": "aplRender",
        "windowId": "tvFullscreen",
        "token": "..."
}"

2022-01-14 10:13:43.260 [  d] 0 EventBuilder:buildJsonEventString:messageId=...,namespace=Alexa.Presentation,name=Dismissed

[SRAF_LOG][1350:1350:20220114/081343.960:console.log:INFO:sraf_render_frame_observer.cc(94):DetailedConsoleMessageAdded()] [file:///basic/webui/alexaGUI/main.bundle.js(2)]: "[jinkexin debug] keydown event code: Escape keyCode: 27"

If any keys were pressed within those 35 seconds, corresponding keydown events aren't being triggered on the UI platform to be handled. As a result, the keydown event handlers aren't triggered. I suspect this could be because of the high CPU usage that you previously mentioned.

2. The logs attached do not have DEBUG9 level logging. While running the C++ Application, please specify the log level -L DEBUG9 for more detailed logs.

3. There are warning messages related to APLCoreEngine too and the componentId potentially relates to the HTML structure being rendered. For this and requests related to CPU usage optimization in APL rendering, I'd recommend reaching out to apl-client-library

2022-01-14 10:13:10.391 [ 19] W AplClientBridge:AplCoreEngine::corecom[   83.316284] <MI3_ERR>MI_TUNER_ResetEwbsAreaCode[9095]: failed to MApi_DigiTuner_ResetIsdbtEwbsAreaCode(0, 0)
mand.cpp:calculateProperties : Illegal command - need to specify a target componentId
2022-01-14 10:13:10.444 [ 19] W AplClientBridge:AplCoreEngine::corecommand.cpp:calculateProperties : Illegal command - need to specify a target componentId
[SRAF_LOG][1350:1350:20220114/081310.451:console.log:INFO:sraf_render_frame_observer.cc(94):DetailedConsoleMessageAdded()] [file:///basic/webui/alexaGUI/main.bundle.js(2)]: "[jinkexin debug] message WW: {

[longjinxiang]

My browser does not support the existence of these two elements at the same time, so I removed the filter effect and the problem was solved