Chromecast performance get severely slow cycling through HTML content

[GoogleCodeExporter]

Background:
In our app, we require the swapping out HTML element to load different 
features. Ex. Cycling through Google Maps, YouTube, and Calendar. However, 
within the first 10 to 15 minutes the Chromecast performance degrades to the 
point where it either hangs or crashes. Basically the user can’t use our app 
for more than 10 minutes which is impacting our entire experience.

What steps will reproduce the problem?
I’ve created a very simple webpage at 
http://www.scarlatt.com/performance_test.html to reproduce the issue. Simply 
Cast this page on Chromecast using an app.  This page basically cycles through 
YouTube (using embedded iframe) and Google maps (also inside an iframe) every 
30 seconds.

What is the expected output? What do you see instead?
Expected behavior: The page should continue to cycle through YouTube and Maps 
non-stop or for a very long time (hours at least).
Actual behavior: When this page is loaded on Chromecast you’ll notice that 
the performance starts to degrade at around 10 minutes. After that Chromecast 
basically hangs, neither YouTube or Google Maps load.

What version of the product are you using? On what operating system?
We’ve verified this issue on both Chromecast V1 and V2. 
The firmware for Chromecast V2 is 1.16.44433
The firmware for Chromecast V1 is 1.7.46278

Please provide any additional information below.
Since profiling doesn’t work well on Chromecast we haven’t been able to get 
it’s performance results. However we’ve profiled on the Chrome browser and 
everything seems stable and runs not stop.

Attached the test html file pointed to by the url above.

Original issue reported on code.google.com by hir...@gmail.com on 24 Nov 2015 at 8:46

Attachments:

• performance_test.html

[GoogleCodeExporter]

Minor correction, the Chromecast V1 firmware version is 1.17.46278

Original comment by hir...@gmail.com on 24 Nov 2015 at 9:42

[GoogleCodeExporter]

Issue 693 has been merged into this issue.

Original comment by na...@google.com on 24 Nov 2015 at 6:42

[GoogleCodeExporter]

Please provide below information:
1) When did you start seeing this issue? 
2) Is it reproducible on both Android and iOS?
3) What type of receiver are you using? 
4) Please reproduce this issue and share the receiver logs. 

Original comment by na...@google.com on 24 Nov 2015 at 6:48

• Changed state: NeedInfo

[GoogleCodeExporter]

Here's the additional information:
1) When did you start seeing this issue? 
- We started noticing the performance issues several weeks ago but we were 
still adding features. Then last week after feature complete we spent a good 
amount of time narrowing down the issue and trying to work around it without 
luck.

2) Is it reproducible on both Android and iOS?
- Currently we are only making an android app. The performance issue seems to 
be a Chromecast issue, unrelated to the Android client.

3) What type of receiver are you using? 
- Custom receiver

4) Please reproduce this issue and share the receiver logs. 
- Attached receiver console logs. This time the app on Chromecast screen only 
runs for a few minutes before stalling, probably due to the remote debugging 
connection. After the Chromecast hangs, sometime it restarts or goes back to 
the default home screen.

Original comment by hir...@gmail.com on 25 Nov 2015 at 12:08

Attachments:

• Chromecast_logs.txt

[GoogleCodeExporter]

Thanks for the info. Is this app in production? If so, please share the Play 
Store link for it. 

Original comment by na...@google.com on 26 Nov 2015 at 12:13

[GoogleCodeExporter]

Thanks for looking into it. 

The app is not yet in production. If you have a test app to cast the URL 
(http://www.scarlatt.com/performance_test.html) it should result in the same 
behavior. Let me know if you do not have such a way to try that.

Original comment by hir...@gmail.com on 26 Nov 2015 at 12:19

[GoogleCodeExporter]

The status is still marked as "NeedInfo". I'm wondering what other info is 
needed. This issue is blocking us from releasing the app. Any way to up the 
priority to Critical?

Original comment by hir...@gmail.com on 1 Dec 2015 at 11:51

[GoogleCodeExporter]

Sorry for the delay here. We are looking into the issue. 

Original comment by na...@google.com on 2 Dec 2015 at 1:08

• Changed state: Acknowledged

[GoogleCodeExporter]

Thanks. Let us know if there is anything we can do to help expedite the 
resolution. Also, do you have an ETA?

Original comment by hi...@getscarlett.com on 7 Dec 2015 at 5:38

[GoogleCodeExporter]

We can not reproduce this issue. Please check your CDN for any issues. 

Original comment by na...@google.com on 11 Dec 2015 at 8:58

• Changed state: NotSDK

[GoogleCodeExporter]

I'm doing something very similar with a custom receiver... I've tried removing 
as many variables as I can but there seems to be some sort of leak happening 
that, as the OP mentions, degrades performance over time until it becomes 
unresponsive. In my case, I am cycling through embedded YouTube videos with a 
scrolling marquee underneath. It works great for 5ish minutes, then degrades 
quickly. At first I suspected maybe the YouTube iFrame API/SDK has some sort 
leak, but I'm not showing any significant leaks in the debugger, just a 
degradation in performance. I'm still trying to see if I can isolate more 
specifically what's going on, possibly thermal / cooling issues perhaps causing 
CPU throttling? I'm running low on ideas at this point. I've tried completely 
destroying and recreating the embedded YouTube player between videos without 
any improvement. I just wanted to add another anecdotal experience, because I 
think there's definitely something weird happening and I've done my best to 
eliminate all possible factors in my code. Next up is going to try actively 
cooling the Chromecast while running my custom receiver to see if that's the 
problem.

Interestingly, the video will sometimes freeze while the marquee continues just 
fine, which makes me wonder if it's a hardware video decoder issue? Although, 
less often, *everything* will lock up, and the Chromecast completely freezes 
(sometimes flickering whatever was on the screen at that point) and stops 
responding to the sender application and also stops showing up as an available 
device at all (CPU too overworked to send/receive broadcasts?)... In these 
cases, I have to hard reboot the Chromecast, though sometimes the Chromecast 
will eventually give up and go back to the idle screen and show up again. I'll 
update here if I narrow down the issue any further. Googling and searching 
stackoverflow shows quite a number of very, very, very similar experiences, so 
it feels a little premature to mark this "NotSDK" in my opinion.

Original comment by evanat...@gmail.com on 13 Dec 2015 at 12:26

[GoogleCodeExporter]

So for the sake of ruling out thermal issues, I just tried running my custom 
receiver with the Chromecast sitting on an insulated ice pack. Unfortunately, 
the issues are persisting, even while keeping the Chromecast nice and cool.

I'm not really sure where to go from here in debugging this. It's not the CDN 
or WiFI...

1. The source of the videos are YouTube.

2. I've tried this on 1st gen and 2nd gen Chromecasts, it affects both.

3. I've tried from various ISP's with generous bandwidth.

4. It happens regardless of 2.4ghz and 5ghz networks (even with no neighboring 
interference).

5. The receiver runs perfectly fine on desktop Chrome for days at a time 
without any issues.

I'll continue debugging, but it's been 2+ weeks of fighting this, and I'm 
running out of ideas. It's almost as if the Chromecast has a bug with garbage 
collection on custom receivers, or there's some sort of leak involved with the 
YouTube Iframe API that is only showing up with the Chromecast's limited 
resources. *shrug*

I'd really appreciate a hand with this from someone at Google — it doesn't 
seem like I'm the only one struggling with this, and my project is at a 
standstill right now until we figure this out. I can't publish the URL to my 
custom receiver here, but I'm more than happy to share it privately via 
email/hangouts any time.

Original comment by evanat...@gmail.com on 13 Dec 2015 at 1:39

[GoogleCodeExporter]

Worth noting that the common theme here seems to be playing YouTube videos on 
custom receivers via an embedded iframe — or at least the use of iframes. 
Makes me wonder if perhaps the Chrome build on Chromecast has some sort of 
garbage collection issue with iframes specifically? No clue, that's just a shot 
in the dark. Apologies for the comment spam — just trying to provide as much 
information as I can to hopefully get this resolved.

Original comment by evanat...@gmail.com on 13 Dec 2015 at 1:46

[GoogleCodeExporter]

Hi Google team, when you say it can't be reproduced:
- How was the issue tested?
- How long was it tested for? As stated in the original bug, watch the behavior 
after 10 minutes. And please keep watching the performance for another 5 to 10 
minutes to get a clear repro.
- What versions of the Chromecast was this tested on?

Four developers on my team have reproduced this bug across at least 6 different 
Chromecasts (including both 1st and 2nd generation) on several different 
wireless networks. Basically it reproduces every time we try, from any network 
and on any Chromecast. It doesn't seem to be network related as Chromecast gets 
super hot in the process likely due to performance issues. Plus, the content 
that we’re loading to reproduce the issue are both from Google properties (a 
Youtube video and Google Maps) which I do not believe runs into any CDN issues 
especially when this works totally fine on a desktop browser.

We've been consistently running into this issue for weeks across our entire 
team, so I'm very curious how this isn't reproducing for you.

Original comment by hir...@gmail.com on 13 Dec 2015 at 6:48

[GoogleCodeExporter]

Also, to re-emphasize this is a show stopper bug for us for a very important 
product. We are local near Mountain View and can try to work with the Google 
team in-person to reproduce the issue. Please email me directly to arrange that.

Original comment by hir...@gmail.com on 13 Dec 2015 at 6:52

[GoogleCodeExporter]

@evanatmtd: "the video will sometimes freeze while the marquee continues just 
fine" - could you please share the logs after reproducing this behavior? Also, 
if your app is in production, please share Feedback report from the Chromecast 
app with the text "for Issue 692" in the description. 

Original comment by na...@google.com on 14 Dec 2015 at 8:02

[GoogleCodeExporter]

@hirals: We have tried reproducing this issue with variations - on both 
Chromecast v1 and v2 having the said firmware - yet we aren't able to repro it. 
The stream seems to be maintaining higher quality (tested through 15 mins 
mark). 
Although, we can gladly revisit this issue. Could you please share sender side 
logs? 

Original comment by na...@google.com on 14 Dec 2015 at 8:22

• Changed state: NeedInfo

[GoogleCodeExporter]

@na...:  couple of questions:

1) When you say sender side logs, I assume you mean the logs on the Android app 
and are referring to the logcat logs. Please let me know if that’s the case 
and what level or settings I should turn on to make sure you get the logs that 
you’ll need.

2) I believe you have the receiver logs from before, and that was done by 
enabling debug logging (cast.receiver.LoggerLevel.DEBUG) in the receiver app. 
Let me know if there were additional logs you needed here.

Original comment by hir...@gmail.com on 14 Dec 2015 at 9:26

[GoogleCodeExporter]

Receiver logs you have shared seems good and yes, by sender side I mean the 
logs from the Android app since it is not in prod yet, you could reproduce the 
issue with your development apk and attach the relevant or full log file but 
please make sure that sensitive info is hidden. 

Original comment by na...@google.com on 14 Dec 2015 at 9:45

[GoogleCodeExporter]

@na... Thanks for the fast response! I'll be working on it today and will get 
sender and receiver logs for as many scenarios as I can. (PS: Our sender is a 
Chrome sender app, not Android... yet.)

The app is in production in the sense that the URL's are publicly accessible, 
but I'd prefer not to post them publicly here. I'll share a feedback report 
with the text "For issue 692" -- does that give you the URL's to the 
sender/receiver? I'm fine with sharing them with the Chromecast team in private 
to help with debugging.

Original comment by evanat...@gmail.com on 14 Dec 2015 at 10:25

[GoogleCodeExporter]

For #20: You could hide out the information you don't want to share publicly 
from receiver logs. We don't require the URL to the receiver; we usually ask 
for sender info to be able to reproduce the issue. Although, please let us know 
if this is a default or custom receiver.
Sharing feedback report however, is the best way to send the debugging info to 
the Chromecast team.

Original comment by na...@google.com on 14 Dec 2015 at 10:50

[GoogleCodeExporter]

Re #21: It's a custom receiver. I'm working on getting those logs / feedback 
reports and a stripped down version that reproduces the issue reliably that I 
can share. I'll post both soon hopefully.

Original comment by evanat...@gmail.com on 14 Dec 2015 at 11:39

[GoogleCodeExporter]

Re #19, let me create a stripped down version of the app or use one of the 
sample android apps to reproduce the issue and send those logs. Thanks!

Original comment by hir...@gmail.com on 14 Dec 2015 at 11:51

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Note: The Chromecast app ID associated with that controller/receiver I posted 
is not published yet, but I'm assuming you guys can still play it internally? 
If not, I can temporarily publish it.

Original comment by evanat...@gmail.com on 15 Dec 2015 at 12:11

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Re. #19, we've reproduced the issues using the sample Chromecast app 
"CastHelloText-android-master". The only things changed from the sample app 
were the app_id and namespace in strings.xml file in order to point it our our 
test page which is now located at 
https://getscarlett.com/static/test/performance_test.html. This is still the 
same test page as before but behind https endpoint so that we can publish the 
chromecast app for your testing.

Here's the information attached:
* (CastHelloText-android-master-debug.apk) - The CastHelloText-android-master 
APK with the app_id and namespace modifications. You can use this to reproduce 
the issue.
* (Sender_logs_no_remote_debugging.txt) - Contains sender logs WITH NO remote 
debugging session connected on the receiver.
* (Behaviour observed.xlsx) - Behavior observed from running the test over the 
course of 18 minutes until the Chromecast crashed WITH NO remote debugging 
session.

The test setup:
* Chromecast generation 2
* Firmware 1.17.48342
* The Chromecast device was power cycled before starting the test.

Additional notes:
In case you power cycle the Chromecast or somehow see a slightly improved 
performance (perhaps due to Firmware upgrades), please give it additional 5 
minutes to see the performance issue. Meaning let the actual clock time go past 
15 minutes, then watch the behavior over the next 5 to 10 minutes.

Original comment by hir...@gmail.com on 15 Dec 2015 at 6:20

Attachments:

• CastHelloText-android-master-debug.apk
• Sender_logs_no_remote_debugging.txt
• [Behaviour observed.xlsx](https://storage.googleapis.com/google-code-attachments/google-cast-sdk/issue-692/comment-27/Behaviour observed.xlsx)

[GoogleCodeExporter]

Thank you for sharing required info. We are looking into it.

Original comment by na...@google.com on 15 Dec 2015 at 9:22

• Changed state: Acknowledged

[GoogleCodeExporter]

Hello Google team, any update on this?

Original comment by hir...@gmail.com on 15 Jan 2016 at 12:49