search

Dash-Industry-Forum / dash.js

A reference client implementation for the playback of MPEG DASH via Javascript and compliant browsers.
http://reference.dashif.org/dash.js/nightly/samples/dash-if-reference-player/index.html
Other
5.11k stars 1.67k forks source link

BrowserStack test failures increased #1222

Closed LloydW93 closed 8 years ago

LloydW93 commented 8 years ago

For the last couple of runs at http://smp-scratch.s3-website-eu-west-1.amazonaws.com/browserstack/, the majority of tests have failed (Firefox is relatively happy, everything else isn't even playing back most streams). Is this related to one of the issues related in the last two days (the first run that failed would have been around 8am UTC on the 24th)?

dsparacio commented 8 years ago

Is this still an issue @LloydW93 ?

LloydW93 commented 8 years ago

Yep - just take a look at http://smp-scratch.s3-website-eu-west-1.amazonaws.com/browserstack/ for a daily summary. The first number in brackets is the number of passes, and the second is the total number of tests run.

dsparacio commented 8 years ago

that is pretty ambiguous any reasoning for the failures?

bwidtmann commented 8 years ago

I don't get it. Where are these tests defined? In a BBC private repo? Or on DASH-IF side? Who is maintaining the jenkins?

JibberJim commented 8 years ago

Lloyd did it yep, Info is here:

https://groups.google.com/d/msg/dashjs/PToS_KTg7JY/CdG0PC5YAwAJ

the tests are, contributions welcome! https://github.com/bbc/dash.js-browserstack

And the Jenkins is ours yes.

On Sat, Mar 12, 2016 at 1:26 PM, Bernhard Widtmann <notifications@github.com

wrote:

I don't get it. Where are these tests defined? In a BBC private repo? Or on DASH-IF side? Who is maintaining the jenkins?

— Reply to this email directly or view it on GitHub https://github.com/Dash-Industry-Forum/dash.js/issues/1222#issuecomment-195742304 .

bwidtmann commented 8 years ago

Great! I want to extend our integration tests. maybe we should share the same code base and I should contribute to yours? The only thing I am missing is throttling browser bandwidth in order to test ABR. We are using browsermob proxy to achieve this.

LloydW93 commented 8 years ago

I believe @AkamaiDASH had plans to take this repo as a base to try to do something similar natively - they don't test the BBC's integration, just that a range of MPDs play against a nightly (well, 8am) build. I'd welcome any input on them as they're very much something I put together quickly. From a quick Google, it looks like BrowserStack doesn't support bandwidth throttling, so I don't think it would be possible :(

dsparacio commented 8 years ago

We built something very raw over the summer with bash, webdrivers, ipfw etc but need to refine it. We will most likely work on something like ABR testing when Kevin is back as intern this summer.

davemevans commented 8 years ago

Normal service seems to have resumed:

jenkins-smp.browserstack.dashjs.upstream-106 163/182
jenkins-smp.browserstack.dashjs.upstream-105 161/182
jenkins-smp.browserstack.dashjs.upstream-104 156/182
jenkins-smp.browserstack.dashjs.upstream-103 62/182
jenkins-smp.browserstack.dashjs.upstream-102 51/182
jenkins-smp.browserstack.dashjs.upstream-101 50/182
dsparacio commented 8 years ago

Just saw that dave. Going to close this for now. We can reopen if pops up again.

jenkins-smp.browserstack.dashjs.upstream-106 163/182 jenkins-smp.browserstack.dashjs.upstream-105 161/182 jenkins-smp.browserstack.dashjs.upstream-104 156/182 jenkins-smp.browserstack.dashjs.upstream-103 62/182 jenkins-smp.browserstack.dashjs.upstream-102 51/182 jenkins-smp.browserstack.dashjs.upstream-101 50/182