search

openzim / youtube

Create a ZIM file from a Youtube channel/username/playlist
GNU General Public License v3.0
46 stars 26 forks source link

test current behavior on all platforms/browsers/readers #84

Closed rgaudin closed 4 years ago

rgaudin commented 4 years ago

I'd like to suggest we run a thorough test of recent zim files on all readers and browsers with current kiwix-serve before making big change like videojs/deps update as it's easy to break things on specific readers/browsers (remember: we patch those libs).

I've noticed some things that were working seems to have difficulties now and since libkiwix changed stuff regarding to videos, we should make sure everything's working as expected first and/or open tickets.

We can split this work across people once we have a methodology (what to test, how)

satyamtg commented 4 years ago

@rgaudin I think that doing the following would be good while testing- Do the following on a zim created by zimfarm and another one manually created using the master branch.

  1. Check if all controls that are shown are actionable and work as expected
  2. Check if the page loads correctly in various sizes (using developer tools for the kiwix-serve versions) If something doesn't work, we may check zims built using previous releases and ensure if it works in a previous version. Also, we should ensure that we have a check on at least 2 devices with the same platform without concluding on anything.
rgaudin commented 4 years ago

zimfarm and non-zimfarm have no reason to produce different output so we should probably select one ZIM produced by a known (latest) version and we can repeat this test after some major change (ideally each release).

here's the matrix I have in mind

Readers/Features Version Platform Platform version Home UI OK Video UI OK VideoJS UI Plays video Has Sound Duration OK Play/pause button Play Pause Fullscreen Seeks Streams OK
Kiwix Android 3.2.1 Android OK OK OK OK OK OK OK OK OK OK OK OK
Kiwix Desktop 2.0.1-1 Windows OK OK OK Sometimes/Sometimes not - Shows corrupted video or using a feature which is not available error (Needs to be tested more) Yes (when video plays) OK (when plays video) OK (when plays video) OK (when plays video) OK (when plays video) OK (when plays video) OK (when plays video) OK (when plays video)
Kiwix Desktop 2.0.1-1 Linux
Kiwix macOS 2.1 macOS OK OK OK OK OK OK OK OK OK Broken Broken OK?
Firefox/kiwix-serve 75.0 Windows OK OK OK OK OK OK OK OK OK OK OK OK
Firefox/kiwix-serve 75.0 macOS OK OK OK OK OK OK OK OK OK OK OK OK
Firefox/kiwix-serve 75.0 Linux OK OK OK OK OK OK OK OK OK OK OK OK
Firefox/kiwix-serve 75.0 Android OK OK OK OK OK OK OK OK OK OK OK OK
Chrome/kiwix-serve 81.0 Windows OK OK OK OK OK OK OK OK OK OK OK OK
Chrome/kiwix-serve 81.0 macOS OK OK OK OK OK OK OK OK OK OK OK OK
Chrome/kiwix-serve 81.0 Linux OK OK OK OK OK OK OK OK OK OK OK OK
Chrome/kiwix-serve 81.0 Android OK OK OK OK OK OK OK OK OK OK OK OK
Safari/kiwix-serve 13.1 macOS OK OK OK OK OK OK OK OK OK Broken video not scaled but on black screen for page video, just black screen without video (but audio continues) on home page Broken OK
Safari/kiwix-serve <800px 13.1 iOS OK OK OK Broken - in-video UI displays but no sound nor video. no error in console Broken OK No No No Broken - moves video to top half of scren No No
Safari/kiwix-serve >800px 13.1 iOS OK OK OK OK OK OK OK OK OK Broken - moves video to top half of scren Broken OK?
Edge/kiwix-serve 81.0 Windows OK OK OK OK OK OK OK OK OK OK OK OK

I think it's worth testing same browser on different platforms as a video relies a lot on the platform but I'm not sure if we should care about Edge as it seems to be based on chrome engine.

rgaudin commented 4 years ago

sound on desktop https://github.com/kiwix/kiwix-desktop/issues/425

satyamtg commented 4 years ago

@rgaudin I have tested on various platforms. Found some errors on Kiwix Desktop on windows

  1. It shows sometimes that video is corrupted or is using some feature which is not available. Screenshot - Annotation 2020-04-24 195748
  2. When playing the video, the Picture-in-picture mode toggle is present but not actionable.

Also, on my network I used kiwix-serve on a Linux (Ubuntu 19.10) PC to serve the ZIM. However, when viewing that from any browser in Windows gave some weird characters at the end of the page. I can confirm this is not an issue with YouTube but with kiwix-serve as ZIMs created by other scrapers also had the same problem in viewing.

For youtube scraper, all I did not re-encoded the videos in low quality to save a bit of time in all my tests. Also, I got a blank screen in Kiwix Desktop on Ubuntu 19.10. I used the appimage version. Can somebody confirm on this?

rgaudin commented 4 years ago

I've also seen many video glitches testing kiwix-desktop on windows (for the other issue). Sometimes it would get really slow, sometimes I would get visual noise, sometime video would freeze but audio continues…

I'm more interested in the kiwix-serve issue if you could elaborate, including kiwix-serve version so we can maybe open a ticket somewhere.

rgaudin commented 4 years ago

@satyamtg what ZIM/format/quality did you use for your tests?

rgaudin commented 4 years ago

Updated with macOS and IOS. We have a regression at the moment. It doesn't work at all on iOS. @kelson42 could you test with your iPad ?

rgaudin commented 4 years ago

Actually works (plays, pause, audio) in landscape.

rgaudin commented 4 years ago

Updated with macOS and IOS. We have a regression at the moment. It doesn't work at all on iOS. @kelson42 could you test with your iPad ?

It's not a regression actually. This is the same behavior with old and new ZIM files. I was testing with my phone which has a widh < 800px so that's the same old bug there.

satyamtg commented 4 years ago

@satyamtg what ZIM/format/quality did you use for your tests?

I actually scraped my own channel which I created long back using the master branch. For format I used webm and didn't use the low-quality option. For a reference, I'll leave my ZIM file here. Didn't bother to change the name though so it still uses "universcience-tv". https://drive.google.com/file/d/1RatdLf8PaD33JQuKMRyGPmUEgL4Wzgcg/view?usp=sharing

It's not a regression actually. This is the same behavior with old and new ZIM files. I was testing with my phone which has a widh < 800px so that's the same old bug there.

@rgaudin is this happening only with Safari? Because my phone's width is also <800px but it seems to work good on Chrome on Android. I can't test on Safari though as I don't own any MacOS/iOS device.

rgaudin commented 4 years ago

A lot of this seems related to #34 where I posted some information.

rgaudin commented 4 years ago

Thank you @satyamtg for your tests. I've moved the table to the wiki so we can update it over time while keeping past revisions.

I have updated issues and opened new ones to track the problems that we identified. I am thus closing this.