will-moss / erin

Self-hostable TikTok feed for your clips. Make a TikTok feed with your own videos
MIT License
29 stars 1 forks source link

Crashes mobile browsers - works on desktop but the scaling is horrible #1

Closed rursache closed 2 months ago

rursache commented 3 months ago
docker run -d \
  --name erin-tiktok \
  --restart unless-stopped \
  -p 50582:80 \
  -e TZ=Europe/Bucharest \
  -e PUBLIC_URL="http://tt.domain.tld" \
  -e AUTH_ENABLED="false" \
  -e VIRTUAL_HOST="tt.domain.tld" \
  -e VIRTUAL_PORT="80" \
  -v "/long/path/to/my/TikTok/Liked Videos/videos":/srv/videos:ro \
  mosswill/erin:latest

using cloudflare tunnels to bridge http://IP:50582 to https://tt.domain.tld

when opening https://tt.domain.tld on desktop chrome it works (after a 3-4 second delay) and the video plays but its stretched to fill the window width instead of keeping the aspect ratio like tiktok does.

on mobile (iOS) however the page crashes the browser

the videos directory has 5700 files, all with tiktok video ids as filenames (eg. 6630704371711610113.mp4)

i'm not sure if this project was ever tested as its broken on both mobile and desktop :(

will-moss commented 3 months ago

Thanks a lot for your feedback and those details.

Regarding each point :

For the rest, please mind that this project was made (coded, tested, packaged with Docker, and released) in less than 24 hours. This was initially to help somebody quickly. However, I'll definitely address every point you mentioned, and make new releases very soon.

Thanks a lot again!

rursache commented 3 months ago

@will-moss i appreciate your effort and i was not aware it was such a new project

i use docker run not compose and yes i do need to map a port. cloudflare tunnels didn't like the 443 (never had this issue before) so i had to use the non ssl port. these however shouldn't matter, that's the purpose of docker

i really look forward using the project once you push a few fixes!

will-moss commented 2 months ago

Hey!

I've just published a big release with new fixes, features, and improvements.

Regarding your issue :

For the future releases, I am working on a cache system to improve the loading speed for users with many files (like you). It's almost done, but I'm running tests to make sure that it works well.

Let me know if that releases works better for you!

rursache commented 2 months ago

the new release indeed look better on desktop and is usable. however on mobile it still crashes chrome on iOS after loading for a few seconds. on safari there is just an error. i do not care about ogg files but on iOS all browsers are actually webkit so it doesn't matter what you use.

will-moss commented 2 months ago

Thank you very much for your feedback on that new release.

Alright, it seems that I have overlooked something. I will try to spin up an iPhone emulator on my Mac to reproduce the issue, and fix it.

(PS : If there is any chance you could access your mobile's developer console while using Erin, and send me a screenshot/copy-paste of the errors in this thread, it could potentially tremendously help me. Here's how to do it : https://stackoverflow.com/questions/4478271/remote-console-log-on-ios-devices. If you can't, no problem at all, I understand that it may take time on your side. I'll try to do my best with an emulator since I don't own a physical iPhone).

will-moss commented 2 months ago

Hi again,

I am about to release a new version today, with many new bug fixes, improvements, and features.

However, I could finally spin up an IOS emulator on my Mac, and I couldn't reproduce any crash. The only thing that consistently generates a crash is using .ogg files (it seems that, webkit-based or not, there's something going on with Safari anyway : https://caniuse.com/ogg-vorbis).

As a result, and if you have a little time to help, I would like to ask you the following questions to try to include a fix for you in the incoming release :

Also, if that's not too much to ask :

Thanks a lot in advance for your help, and crossing my fingers that we find the culprit together!

rursache commented 2 months ago

The iOS version or the iPhone model does not matter (iOS 17.5.1, 15 pro). The browser crash is due to memory limits as the page seems to fill the ram and the OS will kill the app eventually. When using erin this crash happens in about 3-4 seconds. Not sure you can catch this in the iOS simulator. The phone also gets really hot.

How do I know its a memory issue? In chrome for desktop the erin tab uses 1.3GB of RAM and 14% of the CPU (Intel 14900k, DDR5 RAM). for loading a 956kb mp4 file... screenshot

There are no crashes on desktop on any browser as my system has 64GB RAM and does not "feel" the 1.3 increase from that tab

As mentioned I only have mp4 files (full dump of my liked tiktok videos). The contents of the /srv/videos folder:

6980 .mp4 files amounting for 27GB in total. videos have on avrerage 1.2mb but can be as low as 300kb or as high as 5mb

I also tested with a folder of only 200 videos (random copy from the initial folder). It sits at the loading icon for around 10 seconds and then crashes.

Even a folder with 20 videos wont load. A longer aprox. 20 seconds loading, i manage to see the bottom control bar but before playing the video it crashes

I think there is a big memory management issue + cpu usage issue. the web app should only load the current video and maybe the next 2-3 ones. I feel that 50% of the videos are loaded. also i would not include caddy as it made everything so complicated. let the user deal with ssl. I for one use cloudflare tunnels so i had to complicate my docker cli command with 3 more env values to get it working and that after some trial and error runs. there is no need for ssl to stream mp4 videos..

Happy to share with you my instance URL if you can provide a direct contact form or you @ me somewhere!

will-moss commented 2 months ago

Alright, thank you a thousand times for these precious details and explanations.

I have just published a new release, does it work better on your end?

Here's a link to an instance where I have deployed it : https://shorturl.at/xxxxx The password is : xxxxxx

I am not 100% sure, but I think a bug in the previous version could cause the memory issues you've described.

For now, on my end, the Erin tab always takes between 150 Mo and 225 Mo in my browser. I can't get anything to crash, either on mobile or on desktop or on emulated mobile. Also, my setup is made of 210 videos, amounting to ~500 Mo.

Also, the lazy-loading mechanism already exists, and existed even before (but may have been broken by a bug in the previous version). No more than 3 videos are ever loaded at once, and on every scroll, only one additional video is fetched at a time. A way to check that is to open the Network tab in the Developer Tools of the browser, and see that (in the Media sub-category), only 2 medias are loaded at start, and then 1 extra on every scroll.

If the issue persists on your end (even when connecting to my instance), then I have no idea what could be going on, especially since you've described the crashes occurring even with 5 and 20 videos, which, even if loaded all at once, shouldn't cause a crash I believe.

Let me know, and thanks a lot once again for helping me.

rursache commented 2 months ago

The new update fixed it on iOS! I can finally use it on mobile (the entire video folder)

One more thing would be a visible delay between swiping through videos (black screen): https://youtube.com/shorts/ZJQuUW_yRLE

Thanks!

will-moss commented 2 months ago

Good lord! Thanks a lot for the quick test, reply, and video. I'm glad it's fixed.

Working on the delay you've mentioned now.

Thanks again!

will-moss commented 2 months ago

Just released a new version, hoping that it works better for you.

More videos are loaded now (the current one, plus four in the background), and a "preload" hint is given to the browser to try to speed things up further.

Other than that, I have checked and it seems that the header Range is already used to load videos in chunks, which should be optimal.

Ultimately, after doing some research, other ideas for improvement I could see were :

Otherwise, maybe implementing a Youtube-like loading system (generate different versions of the video, 240p, 360p, 480p, etc.) would work, but I'm afraid it would be hard to implement in the current architecture.

rursache commented 2 months ago

There is still a big delay on swipes, tapping the bottom buttons (mute/download/eye) (aprox 1-3 seconds). My network has nothing to do with it and I dont want to alter the downloaded videos in any way. The phone is still getting really hot while using erin. This + the "lag/delay" shows that there are still cpu issues. I also feel that the shuffle is not really shuffle, I've seen videos multiple times and on a such a big collection the odds are really really low Anyway thanks for the effort!

will-moss commented 2 months ago

Alright, thanks a lot for your feedback.

I would be happy to help if you can send me a link to your instance at xxxx@xxxx.com

I will try to connect and, at least, see if I can reproduce the CPU issues (which I think are responsible for all the other problems (lags, delays between taps, etc.)).

As for the shuffling algorithm, it was changed just this morning and it should be "okay". Erin uses the Fisher-Yates algorithm.

will-moss commented 2 months ago

Hi again,

I have just published a new release.

As of now, only 3 HTML elements at most are ever rendered on the page, regardless of the number of files in your instance.

The feed was completely rewritten from scratch, and the same 3 HTML elements themselves are reused all the time, so as not to generate unnecessary re-renders of the page.

Crossing my fingers that it fixes the issue.

rursache commented 2 months ago

Amazing work and progress! Everything is so smooth now, no more hangs and the buttons actually respond instantly to the tap!

The only thing i saw scrolling for 2-3 minutes is one video remained black and i had to scroll pass it. Will have to investigate as the mp4 file itself could be corrupted!

You can also test it on my instance!

Thanks for your effort, your webapp will bring much joy to me and my girlfriend!