WWBN / AVideo

Create Your Own Broadcast Network With AVideo Platform Open-Source. OAVP OVP
https://avideo.tube/AVideo_OpenSource
Other
1.91k stars 972 forks source link

Roku JSON from Program seems to not emit all of the videos into the JSON file #7218

Closed walt93 closed 1 year ago

walt93 commented 2 years ago

Describe the bug I am looking at my Roku channel wondering "why is there missing media?"

I am looking at the program id 74478. It has 18 videos. https://conspyre.tv/program/74478

I am looking at the Roku JSON. It has 3 videos. https://conspyre.tv/roku.json?program_id=74478

To Reproduce Steps to reproduce the behavior:

  1. Go to https://conspyre.tv/program/74478
  2. Observe there are 18 videos
  3. Go to https://conspyre.tv/roku.json?program_id=74478
  4. Observe there are 3 videos

Expected behavior There should be 18 videos in this Roku JSON.

Logs I cleared the cache and generated the Roku JSON. It's tripping on some videos files. No idea what it's tripping on, they're there, I just ran through the playlist and all files play.

[05-Oct-2022 14:38:38 America/Los_Angeles] AVideoLog::DEBUG: clearCache starts  SCRIPT_NAME: /objects/configurationClearCache.json.php
[05-Oct-2022 14:38:40 America/Los_Angeles] AVideoLog::DEBUG: rrmdir: do not delete videos or cache folder /var/www/html/AVideo/videos/cache/ SCRIPT_NAME: /objects/configurationClearCache.json.php
[05-Oct-2022 14:38:40 America/Los_Angeles] AVideoLog::DEBUG: clearCache end in 1.3033330440521 seconds SCRIPT_NAME: /objects/configurationClearCache.json.php
[05-Oct-2022 14:38:40 America/Los_Angeles] AVideoLog::DEBUG: deleteALLCache starts  SCRIPT_NAME: /objects/configurationClearCache.json.php
[05-Oct-2022 14:38:40 America/Los_Angeles] AVideoLog::DEBUG: deleteALLCache rename(/var/www/html/AVideo/videos/cache/YPTObjectCache/, /var/www/html/AVideo/videos/cache/YPTObjectCache633df960433e7)  SCRIPT_NAME: /objects/configurationClearCache.json.php
[05-Oct-2022 14:38:40 America/Los_Angeles] AVideoLog::DEBUG: deleteALLCache 2 rmdir /var/www/html/AVideo/videos/cache/YPTObjectCache633df960433e7 SCRIPT_NAME: /objects/configurationClearCache.json.php
[05-Oct-2022 14:38:40 America/Los_Angeles] AVideoLog::DEBUG: deleteALLCache end in 0.00020098686218262 seconds SCRIPT_NAME: /objects/configurationClearCache.json.php
[05-Oct-2022 14:38:45 America/Los_Angeles] AVideoLog::DEBUG: Warning: Slow process detected [modeYoutube.php] On  Line 358 takes 0.5271 seconds to complete, Limit (0.5). /var/www/html/AVideo/view/modeYoutube.php SCRIPT_NAME: /view/modeYoutube.php
[05-Oct-2022 14:38:50 America/Los_Angeles] AVideoLog::DEBUG: Video::save (IRREFUTABLE EVIDENCE THAT SHOULD SEND ALL WHO KNOWINGLY COOPERATED TO PRISON) Saved id = 8372  SCRIPT_NAME: /view/modeYoutube.php
[05-Oct-2022 14:38:51 America/Los_Angeles] AVideoLog::DEBUG: getSources(video_210826234716_90b7) File not found {"id":8372,"title":"IRREFUTABLE EVIDENCE THAT SHOULD SEND ALL WHO KNOWINGLY COOPERATED TO PRISON","clean_title":"irrefutable-evidence-that-should-send-all-who-knowingly-cooperated-to-prison","description":"He has \u2063leaked Pfizer jab documents & access to the Wuhan labs \"deleted\" database!\nVital information that proves, once again, this entire PLANdemic...Yet the \"board\" don't want to know! WHY??? (There IS only ONE answer!!)\nThey are all in on the scam!!!!","views_count":34,"views_count_25":0,"views_count_50":0,"views_count_75":0,"views_count_100":0,"status":"a","created":"2022-01-01 01:01:01","modified":"2022-10-05 14:38:50","users_id":5,"categories_id":4,"filename":"video_210826234716_90b7","duration":"0:10:39","type":"video","videoDownloadedLink":"","order":1,"rotation":0,"zoom":1,"youtubeId":"","videoLink":"","next_videos_id":null,"isSuggested":0,"trailer1":"","trailer2":"","trailer3":"","rate":0,"can_download":0,"can_share":0,"rrating":"","externalOptions":"{\"HigestResolution\":null}","only_for_paid":0,"serie_playlists_id":null,"sites_id":null,"video_password":"","encoderURL":"https:\/\/conspyre.tv\/Encoder\/","filepath":"","filesize":1189680,"live_transmitions_history_id":null,"total_seconds_watching":0,"duration_in_seconds":639,"likes":0,"dislikes":0,"users_id_company":null,"epg_link":null} SCRIPT_NAME: /view/modeYoutube.php
[05-Oct-2022 14:38:51 America/Los_Angeles] AVideoLog::DEBUG: getDirSize: start /var/www/html/AVideo/videos/video_210826234716_90b7/ SCRIPT_NAME: /view/modeYoutube.php
[05-Oct-2022 14:38:51 America/Los_Angeles] AVideoLog::DEBUG: getDirSize: found 1189680 from - 1189680   /var/www/html/AVideo/videos/video_210826234716_90b7/ SCRIPT_NAME: /view/modeYoutube.php
[05-Oct-2022 14:38:51 America/Los_Angeles] AVideoLog::DEBUG: AWS_S3::getFilesize did not found any file video_210826234716_90b7 SCRIPT_NAME: /view/modeYoutube.php
[05-Oct-2022 14:38:51 America/Los_Angeles] AVideoLog::DEBUG: updateFilesize: No need to update videos_id=8372 filename=video_210826234716_90b7 filesize=1189680 1.13MB SCRIPT_NAME: /view/modeYoutube.php
[05-Oct-2022 14:38:51 America/Los_Angeles] AVideoLog::DEBUG: Warning: Slow process detected [modeYoutube.php] On  Line 358 takes 0.5222 seconds to complete, Limit (0.5). /var/www/html/AVideo/view/modeYoutube.php SCRIPT_NAME: /view/modeYoutube.php
[05-Oct-2022 14:38:54 America/Los_Angeles] AVideoLog::DEBUG: Roku Empty video source 11112, 2011-julian-assange-in-conversation-with-john-pilger, video_220814164051_v116e SCRIPT_NAME: /feed/index.php
[05-Oct-2022 14:39:09 America/Los_Angeles] AVideoLog::DEBUG: Roku Empty video source 12151, energy-from-the-vacuum-02---john-benini---tom-bearden-john-bedini, video_221001230624_vee58 SCRIPT_NAME: /feed/index.php
[05-Oct-2022 14:39:09 America/Los_Angeles] AVideoLog::DEBUG: Roku Empty video source 12150, energy-from-the-vacuum-03---dr--deborah-chung---negative-resistance---tom-bearden-john-bedini, video_221001230454_va525 SCRIPT_NAME: /feed/index.php
[05-Oct-2022 14:39:09 America/Los_Angeles] AVideoLog::DEBUG: Roku Empty video source 12149, energy-from-the-vacuum-04---howard-johnson---magnetic-exchange---tom-bearden-john-bedini, video_221001230317_v8f3d SCRIPT_NAME: /feed/index.php
[05-Oct-2022 14:39:09 America/Los_Angeles] AVideoLog::DEBUG: Roku Empty video source 12147, energy-from-the-vacuum-06---inside-radient-energy---tom-bearden-john-bedini, video_221001225738_v8e14 SCRIPT_NAME: /feed/index.php
[05-Oct-2022 14:39:09 America/Los_Angeles] AVideoLog::DEBUG: Roku Empty video source 12146, energy-from-the-vacuum-07---tesla-s-impulse-technology---tom-bearden-john-bedini, video_221001225511_v8fcf SCRIPT_NAME: /feed/index.php
[05-Oct-2022 14:39:09 America/Los_Angeles] AVideoLog::DEBUG: Roku Empty video source 12145, energy-from-the-vacuum-08---challenging-the-2nd-law-of-thermodynamics---tom-bearden-john-bedini, video_221001225345_vd475 SCRIPT_NAME: /feed/index.php
[05-Oct-2022 14:39:09 America/Los_Angeles] AVideoLog::DEBUG: Roku Empty video source 12144, energy-from-the-vacuum-09---moray-sweet-and-antigravity-2009-x264-tom-bearden-john-bedini, video_221001225220_v9ab5 SCRIPT_NAME: /feed/index.php
[05-Oct-2022 14:39:09 America/Los_Angeles] AVideoLog::DEBUG: Roku Empty video source 12143, energy-from-the-vacuum-10---the-kromrey-converter-2009-x264-tom-bearden-john-bedini, video_221001225137_v56d9 SCRIPT_NAME: /feed/index.php
[05-Oct-2022 14:39:09 America/Los_Angeles] AVideoLog::DEBUG: Roku Empty video source 12142, energy-from-the-vacuum-11---magnetic-gates-and-howard-johnson-2009-x264-tom-bearden-john-bedini, video_221001225032_v0501 SCRIPT_NAME: /feed/index.php
[05-Oct-2022 14:39:09 America/Los_Angeles] AVideoLog::DEBUG: Roku Empty video source 12141, energy-from-the-vacuum-12---petrovoltaics-and-the-faraday-motor-2009-x264-tom-bearden-john-bedini, video_221001224808_vf109 SCRIPT_NAME: /feed/index.php
[05-Oct-2022 14:39:09 America/Los_Angeles] AVideoLog::DEBUG: Roku Empty video source 12140, energy-from-the-vacuum-13---electrodynamics-its-origins-history-and-struggles-2009-x264-tom-bearden-john-bedini, video_221001224648_vfc98 SCRIPT_NAME: /feed/index.php
[05-Oct-2022 14:39:09 America/Los_Angeles] AVideoLog::DEBUG: Roku Empty video source 12138, energy-from-the-vacuum-15---seeking-overunity-2009-x264-tom-bearden-john-bedini, video_221001222954_vc410 SCRIPT_NAME: /feed/index.php
[05-Oct-2022 14:39:09 America/Los_Angeles] AVideoLog::DEBUG: Roku Empty video source 12137, energy-from-the-vacuum-16---equilibrium-entropy-energy-defining-moments-2009-x264-tom-bearden-john-bedini, video_221001222606_v6779 SCRIPT_NAME: /feed/index.php
[05-Oct-2022 14:39:09 America/Los_Angeles] AVideoLog::DEBUG: Roku Empty video source 12136, energy-from-the-vacuum-17---the-tape-drive-motor-2010-x264-tom-bearden-john-bedini, video_221001222422_v4b12 SCRIPT_NAME: /feed/index.php
[05-Oct-2022 14:39:09 America/Los_Angeles] AVideoLog::DEBUG: Roku Empty video source 12135, energy-from-the-vacuum-18---transmutation---lifting-the-veil-2010-tom-bearden-john-bedini-1, video_221001222236_v34fa SCRIPT_NAME: /feed/index.php
[05-Oct-2022 14:39:13 America/Los_Angeles] AVideoLog::DEBUG: createStprits: videosURL: https://conspyre-1.s3.us-west-1.wasabisys.com/video_220929110902_ve91b_480.mp4 SCRIPT_NAME: /view/modeYoutube.php
[05-Oct-2022 14:39:13 America/Los_Angeles] AVideoLog::DEBUG: createStprits: videosURL+token: https://conspyre-1.s3.us-west-1.wasabisys.com/video_220929110902_ve91b_480.mp4 SCRIPT_NAME: /view/modeYoutube.php
[05-Oct-2022 14:39:13 America/Los_Angeles] AVideoLog::DEBUG: createStprits: creating https://conspyre-1.s3.us-west-1.wasabisys.com/video_220929110902_ve91b_480.mp4 SCRIPT_NAME: /view/modeYoutube.php
[05-Oct-2022 14:39:13 America/Los_Angeles] AVideoLog::DEBUG: createStprits: https://pepe.conspyre.tv/Network/getSpiritsFromVideo/aHR0cHM6Ly9jb25zcHlyZS0xLnMzLnVzLXdlc3QtMS53YXNhYmlzeXMuY29tL3ZpZGVvXzIyMDkyOTExMDkwMl92ZTkxYl80ODAubXA0/150/100/0%3A03%3A45?disableCache=633df981e5892 SCRIPT_NAME: /view/modeYoutube.php
[05-Oct-2022 14:39:14 America/Los_Angeles] AVideoLog::DEBUG: wget: ERROR the url does not download 'https://pepe.conspyre.tv/Network/getSpiritsFromVideo/aHR0cHM6Ly9jb25zcHlyZS0xLnMzLnVzLXdlc3QtMS53YXNhYmlzeXMuY29tL3ZpZGVvXzIyMDkyOTExMDkwMl92ZTkxYl80ODAubXA0/150/100/0%3A03%3A45?disableCache=633df981e5892', '/var/www/html/AVideo/videos/cache/YPTurl_get_contents/cfc04671788ec6cdbfc377760f83648a' SCRIPT_NAME: /view/modeYoutube.php
[05-Oct-2022 14:39:14 America/Los_Angeles] AVideoLog::DEBUG: createStprits: created old=[0] new=[0] /var/www/html/AVideo/videos/video_220929110902_ve91b/video_220929110902_ve91b_thumbsSprit.jpg SCRIPT_NAME: /view/modeYoutube.php
[05-Oct-2022 14:39:14 America/Los_Angeles] AVideoLog::DEBUG: Warning: Slow process detected [modeYoutube.php] On  Line 358 takes 0.5406 seconds to complete, Limit (0.5). /var/www/html/AVideo/view/modeYoutube.php SCRIPT_NAME: /view/modeYoutube.php
DanielnetoDotCom commented 2 years ago

Hi

I got one video as a sample from you and I saw this source:

https://conspyre-1.s3.us-west-1.wasabisys.com/video_221001230624_vee58_400.mp4

how come you have a video resolution of 400p?

I am not 100% but I think the resolutions should follow the Encoder resolution

image

did you use the Encoder on those videos or direct upload?

walt93 commented 2 years ago

Daniel, everything goes through the encoder - we never use direct upload (ever), but not every video has a source resolution at 480, 540, 720 or 1080p. I believe I read somewhere about ffmpeg that by default it isn't going to upsample.

Can Roku not handle odd resolutions? We've got a lot of videos to up-and-or-downsample if so.

Is it possible to tell AVideo to re-encode a video? I suspect I have to turn on some lower resolutions, like 360P here so that 400P video gets downsampled to a common resolution (in this case 360P).

walt93 commented 2 years ago

Looking into this more, I had turned off 360p/480p/540p - thinking this is a "do not care" situation.

It seems that if I had left these on, for example, those 400P videos would have a 360P available & would work on Roku.

I've gone through and gotten a list of ≈360 videos that I need to re-encode.

Is there any workflow that has me scripting these into the encoder? Otherwise this is a daunting amount of work to re-encode.


Edited: I see the script reencodeAllVideos.php, but there are more than 11,000 of them - need to make this thing use just a list of IDs so I can re-encode those 360 vids…


Ok so it looks like it's just setting the state to "queued" for every video where id > 0, so, if it accepts the "in (...)" syntax here we can give it a list of IDs?

<?php

//streamer config
require_once '../videos/configuration.php';

if (!isCommandLineInterface()) {
    return die('Command Line only');
}

$sql = "UPDATE {$global['tablesPrefix']}encoder_queue SET status = 'queue' where id > 0";
echo $sql . PHP_EOL;
$insert_row = $global['mysqli']->query($sql);

if ($insert_row) {
    echo "All set to queue" . PHP_EOL;
} else {
    die($sql . ' Error : (' . $global['mysqli']->errno . ') ' . $global['mysqli']->error);
}
echo "Bye";
echo "\n";
die();

...but doing this, it doesn't seem that it is adding any videos to the encoder queue, no matter which of my encoders I run this script from.

DanielnetoDotCom commented 2 years ago

the thing is, it always saves the original resolution.

your video is probably a 400p video. and there is no lower resolution than that to convert. so the video has only 400p (which is a non-standard resolution).

the problem is, the streamer script check for all standard resolutions in a video, and I cannot make it search for all non-standard resolutions 1,2,3,4...200,201,...400,... If I do that, this will slow down the script a lot.

maybe you should try to enable the 240p at least, so this will guarantee you will have at least this resolution.

or maybe I will update the encoder, and force it to encode at least 1 lower resolution

walt93 commented 2 years ago

Update: even with 240P turned on, files that are >240P and <360P are not being transcoded. I used Handbrake to conform them, but I think there's a bug here.

DanielnetoDotCom commented 2 years ago

https://github.com/WWBN/AVideo/issues/7224#issuecomment-1273428793

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.