How could we serve images through the CDN? Because AVideo/Roku combined are murdering my web server.

[walt93]

Describe the question My web server is being beaten to death by my Roku channel, which is serving videos and thumbnails from the AVideo generated Roku.json.

I have to get this bandwidth off my network NOW and I don't know how to.

My images are served by URLs like:

https://conspyre.tv/videos/video_240507164637_v5c84/video_240507164637_v5c84.jpg

Reminder that these are from the generated JSON, so like:

"movies": [
        {
            "id": "video_18510",
            "title": "Dr. Moray's Radiant Free Energy Device",
            "longDescription": "...",
            "shortDescription": "...",
            "thumbnail": "https://conspyre.tv/videos/video_240507164637_v5c84/video_240507164637_v5c84.jpg",
            "tags": [
                "Documentaries"
            ],
"genres": [
                "special"
            ],
            "releaseDate": "2024-05-07T21:46:37-05:00",
            "categories_id": 16,
            "rating": {
                "rating": "UNRATED",
                "ratingSource": "MPAA"
            },
            "content": {
                "dateAdded": "2024-05-07T21:46:37-05:00",
                "captions": [],
                "duration": 1380,
                "language": "en",
                "adBreaks": [
                    "00:00:00"
                ],
                "vmap_xml": "https://conspyre.tv/plugin/API/get.json.php?APIName=vmap&videos_id=18510",
                "vmap_json": "https://conspyre.tv/plugin/API/get.json.php?APIName=vmap&videos_id=18510&json=1",
                "vast": "https://conspyre.tv/plugin/API/get.json.php?APIName=vast&videos_id=18510",
                "videos": [
                    {
                        "url": "https://b-low.b-cdn.net/video_240507164637_v5c84_720.mp4",
                        "quality": "HD",
                        "videoType": "MP4"
                    }
                ]
            }

When I try to "wrap" these thumbnail URLs in a Bunny CDN URL (using the same bucket that AVideo stores its files in), I can't make it work. However, the video file is being served by the CDN, look at the videos.url structure above and observe 1) that it has the Bunny CDN URL ("b-low.b-cdn.net") and 2) points directly at the video file.

https://conspyre.tv/videos/video_240507164637_v5c84/video_240507164637_v5c84.jpg THIS WORKS https://b-low.b-cdn.net/videos/video_240507164637_v5c84/video_240507164637_v5c84.jpg DOESN'T WORK https://b-low.b-cdn.net/video_240507164637_v5c84/video_240507164637_v5c84.jpg DOESN'T WORK https://b-low.b-cdn.net/video_240507164637_v5c84.jpg DOESN'T WORK

So I do not know how to make the CDN URL work with the video thumbnail.

In the file system, the file exists at /var/www/html/conspyre.tv/videos/videos/video_240507164637_v5c84/video_240507164637_v5c84.jpg

AFAIK the Wasabi bucket points to the root at "videos"

HELP!

Logs Imagine gigabytes of this scrolling at high speed:

108.219.40.108 - - [01/Oct/2024:23:41:43 +0000] "GET /videos/IsQaPsyOpWhateffecthasQhadontheUSAAnonsinfil_656da65de985f1.97616666/IsQaPsyOpWhateffecthasQhadontheUSAAnonsinfil_656da65de985f1.97616666.jpg?cache=1701685464_1724786461 HTTP/1.1" 200 419860 "-" "Roku/DVP-13.1 (13.1.4.01510-C2)"
108.219.40.108 - - [01/Oct/2024:23:41:43 +0000] "GET /videos/v_240408030304_v9e79/v_240408030304_v9e79.jpg?cache=1712784234_1724786461 HTTP/1.1" 200 181341 "-" "**Roku/DVP-13.1** (13.1.4.01510-C2)"
107.142.140.88 - - [01/Oct/2024:23:41:43 +0000] "GET /videos/v_230924022242_vfe58/v_230924022242_vfe58.jpg?cache=1701255497_1724786460 HTTP/1.1" 200 198758 "-" "**Roku/DVP-13.1** (13.1.4.01510-C2)"
108.219.40.108 - - [01/Oct/2024:23:41:43 +0000] "GET /videos/video_240331014747_v2484/video_240331014747_v2484.jpg?cache=1711997007_1724786461 HTTP/1.1" 200 1136540 "-" "**Roku/DVP-13.1** (13.1.4.01510-C2)"
107.142.140.88 - - [01/Oct/2024:23:41:43 +0000] "GET /videos/video_220329122430_vc5fa/video_220329122430_vc5fa.jpg?cache=1701255192_1724786456 HTTP/1.1" 200 25927 "-" "**Roku/DVP-13.1** (13.1.4.01510-C2)"

Note that these accesses are all by ROKU devices reading image URLs supplied by AVideo.

[DanielnetoDotCom]

is the issue the bandwidth only or also CPU?

For best CDN results you should use our CDN

https://github.com/WWBN/AVideo/wiki/CDN-Plugin

Depending on your usage I can offer special prices.

[walt93]

Daniel-

I just want the images served by my CDN. I don't know how it used to work. For the last 1.5 years it has worked smoothly.

Now, none of the images are being served by the CDN. I don't want to pay a 2x markup on Bunny CDN, I'm barely treading water as it is.

Is that what happened? Did you switch off serving images over our CDN URL (supplied in the S3 plugin) to drive traffic to your CDN?

[DanielnetoDotCom]

I did not, but now you made clear are you using S3/CDN plugin ?

[DanielnetoDotCom]

Please check if this update makes any difference for you

[DanielnetoDotCom]

Sorry, nevermind, I had to revert, the S3 CDN supposed to be only for videos, the images still hosted in your server, so you should use the advanced custom -> videosCDN

[walt93]

So I guess that videosCDN must have been set up correctly in the past to make this work, and it got cleared during a recent update is my best theory.

I forgot to ask what format to put in for videosCDN.

I used https://b-low.b-cdn.net/ which is correct for my CDN proxy. I guess it doesn't like "https://" there or something (?)

Now my site won't load any javascript, can't log in, not sure where to fix the CDN URL.

Can you show me where to fix this in the database or something? Apache logs are still scrolling madly so putting in the CDN URL in that form definitely is not working.

[walt93]

Update: I found it in table 'plugins' id 70 is a in field "object_data" is a JSON blob with all of the advancedConfiguration data. I tried to unf**k the site by removing this. A surgical SQL command written by LLM does the trick (it took a couple of iterations because we had to screw with SQL backquoting rules. GPT FTW.):

UPDATE plugins
SET object_data = REPLACE(object_data, 'https:\\/\\/b-low.b-cdn.net', '')
WHERE id = 70;

After making this fix, the site is still not loading. Probably we need to do the dance (delete / rebuild cache)?

[walt93]

I tried changing the form of the CDN URL, and this time I did not use "https://" and just put in the base URL for the CDN.

This seems to have broken the conspyre.tv website, now none of the videos have thumbnails on the main page. Thumbnails are visible from the videos page. The video plays. And Apache continues to scroll accesses to the video thumbnails from a lot of Roku devices.

So I can't put in the videosCDN under configuration advanced, either https://b-low.b-cdn.net <--- this kills javascript on the AVideo site, doesn't change the behavior of apache serving all of the Roku images. b-low.b-cdn.net <---- this kills thumbnails on the main videos page, doesn't change the behavior of apache serving all of the Roku images either.

I guess I could write a script or something that scans the Roku JSONs, copies the images up to CDN, mangles the URL in the Roku JSON and fixes everything up nice. :-/