Unable to transfer large files

[tbprince]

Hello,

our instance of AVideo has external storage (YPT) and external encoder deployed. Encoding and transferring to the storage small or medium-size files has no issues. It works fine.

However, when we deal with very large files (say, 1.9GB) encoding seems to be working fine even if takes a long time. Then, moving files to the storage server somewhat fails so the video ends up in an status of "missing source files". A situation like this one:

As you can see, size is then reported to be about 1MB and it reports broken/missing files.

Accessing the storage to check if anything has been transferred shows nothing. No folder created, no files moved.

What could the issue be since smaller/medium sized files are ok? Encoding files up to 700-800MB works fine.

Thanks.

[DanielnetoDotCom]

Check the streamer encoder and storage logs

[tbprince]

I can see this error:

[16-Nov-2023 02:10:31 Europe/Berlin] sendToStreamer https://(redacted)/aVideoEncoder.json in 15.918 seconds {"error":true,"target":"aVideoEncoder.json","postFields":{"duration":"EE:EE:EE","title":"(redacted)","categories_id":0,"format":"zip","resolution":null,"videoDownloadedLink":"","description":"","downloadURL":"https:\/\/(redacted)\/videos\/avideoTmpFile_47_streamers_id_2_.zip","encoderURL":"https:\/\/(redacted)\/","streamers_id":"2","user":"(redacted)","pass":"(redacted)","return_vars":"{\"videos_id\":50,\"video_id_hash\":\"(redacted)\",\"releaseDate\":\"2023-11-16 00:04:00\"}","releaseDate":"2023-11-16 00:04:00","releaseTime":1700089440,"videos_id":50,"video_id_hash":"(redacted)","timezone":"Europe\/Berlin"},"response_raw":"array(4) {\n  [\"type\"]=>\n  int(1)\n  [\"message\"]=>\n  string(73) \"**Out of memory (allocated 3229609984) (tried to allocate 3219128352 bytes)**\"\n  [\"file\"]=>\n  string(37) \"\/var\/www\/AVideo\/objects\/functions.php\"\n  [\"line\"]=>\n  int(3209)\n}\n{\"error\":true,\"msg\":\"Under Maintanance\"}","response":null,"msg":"Response was not an json object"}
[16-Nov-2023 02:10:31 Europe/Berlin] Encoder:sendFileChunk(/var/www/html/videos/avideoTmpFile_47_streamers_id_2_.zip,50, zip, object, , 3)
[16-Nov-2023 02:10:31 Europe/Berlin] Encoder::sendFileChunk file=/var/www/html/videos/avideoTmpFile_47_streamers_id_2_.zip
[16-Nov-2023 02:11:46 Europe/Berlin] AVideo-Streamer countCURLOPT_READFUNCTION = (3647647) chunk answer {"file":"\/tmp\/YTPChunk_ASLhP5","filesize":0}
[16-Nov-2023 02:11:46 Europe/Berlin] cURL error, file size is smaller, trying again (4) (0):
 No error 

Could this be an out of memory error? How come? Does the encoder need to load the whole file in memory? 😐

Thanks.

[DanielnetoDotCom]

I need a lot of more logs, check the streamer encoder and storage logs