Video encoding never ends

[lerd2805]

Hi @DanielnetoDotCom can you help me please,

I am trying to encode a video and the process repeats itself over and over it never ends, I have looked for errors in the logs and the only warning I find are these:

[15-Apr-2021 17:50:11 UTC] PHP Warning: mysqli::query(): MySQL server has gone away in /var/www/html/objects/Object.php on line 187 [15-Apr-2021 17:50:11 UTC] PHP Warning: mysqli::query(): Error reading result set's header in /var/www/html/objects/Object.php on line 187

I am using ubuntu 18.04 and mysql 5.7.

[DanielnetoDotCom]

I am not sure but looks like the encoder process is too much for your server.

[lerd2805]

This is my phisical server: CPU 1 Intel(R) Xeon(R) CPU X5650 @ 2.67GHz (6 Cores) CPU 2 Intel(R) Xeon(R) CPU X5650 @ 2.67GHz (6 Cores) RAM 65536 MB

The Avideo and Avideo encoder runs in Docker Container and has no restrictions on memory or CPU, that is, the container can occupy all cpus or ram if need it

i use the oficcial image for mysql 5.7 and this is my dockerfile for the encoder:

FROM ubuntu:18.04

ARG DEBIAN_FRONTEND=noninteractive

RUN apt-get update \ && apt install -y git logrotate --no-install-recommends \ && apt install -y apache2 curl ffmpeg libimage-exiftool-perl \ && apt install -y php7.2 \ php7.2-common \ php7.2-cli \ libapache2-mod-php7.2 \ php7.2-intl \ php7.2-json \ php7.2-mbstring \ php7.2-curl \ php7.2-mysql \ php7.2-bcmath \ php7.2-xml \ php7.2-gd \ php7.2-zip \ --no-install-recommends \ && apt-get clean \ && rm -rf /var/lib/apt/lists/*

Change php.ini configuration

RUN sed -i 's/memory_limit\ = 128M/memory_limit\ = 512M/' /etc/php/7.2/apache2/php.ini \ && sed -i 's/max_execution_time\ = 30/max_execution_time\ = 7200/' /etc/php/7.2/apache2/php.ini \ && sed -i 's/post_max_size\ = 8M/post_max_size\ = 5000M/' /etc/php/7.2/apache2/php.ini \ && sed -i 's/upload_max_filesize\ = 2M/upload_max_filesize\ = 5000M/' /etc/php/7.2/apache2/php.ini

Get AVideo Encoder & Youtube-dl

RUN rm -rf /var/www/html/* \ && git clone https://github.com/WWBN/AVideo-Encoder.git /var/www/html \ && chown -R www-data:www-data /var/www/html \ && curl -L https://yt-dl.org/downloads/latest/youtube-dl -o /usr/local/bin/youtube-dl \ && chmod a+rx /usr/local/bin/youtube-dl \ && sed -i 's/AllowOverride None/AllowOverride All/' /etc/apache2/apache2.conf \ && a2enmod rewrite

WORKDIR /var/www/html

VOLUME ["/var/www/html"]

EXPOSE 80

CMD /usr/sbin/apache2ctl -D FOREGROUND

[akhilleusuggo]

I can confirm this issue in a new installation that I have made . This issue is from the command line that comes by default , but the command line itself if not faulty . I believe is something on how the database and script communicates .

I also have a strong server , and the issue is not from server. The encoding is looping . After changing this code line ( the one on the encoder ) , fixed my issue .

https://github.com/WWBN/AVideo-Encoder/blob/c9e38df0e6daea9e0158d51b315204b5c7802c35/install/database.sql#L152

Logs for debuggin.

[15-Apr-2021 20:13:06 UTC] Encoder:Format:: getDynamicCommandFromFormat::return(ffmpeg   -i {$pathFileName}  -vf scale=-2:720 -movflags +faststart -preset veryfast -vcodec h264 -acodec aac -strict -2 -b:a 192k  -max_muxing_queue_size 1024 -y /var/www/html/AVideo-Encoder/videos/avideoTmpFile_2_streamers_id_1__720.mp4  -c copy  -movflags +faststart -preset veryfast -y /var/www/html/AVideo-Encoder/videos/avideoTmpFile_2_streamers_id_1__1080.mp4 ) 
[15-Apr-2021 20:13:06 UTC] AVideo-Encoder Format::exec  Start Encoder [ffmpeg     -i /var/www/html/AVideo-Encoder/videos/2_tmpFile.mp4  -vf scale=-2:720 -movflags +faststart -preset veryfast -vcodec h264 -acodec aac -strict -2 -b:a 192k  -max_muxing_queue_size 1024 -y /var/www/html/AVideo-Encoder/videos/avideoTmpFile_2_streamers_id_1__720.mp4  -c copy  -movflags +faststart -preset veryfast -y /var/www/html/AVideo-Encoder/videos/avideoTmpFile_2_streamers_id_1__1080.mp4 ]

As above it's encoding the 720p normaly , but the 1080p is re-encoded and using a -c copy meanwhile trying to encode the audio , but that doesn't work.

PS: I think I have found the issue.

[akhilleusuggo]

I believe this option is deprecated or mixed up. Because does not follow the encoder/database. Is hard-coded.

[lerd2805]

Hi @akhilleusuggo I will try your solution, I will comment later if it also solves my problem and thank you.

[lerd2805]

@akhilleusuggo not work for me. I replace the copy for h264 in the line 509 on objects/Format.php

Logs:

[16-Apr-2021 19:11:21 UTC] AVideo-Encoder Format::exec Start Encoder [ffmpeg -i /var/www/html/videos/4_tmpFile.mp4 -vf scale=-2:480 -movflags +faststart -preset veryfast -vcodec h264 -acodec aac -strict -2 -b:a 128k -max_muxing_queue_size 1024 -y /var/www/html/videos/avideoTmpFile_4_streamers_id_1480.mp4 -vf scale=-2:720 -movflags +faststart -preset veryfast -vcodec h264 -acodec aac -strict -2 -b:a 192k -max_muxing_queue_size 1024 -y /var/www/html/videos/avideoTmpFile_4_streamers_id_1720.mp4 -c h264 -movflags +faststart -preset veryfast -y /var/www/html/videos/avideoTmpFile_4_streamers_id_11080.mp4 ] [16-Apr-2021 19:11:21 UTC] Encoder::save id=(4) title=(BIOMUSEO DOCUMENTAL BAJA H264) [16-Apr-2021 19:11:21 UTC] Encoder::save id=(4) title=(BIOMUSEO DOCUMENTAL BAJA H264) [16-Apr-2021 19:11:21 UTC] AVideo-Encoder Format::exec ERROR (1) progressFile=/var/www/html/videos/4_tmpFile_progress.txt null [16-Apr-2021 19:11:21 UTC] Encoder::save id=(4) title=(BIOMUSEO DOCUMENTAL BAJA H264) [16-Apr-2021 19:11:21 UTC] Trying again: [1] => Execute code error 1 "" Code: ffmpeg -i /var/www/html/videos/4_tmpFile.mp4 -vf scale=-2:480 -movflags +faststart -preset veryfast -vcodec h264 -acodec aac -strict -2 -b:a 128k -max_muxing_queue_size 1024 -y /var/www/html/videos/avideoTmpFile_4_streamers_id_1__480.mp4 -vf scale=-2:720 -movflags +faststart -preset veryfast -vcodec h264 -acodec aac -strict -2 -b:a 192k -max_muxing_queue_size 1024 -y /var/www/html/videos/avideoTmpFile_4_streamers_id_1720.mp4 -c h264 -movflags +faststart -preset veryfast -y /var/www/html/videos/avideoTmpFile_4_streamers_id_1__1080.mp4

[akhilleusuggo]

Can you change your configuration to something else . Take this example and let me know . We have many deprecated options ( from ffmpeg ). I use often ffmpeg , but I'm also not updated. Try this :

-vf scale=-2:{$resolution} -c:v libx264 -crf 23 -preset veryfast -profile:v high -pix_fmt yuv420p -movflags +faststart -c:a aac -strict -2 -b:a {$autioBitrate}k -profile:a aac_low -max_muxing_queue_size 8192 -y {$destinationFile}

[lerd2805]

I think I found my problem but I don't know how to solve it, let me explain if the database is defined in configuration.php as a DNS in my case I use the name of the docker service something like Stack_Encoder_db even though there is a connection to the database In some encoding processes with specific videos the Warning appears "PHP Warning: mysqli :: query (): MySQL server has gone away in /var/www/html/objects/Object.php on line 187" but if I change the definition of the database to an IP and I use the same video if it completes the encoding process. Which is strange since if I ping the docker service name the latency is 0.400 or less.

Maybe you can help me, while I will have to change the ip manually.

[akhilleusuggo]

as a DNS in my case I use the name of the docker service

Add your DNS record on /etc/hosts

Something like ,

123.45.67.89 subdom.myserver.com subdom