search

WWBN / AVideo

Create Your Own Broadcast Network With AVideo Platform Open-Source. OAVP OVP
https://avideo.tube/AVideo_OpenSource
Other
1.9k stars 972 forks source link

SendRecordedToEncoder does not work with http -> https redirection in apache #2633

Closed maxlinux2000 closed 4 years ago

maxlinux2000 commented 4 years ago

Because my installation of Avideo allowed to connect via http, I added a rule in apache to redirect to the https protocol.

Avideo continues to work correctly everywhere, even when I launch a Live, the Home shows the thumbnail with the e the warning bell signals the event.

The problem is that now Live is not recorded. Apparently the SendRecordedToEncoder plugin did not like the change in apache configuration

I put here the Apache configuration that I modified

<VirtualHost *:80> 
    DocumentRoot "/var/www/html/cathotube.com/YouPHPTube" 
    ServerName cathotube.com 
    ServerAlias live.cathotube.com, s1.cathotube.com

    # Other directives here 
    RewriteEngine on
    RewriteCond %{SERVER_NAME} =cathotube.com 
    RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent] 
</VirtualHost>
<VirtualHost *:80>
    DocumentRoot "/var/www/html/cathotube.com/YouPHPTube-Encoder"
    ServerName encoder.cathotube.com
    # Other directives here
    RewriteEngine on
    RewriteCond %{SERVER_NAME} =encoder.cathotube.com
    RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>
<VirtualHost *:443>
    DocumentRoot "/var/www/html/cathotube.com/YouPHPTube-Encoder"
    ServerName encoder.cathotube.com
    # Other directives here
    Include /etc/letsencrypt/options-ssl-apache.conf
    SSLCertificateFile /etc/letsencrypt/live/cathotube.com-0001/fullchain.pem
    SSLCertificateKeyFile /etc/letsencrypt/live/cathotube.com-0001/privkey.pem
</VirtualHost>

I added the rules in the port 80 here:

    RewriteEngine on
    RewriteCond %{SERVER_NAME} =cathotube.com 
    RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]

and here all the logs

Check if NGINX is Running  =========================================================================

  829 ?        00:00:00 nginx
  832 ?        01:58:19 nginx
  833 ?        00:00:25 nginx

NGINX Error ========================================================================================
/usr/local/nginx/logs/error.log---------------------------------------------------------

2020/02/12 22:49:11 [info] 832#0: *121314 client connected '192.168.0.2'
2020/02/12 22:49:11 [info] 832#0: *121314 connect: app='live' args='p=78f6520b4ecee62b24eb474a3fe3ce4a' flashver='FMLE/3.0 (compatible; Lavf58.20' swf_url='' tc_url='rtmp://live.cathotube.com:1935/live?p=78f6520b4ecee62b24eb474a3fe3ce4a' page_url='' acodecs=0 vcodecs=0 object_encoding=0, client: 192.168.0.2, server: 0.0.0.0:1935
2020/02/12 22:49:11 [info] 832#0: *121314 createStream, client: 192.168.0.2, server: 0.0.0.0:1935
2020/02/12 22:49:11 [info] 832#0: *121314 publish: name='5e26cddbcf78d' args='' type=live silent=0, client: 192.168.0.2, server: 0.0.0.0:1935
2020/02/12 22:49:11 [info] 832#0: *121314 notify: publish 'live.cathotube.com/plugin/Live/on_publish.php', client: 192.168.0.2, server: 0.0.0.0:1935
2020/02/12 22:50:08 [info] 832#0: *121314 deleteStream, client: 192.168.0.2, server: 0.0.0.0:1935
2020/02/12 22:50:08 [info] 832#0: *121314 send() failed (32: Broken pipe), client: 192.168.0.2, server: 0.0.0.0:1935
2020/02/12 22:50:08 [info] 832#0: *121314 notify: record_done recorder=video path='/var/www/tmp/5e26cddbcf78d-12-Feb-20.flv' url='cathotube.com/plugin/Live/on_record_done.php', client: 192.168.0.2, server: 0.0.0.0:1935
2020/02/12 22:50:08 [info] 832#0: *121314 disconnect, client: 192.168.0.2, server: 0.0.0.0:1935
2020/02/12 22:50:08 [info] 832#0: *121314 deleteStream, client: 192.168.0.2, server: 0.0.0.0:1935

NGNX Access ========================================================================================
/usr/local/nginx/logs/access.log--------------------------------------------------------

122.228.19.80 - - [12/Feb/2020:22:49:04 +0100] "GET / HTTP/1.1" 200 612 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:47.0) Gecko/20100101 Firefox/47.0"
192.168.0.2 [12/Feb/2020:22:50:08 +0100] PUBLISH "live" "5e26cddbcf78d" "" - 3270748 634 "" "FMLE/3.0 (compatible; Lavf58.20" (57s)
send() failed (32: Broken pipe), client: 192.168.0.2, server: 0.0.0.0:1935

STREAMER ========================================================================================
/var/www/html/cathotube.com/YouPHPTube/videos/avideo.log----------------------------

    [path] => /live
    [query] => p=78f6520b4ecee62b24eb474a3fe3ce4a
)

[12-Feb-2020 22:49:11 Europe/Berlin] Array
(
    [p] => 78f6520b4ecee62b24eb474a3fe3ce4a
)

[12-Feb-2020 22:49:30 Europe/Berlin] Bot Detected, NOT showing the cache (/robots.txt) FROM: 66.249.70.12 Browser: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

ENCODER ========================================================================================
/var/www/html/cathotube.com/YouPHPTube-Encoder/videos/avideo.log-------------------

list files of ========================================================================================
/var/www/tmp/-------------------

total 3.2M
4.0K drwxrwxrwx 2 root     root     4.0K feb 12 22:49 .
4.0K drwxr-xr-x 4 root     root     4.0K nov 14 21:53 ..
3.2M -rw-r--r-- 1 www-data www-data 3.2M feb 12 22:50 5e26cddbcf78d-12-Feb-20.flv

list files of ========================================================================================
/HLS-------------------

/HLS/:
total 12K
4.0K drwxrwxrwx  3 www-data www-data 4.0K oct 20 07:36 .
4.0K drwxr-xr-x 22 root     root     4.0K feb  9 15:04 ..
4.0K drwxrwxrwx  3 www-data www-data 4.0K feb 12 22:49 live

/HLS/live:
total 12K
4.0K drwxrwxrwx 3 www-data www-data 4.0K feb 12 22:49 .
4.0K drwxrwxrwx 3 www-data www-data 4.0K oct 20 07:36 ..
4.0K drwxr--r-- 2 www-data www-data 4.0K feb 12 22:50 5e26cddbcf78d

/HLS/live/5e26cddbcf78d:
total 3.4M
4.0K drwxr--r-- 2 www-data www-data 4.0K feb 12 22:50 .
4.0K drwxrwxrwx 3 www-data www-data 4.0K feb 12 22:49 ..
372K -rw-r--r-- 1 www-data www-data 371K feb 12 22:49 0.ts
576K -rw-r--r-- 1 www-data www-data 573K feb 12 22:49 1.ts
660K -rw-r--r-- 1 www-data www-data 659K feb 12 22:49 2.ts
972K -rw-r--r-- 1 www-data www-data 971K feb 12 22:49 3.ts
580K -rw-r--r-- 1 www-data www-data 580K feb 12 22:50 4.ts
272K -rw-r--r-- 1 www-data www-data 270K feb 12 22:50 5.ts
4.0K -rw-r--r-- 1 www-data www-data  136 feb 12 22:50 index.m3u8

list files of ========================================================================================
/var/www/html/cathotube.com/YouPHPTube-Encoder/videos/

total 52K
 40K drwxr-xr-x  3 www-data www-data  36K feb 12 22:47 .
4.0K drwxr-xr-x 12 www-data www-data 4.0K feb 12 22:47 ..
   0 -rw-r--r--  1 www-data www-data    0 feb 12 22:48 avideo.log
4.0K drwxr-xr-x 13 www-data www-data 4.0K ene 21 16:30 chunk
4.0K -rw-r--r--  1 www-data www-data  671 oct 20 20:02 configuration.php
root@s1:/var/www/html/cathotube.com#

I can see a send() failed (32: Broken pipe), client: 192.168.0.2, server: 0.0.0.0:1935

In /var/www/tmp after the test live, there is the .flv file... but the the Encoder don't get any file to process.

akhilleusuggo commented 4 years ago

and what about the encoder logs ? you're connected from home to a server ? client: 192.168.0.2 , or at your home is the server hosted ? RewriteCond %{SERVER_NAME} =cathotube.com

maxlinux2000 commented 4 years ago

the encoder don't receive any files y stay quiet. The logs above is a compilation of all logs including the encoder one


ENCODER ========================================================================================
/var/www/html/cathotube.com/YouPHPTube-Encoder/videos/avideo.log-------------------

The streamer and the encoder stay in the same server (192.168.0.2) and there is not other encoder in the network

maxlinux2000 commented 4 years ago

If someone wants to use my log collector, I put here two scripts. The first will read to reset the logs (delete all of the above, so much care) and the second collects the logs and various information, such as if nginx is running.

Here the ResetLogs.sh script. Install it in /usr/local/bin Please change the path in the 2 variables

#!/bin/bash
YOUPHPTUBE_STREAMER_LOG=/var/www/html/AVideo-Streamer/videos/avideo.log
YOUPHPTUBE_ENCODER_LOG=/var/www/html/AVideo-Encoder/videos/avideo.log
:> /usr/local/nginx/logs/error.log 
:> /usr/local/nginx/logs/access.log 
:> $YOUPHPTUBE_STREAMER_LOG 
:> $YOUPHPTUBE_ENCODER_LOG

Here the GetLogs.sh script. Install it in /usr/local/bin Please change the path in the 2 variables


#!/bin/bash

YOUPHPTUBE_STREAMER_LOG=/var/www/html/AVideo-Streamer/videos/avideo.log
YOUPHPTUBE_ENCODER_LOG=/var/www/html/AVideo-Encoder/videos/avideo.log
clear
echo "
Check if NGINX is Running  =========================================================================
"
ps -A | grep nginx

echo "

NGINX Error ========================================================================================
/usr/local/nginx/logs/error.log---------------------------------------------------------
"
tail /usr/local/nginx/logs/error.log 

echo "

NGNX Access ========================================================================================
/usr/local/nginx/logs/access.log--------------------------------------------------------
"
tail /usr/local/nginx/logs/access.log 

echo "

STREAMER ========================================================================================
$YOUPHPTUBE_STREAMER_LOG <----------------------------
"
tail $YOUPHPTUBE_STREAMER_LOG

echo "

ENCODER ========================================================================================
$YOUPHPTUBE_ENCODER_LOG <-------------------
"
tail $YOUPHPTUBE_ENCODER_LOG

echo "

list files of ========================================================================================
/var/www/tmp/-------------------
"
ls -lash /var/www/tmp/

echo "

list files of ========================================================================================
/HLS-------------------
"
ls -lashR /HLS/

echo "

list files of ========================================================================================
$YOUPHPTUBE_ENCODER_LOG
"
ls -lash $YOUPHPTUBE_ENCODER_LOG
maxlinux2000 commented 4 years ago

I forget to mention:

SendRecordedToEncoder (SendRecordedToEncoder) UUID: eb6e2808-d876-4488-94cb-2448a6b14e0b Version: @1.0

and the parameters:

Screenshot_2020-02-13_16-23-45

DanielnetoDotCom commented 4 years ago

@maxlinux2000

I did not see your streamer log

and please paste your nginx.conf too

maxlinux2000 commented 4 years ago

Hi, here my nginx.conf


root@s1:~# cd /usr/local/nginx/conf/
root@s1:/usr/local/nginx/conf# cat nginx.conf
 user  www-data;
 worker_processes  1;
    error_log  logs/error.log debug;
    events {
            worker_connections  1024;
    }
    rtmp {
            server {
                    listen 1935;
                    allow play all;
                    #creates our "live" full-resolution HLS videostream from our incoming encoder stream and tells where to put the HLS video manifest and video fragments
                    application live {
                            allow play all;
                            live on;
                            #record all;
                            #record_path /video_recordings;
                            #record_unique on;
                            hls on;
                            hls_nested on;
                            hls_path /HLS/live;
                            #hls_playlist_length 4s;
                            #hls_fragment 1s;
                            hls_fragment 10s;
                            on_publish http://live.cathotube.com/plugin/Live/on_publish.php;
                            on_play http://cathotube.com/plugin/Live/on_play.php;
                            on_record_done http://cathotube.com/plugin/Live/on_record_done.php;
                           recorder video{
                               record all;
                               record_path /var/www/tmp;
                               record_notify on;
                               record_max_size 2000M;
                               #will produce files of the form yourUserKey-24-Apr-13-18:23:38.flv
                               #record_suffix -%d-%b-%y-%T.flv;
                               record_suffix -%d-%b-%y.flv;

                           }
                    }
            }
    }
    http {
            include       mime.types;
            default_type  application/octet-stream;
            server {
                    listen 8080;
                    server_name localhost cathotube.com;
                    #creates the http-location for our full-resolution (desktop) HLS stream - "http://my-ip/live/my-stream-key/index.m3u8"      
                    location /live {
                            # Disable cache
                            add_header 'Cache-Control' 'no-cache';

                            # CORS setup
                            add_header 'Access-Control-Allow-Origin' '*' always;
                            add_header 'Access-Control-Expose-Headers' 'Content-Length';

                            # allow CORS preflight requests
                            if ($request_method = 'OPTIONS') {
                                    add_header 'Access-Control-Allow-Origin' '*';
                                    add_header 'Access-Control-Max-Age' 1728000;
                                    add_header 'Content-Type' 'text/plain charset=UTF-8';
                                    add_header 'Content-Length' 0;
                                    return 204;
                            }
                            types {
                                    application/vnd.apple.mpegurl m3u8;
                            }
                            alias /HLS/live;
                    }
                    #allows us to see how stats on viewers on our Nginx site using a URL like: "http://my-ip/stats"     
                    #location /stats {
                    #        stub_status;
                    #}
                    location /stat {
                            rtmp_stat all;
                            rtmp_stat_stylesheet stat.xsl;
                    }
                    location /stat.xsl {
                            root html;
                    }
                    location /control {
                            rtmp_control all;
                    }
                    #allows us to host some webpages which can show our videos: "http://my-ip/my-page.html"     
                    location / {
                            root   html;
                            index  index.html index.htm;
                    }

            listen 444 ssl; # managed by Certbot
            ssl_certificate /etc/letsencrypt/live/cathotube.com-0001/fullchain.pem; # managed by Certbot
            ssl_certificate_key /etc/letsencrypt/live/cathotube.com-0001/privkey.pem; # managed by Certbot
            include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot

            }
    }

Now the experiment: I reset all the logs, for exampl the encoder log is empty now

# tail /var/www/html/cathotube.com/YouPHPTube-Encoder/videos/avideo.log
#

Now with ffmpeg I will send a film in streaming Live (I will change the passwd and key asap)

Screenshot_2020-02-14_10-34-11 Screenshot_2020-02-14_10-35-22 Screenshot_2020-02-14_10-36-33

During the streaming the encoder log show:

root@s1:~# tail /var/www/html/cathotube.com/YouPHPTube-Encoder/videos/avideo.log 
[14-Feb-2020 10:34:50 Europe/Berlin] Create Gif with Ppallet: ffmpeg -y -t 3 -i https://cathotube.com:444/live/5e26cddbcf78d/index.m3u8 -vf fps=10,scale=320:-1:flags=lanczos,palettegen /tmp/011210adae51bfcd62acb702dc6e07a7palette.png &> /dev/null &
[14-Feb-2020 10:34:50 Europe/Berlin] /var/www/html/cathotube.com/YouPHPTube-Encoder/objects/getImage.php 102Execution time : 1.6467478275299 seconds
[14-Feb-2020 10:36:59 Europe/Berlin] getImage  Destination get Image from https://cathotube.com:444/live/5e26cddbcf78d/index.m3u8 jpg: /tmp/011210adae51bfcd62acb702dc6e07a7.jpg
[14-Feb-2020 10:36:59 Europe/Berlin] getImage  Destination get Image from https://cathotube.com:444/live/5e26cddbcf78d/index.m3u8 gif: /tmp/011210adae51bfcd62acb702dc6e07a7.gif
[14-Feb-2020 10:37:07 Europe/Berlin] https://cathotube.com/ == https://cathotube.com/
[14-Feb-2020 10:37:07 Europe/Berlin] login.json: Login::run
[14-Feb-2020 10:37:07 Europe/Berlin] Login::run (peliculas, ***, https://cathotube.com/, true)
[14-Feb-2020 10:37:07 Europe/Berlin] Verification Start https://cathotube.com/
[14-Feb-2020 10:37:07 Europe/Berlin] Verification Creating the Cache https://cathotube.com/
[14-Feb-2020 10:37:09 Europe/Berlin] Verification Response (https://search.avideo.com/verify.php?url=https%3A%2F%2Fcathotube.com%2F): {"msg":["Start","https:\/\/cathotube.com\/ Site verified"],"verified":true,"status":"a"}
root@s1:~#

Now I will stop the streaming from ffmpeg

Screenshot_2020-02-14_10-42-17

And here the log of encoder


root@s1:~# tail /var/www/html/cathotube.com/YouPHPTube-Encoder/videos/avideo.log 
[14-Feb-2020 10:34:50 Europe/Berlin] Create Gif with Ppallet: ffmpeg -y -t 3 -i https://cathotube.com:444/live/5e26cddbcf78d/index.m3u8 -vf fps=10,scale=320:-1:flags=lanczos,palettegen /tmp/011210adae51bfcd62acb702dc6e07a7palette.png &> /dev/null &
[14-Feb-2020 10:34:50 Europe/Berlin] /var/www/html/cathotube.com/YouPHPTube-Encoder/objects/getImage.php 102Execution time : 1.6467478275299 seconds
[14-Feb-2020 10:36:59 Europe/Berlin] getImage  Destination get Image from https://cathotube.com:444/live/5e26cddbcf78d/index.m3u8 jpg: /tmp/011210adae51bfcd62acb702dc6e07a7.jpg
[14-Feb-2020 10:36:59 Europe/Berlin] getImage  Destination get Image from https://cathotube.com:444/live/5e26cddbcf78d/index.m3u8 gif: /tmp/011210adae51bfcd62acb702dc6e07a7.gif
[14-Feb-2020 10:37:07 Europe/Berlin] https://cathotube.com/ == https://cathotube.com/
[14-Feb-2020 10:37:07 Europe/Berlin] login.json: Login::run
[14-Feb-2020 10:37:07 Europe/Berlin] Login::run (peliculas, ***, https://cathotube.com/, true)
[14-Feb-2020 10:37:07 Europe/Berlin] Verification Start https://cathotube.com/
[14-Feb-2020 10:37:07 Europe/Berlin] Verification Creating the Cache https://cathotube.com/
[14-Feb-2020 10:37:09 Europe/Berlin] Verification Response (https://search.avideo.com/verify.php?url=https%3A%2F%2Fcathotube.com%2F): {"msg":["Start","https:\/\/cathotube.com\/ Site verified"],"verified":true,"status":"a"}
root@s1:~#

and in the encoder interface

Screenshot_2020-02-14_10-44-28

but I can find the flv in /var/www/tmp

# ls -lash /var/www/tmp/
total 25M
4.0K drwxrwxrwx 2 root     root     4.0K feb 14 10:46 .
4.0K drwxr-xr-x 4 root     root     4.0K nov 14 21:53 ..
 25M -rw-r--r-- 1 www-data www-data  25M feb 14 10:41 5e26cddbcf78d-14-Feb-20.flv
#

If I comment the redirect rule (http to https) in apache, All system works and the plugn SendRedordtoencoder come back to work. But I need to have it in order to solve another problem.

From


<VirtualHost *:80> 
    DocumentRoot "/var/www/html/cathotube.com/YouPHPTube" 
    ServerName cathotube.com 
    ServerAlias live.cathotube.com, s1.cathotube.com

    # Other directives here 
    RewriteEngine on
    RewriteCond %{SERVER_NAME} =cathotube.com 
    RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent] 
</VirtualHost>
<VirtualHost *:80>
    DocumentRoot "/var/www/html/cathotube.com/YouPHPTube-Encoder"
    ServerName encoder.cathotube.com
    # Other directives here
    RewriteEngine on
    RewriteCond %{SERVER_NAME} =encoder.cathotube.com
    RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>
<VirtualHost *:443>
    DocumentRoot "/var/www/html/cathotube.com/YouPHPTube-Encoder"
    ServerName encoder.cathotube.com
    # Other directives here
    Include /etc/letsencrypt/options-ssl-apache.conf
    SSLCertificateFile /etc/letsencrypt/live/cathotube.com-0001/fullchain.pem
    SSLCertificateKeyFile /etc/letsencrypt/live/cathotube.com-0001/privkey.pem
</VirtualHost>

to

<VirtualHost *:80> 
    DocumentRoot "/var/www/html/cathotube.com/YouPHPTube" 
    ServerName cathotube.com 
    ServerAlias live.cathotube.com, s1.cathotube.com

    # Other directives here 
    # RewriteEngine on
    # RewriteCond %{SERVER_NAME} =cathotube.com 
    # RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>
<VirtualHost *:80>
    DocumentRoot "/var/www/html/cathotube.com/YouPHPTube-Encoder"
    ServerName encoder.cathotube.com
    # Other directives here
    RewriteEngine on
    RewriteCond %{SERVER_NAME} =encoder.cathotube.com
    RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>
<VirtualHost *:443>
    DocumentRoot "/var/www/html/cathotube.com/YouPHPTube-Encoder"
    ServerName encoder.cathotube.com
    # Other directives here
    Include /etc/letsencrypt/options-ssl-apache.conf
    SSLCertificateFile /etc/letsencrypt/live/cathotube.com-0001/fullchain.pem
    SSLCertificateKeyFile /etc/letsencrypt/live/cathotube.com-0001/privkey.pem
</VirtualHost>
DanielnetoDotCom commented 4 years ago

Hi,

thanks for the info. I do not think that is the problem but change your onpublish to the same domain as your other on* directives removing the live.

on_publish http://cathotube.com/plugin/Live/on_publish.php;
on_play http://cathotube.com/plugin/Live/on_play.php;
on_record_done http://cathotube.com/plugin/Live/on_record_done.php;

but you can not redirect the on_* directives to https.

if you want you can use a separate domain just for the on directives or you can use the IP address and let apache redirect the domain.

on_publish http://192.166.150.7/plugin/Live/on_publish.php;
on_play http://192.166.150.7/plugin/Live/on_play.php;
on_record_done http://192.166.150.7/plugin/Live/on_record_done.php;
maxlinux2000 commented 4 years ago

That's Works!!! TNX!!