search

Rudloff / alltube

Web GUI for youtube-dl
GNU General Public License v3.0
2.96k stars 583 forks source link

Unable to download audio only from YT #136

Closed seanvree closed 6 years ago

seanvree commented 6 years ago

OS: Windows 10E x64 1709 Webserver: IIS 10 Python v: 2.7.14 youtube-dl v: 2017.10.20

Issue:

When I try to DL audio only from YouTube it DLs an empty 1kb MP3 file. I Can DL the video with audio fine.

No errors in the browser or in IIS.

@Rudloff I can give you access to my server so you can test if need be, hard to pinpoint without any errors, but there's literally no errors in IIS, or the browser.

Link tested: https://www.youtube.com/watch?v=YmDf2OokfFw&feature=youtu.be

FFMPEG:

C:\WINDOWS\system32>ffmpeg --version
ffmpeg version N-87353-g183fd30 Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 7.2.0 (GCC)
  configuration: --enable-gpl --enable-version3 --enable-cuda --enable-cuvid --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-nvenc --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib
  libavutil      55. 76.100 / 55. 76.100
  libavcodec     57.106.101 / 57.106.101
  libavformat    57. 82.101 / 57. 82.101
  libavdevice    57.  8.101 / 57.  8.101
  libavfilter     6.105.100 /  6.105.100
  libswscale      4.  7.103 /  4.  7.103
  libswresample   2.  8.100 /  2.  8.100
  libpostproc    54.  6.100 / 54.  6.100

My config looks like this:

youtubedl: C:\Program Files (x86)\YoutubeDLG\bin\youtube-dl
python: C:\Python27\python.exe
params:
    - --no-playlist
    - --no-warnings
    - --playlist-end
    - 1
convert: true
avconv: C:\FFMPEG-win64\bin\ffmpeg.exe
rtmpdump: C:\rtmpdump\rtmpdump.exe
uglyUrls: true
stream: true
remux: true
Rudloff commented 6 years ago

Can you try running tests with composer exec -v phpunit and see if it outputs any useful error?

seanvree commented 6 years ago

@Rudloff I'm unfamiliar with that route and YT CLI in general. Can you give me a bit more ?

Rudloff commented 6 years ago

Sorry, I assumed you installed Alltube with Git/Composer. If not, you first need to install Composer. Then you need to run composer install and composer exec -v phpunit from the commandline when in Alltube's directory. It will execute various tests in order to find what part of the code fails to execute correctly.

seanvree commented 6 years ago

@Rudloff got it. Got Composer installed, but when I run it from within the Alltube Dir, I get the following:

C:\inetpub\wwwroot\alltube>composer --version
Composer version 1.5.2 2017-09-11 16:59:25

C:\inetpub\wwwroot\alltube>composer install
Loading composer repositories with package information
Installing dependencies (including require-dev) from lock file
Package operations: 9 installs, 3 updates, 0 removals
  - Installing ffmpeg/ffmpeg (dev-release): Downloading (100%)

  [RuntimeException]
  Failed to execute tar -xJf "C:\inetpub\wwwroot\alltube\vendor/ffmpeg/ffmpeg/ffmpeg-release-64bit-static.tar.xz" -C
  "C:\inetpub\wwwroot\alltube/vendor/composer/589f4a9b"
  tar (child): C\:\\inetpub\\wwwroot\\alltube\\vendor/ffmpeg/ffmpeg/ffmpeg-release-64bit-static.tar.xz: Cannot open:
  Input/output error
  tar (child): Error is not recoverable: exiting now
  xz: (stdin): File format not recognized
  tar: Child returned status 2
  tar: Error is not recoverable: exiting now

install [--prefer-source] [--prefer-dist] [--dry-run] [--dev] [--no-dev] [--no-custom-installers] [--no-autoloader] [--no-scripts] [--no-progress] [--no-suggest] [-v|vv|vvv|--verbose] [-o|--optimize-autoloader] [-a|--classmap-authoritative] [--apcu-autoloader] [--ignore-platform-reqs] [--] [<packages>]...

C:\inetpub\wwwroot\alltube>
Rudloff commented 6 years ago

Hum, that's strange. It seems that for some reason the tar binary on your system can't open xz files.

Alternatively, you can download phpunit-6.4.phar manually from here and just run php phpunit-6.4.phar in Alltube's directory.

You might also need to add these paths to config/config_test.yml (since by default tests use Linux binaries in vendor/):

youtubedl: C:\Program Files (x86)\YoutubeDLG\bin\youtube-dl
python: C:\Python27\python.exe
avconv: C:\FFMPEG-win64\bin\ffmpeg.exe
rtmpdump: C:\rtmpdump\rtmpdump.exe
seanvree commented 6 years ago

@Rudloff What am I doing wrong here?

C:\WINDOWS\system32>cd C:\inetpub\wwwroot\alltube

C:\inetpub\wwwroot\alltube>php -v
PHP 7.1.11 (cli) (built: Oct 25 2017 20:54:15) ( NTS MSVC14 (Visual C++ 2015) x64 )
Copyright (c) 1997-2017 The PHP Group
Zend Engine v3.1.0, Copyright (c) 1998-2017 Zend Technologies

C:\inetpub\wwwroot\alltube>php phpunit-6.4.phar
Could not open input file: phpunit-6.4.phar

C:\inetpub\wwwroot\alltube>

image

Rudloff commented 6 years ago

Sorry, I made a typo, the file is called phpunit-6.4.4.phar. However I just tested on Windows and our tests don't run correctly so that won't be really helpful.

I guess we are back to square one... Are you sure you don't have any PHP error in your logs? You can check if your PHP install uses a separate log file by creating a PHP file containing this and running it on your server:

<?php
var_dump(ini_get('error_log'));
seanvree commented 6 years ago

@Rudloff well good to know I wasn't going crazy!

Correct, NO errors from the webserver or PHP. PHP is writing to syslog. see the bottom test of the phpinfo.ini file on the right frame:

string(6) "syslog"

image

seanvree commented 6 years ago

@Rudloff also, here is my full phpinfo config file. LMK when you take a look so I can add AUTH back on

seanvree . com / php /

seanvree commented 6 years ago

@Rudloff just to rule out errors not being logged, I changed the logging to a static file, and ran an error via PHP to see if it was logged, and it is being logged (as you see below). So again, there are literally NO erros anywhere when I try to DL with audio only. 

Nov 10 23:05:28 PHP php_errors.log: [10-Nov-2017 23:05:28 US/Pacific] PHP Notice:  A non well formed numeric value encountered in C:\inetpub\wwwroot\Monitorr\assets\php\functions.php on line 180
Nov 10 23:05:28 PHP php_errors.log: [10-Nov-2017 23:05:28 US/Pacific] PHP Notice:  A non well formed numeric value encountered in C:\inetpub\wwwroot\Monitorr\assets\php\functions.php on line 182
Nov 10 23:05:32 PHP php_errors.log: [10-Nov-2017 23:05:31 US/Pacific] PHP Notice:  A non well formed numeric value encountered in C:\inetpub\wwwroot\Monitorr\assets\php\functions.php on line 176
Nov 10 23:05:32 PHP php_errors.log: [10-Nov-2017 23:05:31 US/Pacific] PHP Notice:  A non well formed numeric value encountered in C:\inetpub\wwwroot\Monitorr\assets\php\functions.php on line 178
Nov 10 23:05:32 PHP php_errors.log: [10-Nov-2017 23:05:31 US/Pacific] PHP Notice:  A non well formed numeric value encountered in C:\inetpub\wwwroot\Monitorr\assets\php\functions.php on line 180
Nov 10 23:05:32 PHP php_errors.log: [10-Nov-2017 23:05:31 US/Pacific] PHP Notice:  A non well formed numeric value encountered in C:\inetpub\wwwroot\Monitorr\assets\php\functions.php on line 182
Nov 10 23:05:33 PHP php_errors.log: [10-Nov-2017 23:05:33 US/Pacific] PHP Notice:  A non well formed numeric value encountered in C:\inetpub\wwwroot\Monitorr\assets\php\functions.php on line 176
Rudloff commented 6 years ago

I had a look at your phpinfo but did not find anything strange (but again, I'm not used to running PHP on Windows). I also ran some tests on Windows but could not reproduce your issue.

Here is one last thing you can try before I give up. Make theses changes to the code:

diff --git a/classes/VideoDownload.php b/classes/VideoDownload.php
index a1c45a2..7f5db26 100644
--- a/classes/VideoDownload.php
+++ b/classes/VideoDownload.php
@@ -272,15 +272,15 @@ class VideoDownload
         $builder = ProcessBuilder::create(
             [
                 $this->config->avconv,
-                '-v', 'quiet',
                 //Vimeo needs a correct user-agent
                 '-user-agent', $this->getProp(null, null, 'dump-user-agent'),
                 '-i', $url,
                 '-f', 'mp3',
                 '-vn',
-                'pipe:1',
+                'test.mp3',
             ]
         );
+        var_dump($builder->getProcess()->getCommandLine()); die;

         return $builder->getProcess();
     }

Then try to convert your Youtube video in the browser. It should output a string containing a commandline script that runs ffmpeg. You can then try to copy this script in your terminal in order to see why it fails.

seanvree commented 6 years ago

@Rudloff wow, I thought that would never work, but it worked perfect.

Changed the .php file, got the following output in the browser:

string(907) "C:\FFMPEG-win64\bin\ffmpeg.exe -user-agent "Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)" -i "https://r4---sn-nx5e6n76.googlevideo.com/videoplayback?gir=yes&expire=1510515243&clen=63274484&pl=26&keepalive=yes&requiressl=yes&ip=2601"^%"3A602"^%"3A9a00"^%"3A3bb3"^%"3A39fb"^%"3A7b5c"^%"3Acafe"^%"3Aadf8&ei=yk0IWuHNPMTh-gOfto7ICw&sparams=clen"^%"2Cdur"^%"2Cei"^%"2Cgir"^%"2Cid"^%"2Cinitcwndbps"^%"2Cip"^%"2Cipbits"^%"2Citag"^%"2Ckeepalive"^%"2Clmt"^%"2Cmime"^%"2Cmm"^%"2Cmn"^%"2Cms"^%"2Cmv"^%"2Cpl"^%"2Crequiressl"^%"2Csource"^%"2Cexpire&id=o-APoj9nqsWjk1bJU7RW7PcL2DfxB_nH0OZ9xQnq4ODPmO&initcwndbps=1426250&mn=sn-nx5e6n76&mm=31&source=youtube&dur=4044.761&mv=m&mt=1510493554&ms=au&key=yt6&itag=171&mime=audio"^%"2Fwebm&ipbits=0&lmt=1397837085228841&signature=5350AA65440AC72D8EB426F1F40710B62E459E60.E29AD9CC66F72AD0D9BB0F235A43EAE8EC526271&ratebypass=yes" -f mp3 -vn test.mp3"

Ran the following script in the console:

C:\FFMPEG-win64\bin>ffmpeg.exe -user-agent "Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)" -i "https://r4---sn-nx5e6n76.googlevideo.com/videoplayback?gir=yes&expire=1510515243&clen=63274484&pl=26&keepalive=yes&requiressl=yes&ip=2601"^%"3A602"^%"3A9a00"^%"3A3bb3"^%"3A39fb"^%"3A7b5c"^%"3Acafe"^%"3Aadf8&ei=yk0IWuHNPMTh-gOfto7ICw&sparams=clen"^%"2Cdur"^%"2Cei"^%"2Cgir"^%"2Cid"^%"2Cinitcwndbps"^%"2Cip"^%"2Cipbits"^%"2Citag"^%"2Ckeepalive"^%"2Clmt"^%"2Cmime"^%"2Cmm"^%"2Cmn"^%"2Cms"^%"2Cmv"^%"2Cpl"^%"2Crequiressl"^%"2Csource"^%"2Cexpire&id=o-APoj9nqsWjk1bJU7RW7PcL2DfxB_nH0OZ9xQnq4ODPmO&initcwndbps=1426250&mn=sn-nx5e6n76&mm=31&source=youtube&dur=4044.761&mv=m&mt=1510493554&ms=au&key=yt6&itag=171&mime=audio"^%"2Fwebm&ipbits=0&lmt=1397837085228841&signature=5350AA65440AC72D8EB426F1F40710B62E459E60.E29AD9CC66F72AD0D9BB0F235A43EAE8EC526271&ratebypass=yes" -f mp3 -vn test.mp3

Output

ffmpeg version N-87353-g183fd30 Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 7.2.0 (GCC)
  configuration: --enable-gpl --enable-version3 --enable-cuda --enable-cuvid --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-nvenc --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib
  libavutil      55. 76.100 / 55. 76.100
  libavcodec     57.106.101 / 57.106.101
  libavformat    57. 82.101 / 57. 82.101
  libavdevice    57.  8.101 / 57.  8.101
  libavfilter     6.105.100 /  6.105.100
  libswscale      4.  7.103 /  4.  7.103
  libswresample   2.  8.100 /  2.  8.100
  libpostproc    54.  6.100 / 54.  6.100
[https @ 0000000002e18720] the user-agent option is deprecated, please use user_agent option
Input #0, matroska,webm, from 'https://r4---sn-nx5e6n76.googlevideo.com/videoplayback?gir=yes&expire=1510515243&clen=63274484&pl=26&keepalive=yes&requiressl=yes&ip=2601%3A602%3A9a00%3A3bb3%3A39fb%3A7b5c%3Acafe%3Aadf8&ei=yk0IWuHNPMTh-gOfto7ICw&sparams=clen%2Cdur%2Cei%2Cgir%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Ckeepalive%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Crequiressl%2Csource%2Cexpire&id=o-APoj9nqsWjk1bJU7RW7PcL2DfxB_nH0OZ9xQnq4ODPmO&initcwndbps=1426250&mn=sn-nx5e6n76&mm=31&source=youtube&dur=4044.761&mv=m&mt=1510493554&ms=au&key=yt6&itag=171&mime=audio%2Fwebm&ipbits=0&lmt=1397837085228841&signature=5350AA65440AC72D8EB426F1F40710B62E459E60.E29AD9CC66F72AD0D9BB0F235A43EAE8EC526271&ratebypass=yes':
  Metadata:
    encoder         : google
  Duration: 01:07:24.76, start: 0.000000, bitrate: 125 kb/s
    Stream #0:0(eng): Audio: vorbis, 44100 Hz, stereo, fltp (default)
Stream mapping:
  Stream #0:0 -> #0:0 (vorbis (native) -> mp3 (libmp3lame))
Press [q] to stop, [?] for help
Output #0, mp3, to 'test.mp3':
  Metadata:
    TSSE            : Lavf57.82.101
    Stream #0:0(eng): Audio: mp3 (libmp3lame), 44100 Hz, stereo, fltp (default)
    Metadata:
      encoder         : Lavc57.106.101 libmp3lame
size=   63201kB time=01:07:24.80 bitrate= 128.0kbits/s speed=52.4x
video:0kB audio:63200kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000391%

C:\FFMPEG-win64\bin>

MP3 downloaded perfectly.

So where does that leave us? It's not FFMPEG that's the problem then, right?

image

Rudloff commented 6 years ago

I expected ffmpeg to output some error. It's really strange that it works when calling it directly but not when called from PHP.

Here is another test you can run (edit the code then try to convert something and see if an error is displayed in the browser):

diff --git a/classes/VideoDownload.php b/classes/VideoDownload.php
index a1c45a2..034309a 100644
--- a/classes/VideoDownload.php
+++ b/classes/VideoDownload.php
@@ -272,7 +272,6 @@ class VideoDownload
         $builder = ProcessBuilder::create(
             [
                 $this->config->avconv,
-                '-v', 'quiet',
                 //Vimeo needs a correct user-agent
                 '-user-agent', $this->getProp(null, null, 'dump-user-agent'),
                 '-i', $url,
@@ -281,6 +280,10 @@ class VideoDownload
                 'pipe:1',
             ]
         );
+        $process = $builder->getProcess();
+        $process->mustRun();
+        var_dump($process->getOutput(), $process->getErrorOutput());
+        die;

         return $builder->getProcess();
     }

(Warning: this can run for a long time and return an huge output if the conversion works correctly. But since it does not work on your server, it will probably end quickly and return an error or nothing.)

seanvree commented 6 years ago

@Rudloff okay! NOW we are getting somewhere.

So, normally, when I try to DL the .mp3, it DLs a MP3 file RIGHT AWAY, but it's only 1kb. NOW, after editing the code, it didn't download anything, however FFMPEG did launch in the task manager, PHP threw this error right away in it's logs:


Nov 12 06:52:46 PHP php_errors.log: [12-Nov-2017 06:52:45 US/Pacific] PHP Notice:  Object of class Symfony\Component\Process\Process could not be converted to int in C:\inetpub\wwwroot\alltube\classes\VideoDownload.php on line 283

And then got this error in the browser:

The process "C:\FFMPEG-win64\bin\ffmpeg.exe -user-agent "Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)" -i "https://r4---sn-nx5e6n76.googlevideo.com/videoplayback?ip=2601"^%"3A602"^%"3A9a00"^%"3A3bb3"^%"3A39fb"^%"3A7b5c"^%"3Acafe"^%"3Aadf8&mv=m&sparams=clen"^%"2Cdur"^%"2Cei"^%"2Cgir"^%"2Cid"^%"2Cinitcwndbps"^%"2Cip"^%"2Cipbits"^%"2Citag"^%"2Ckeepalive"^%"2Clmt"^%"2Cmime"^%"2Cmm"^%"2Cmn"^%"2Cms"^%"2Cmv"^%"2Cpl"^%"2Crequiressl"^%"2Csource"^%"2Cexpire&mt=1510498474&ms=au&mn=sn-nx5e6n76&mm=31&keepalive=yes&id=o-AOL3V2On4OYn-TacM4n6QznKxFycN2HDTNUp0qBzhRl5&clen=63274484&source=youtube&pl=26&itag=171&ei=E2EIWrXzN8P6-wPdl55I&lmt=1397837085228841&dur=4044.761&ipbits=0&expire=1510520180&requiressl=yes&initcwndbps=1528750&key=yt6&gir=yes&mime=audio"^%"2Fwebm&signature=36A0769AD1343E5662A6023BD2B0AAB29FA39CEC.1149C7E9FD2826D45CC1F24EC992217D0B48B3D9&ratebypass=yes" -f mp3 -vn pipe:1" exceeded the timeout of 60 seconds.
Rudloff commented 6 years ago

This keeps getting stranger and stranger... Thanks a lot for your patience. This one is really hard to diagnosticate.

It seems that ffmpeg can indeed be executed from PHP when called with Process. I guess it only fails when called with popen().

Can you please create a test.php file in Alltube's folder with this content:

<?php
use Alltube\Config;
use Alltube\VideoDownload;

require_once __DIR__.'/vendor/autoload.php';

header('Content-Type: text/plain');

$config = Config::getInstance();
$downloader = new VideoDownload();

$stream = $downloader->getAudioStream('https://www.youtube.com/watch?v=dQw4w9WgXcQ', 'worst');
var_dump($stream);
var_dump(fread($stream, 2096));
var_dump(feof($stream));

echo PHP_EOL;

$stream = popen($config->avconv.' -version', 'r');
var_dump($stream);
var_dump(fread($stream, 2096));
var_dump(feof($stream));

then run it in your browser and see what the output is.

seanvree commented 6 years ago

I"m assuming you mean to add a " ?> " at the end? Also, the DIR " \Alltube\VideoDownload" doesn't exist in my alltube root Dir. either way, it downloaded text.php, within that downloaded text.php was a bunch of garbage:

string(848909) "ID3     TXXX     major_brand mp42 TXXX     minor_version 0 TXXX     compatible_brands isommp42 TSSE     Lavf57.82.101           ÿó@Ä   H    LAME3.99.5UUUUUUUUUäj‹EbÏ딺U¨‰á„A÷’LÓ^ÐËÿÿÿi´ddšLÿÿÿêìì840xáãƒAG,840xçgc‘ŸS3‘ŸÿóBÄ[  H    ÙÉëg"h¨ÎÝÌìÎÏvrÑQLQãǐ1eÝþªË*@   ™GLÏàs)=í¢#¹4ÊBö";Ýå§~ É I2tBAÒ•š OMÏˉÄÍ¿õÍ?@„æñ*S•çÿó@Ĥ  H    ÿ¼Oýøý~»ºzÿ¥.^‘Ÿõ2çåÁðÁB‚Áš|
"—Œh¾c§Ä¶Má–ãÑÔ"³å°N
a ]tn7¡£x®!L[.€©AóÊ
Ê”ÿóBÄÿËD²ÀJ¹TáÒc¤FQ·4‘LãΨ­‚¢"¥âÒ   Ä9T+¤›,=–Øl€¹M¿¤¨¹&+7ZI3h_Î
8)GžLÕ#_ÕÆ^2«Qš„ö® Ä˜¹&¸BïÛvþ§ÿó@ÄóbZ,a†~‚9ôÖ7Š'U‰U^¤²ÙPEÅDAQŒsPùyûµU1z..ª¢æ÷ÜÞ—ÿ\Õº'‘ŠF¸¶æ‹[}ÆÊVSÇ1b5eõr’S¤{[}…+mkPõו@t¿ÿóBÄð'Ë:<
Ã
Ò2¯³¸<jÄhŸ5i¶ÜÛß붡Z$z¿˜Ù„±îQ®­ŸKÎòÞu¦!¥e•×È]Okû<Ëôß[FE¶n7çàUfi+šºÆB! ÌV
ƒB¯©£\W?p?ÿó@Ĭ%k\_H`†oæEKÙ‘Ó&Bè“ž<Ü ˆâoHO {—ÈEƒÇåór‘Ds2`—
"iüÝ
èàŸ› Ùÿô™º ¦h_.J‡±&HÿýH Šfè&n£CÎl‰}ËÿóBÄq%ëÆò_h‰s$ɦ_ÿþ‚ÝÿØÑ^jp˜b|ÜÁÒÿÿùMÍÿüà%ðP-,ÉC躔–±FΙVföH Ö,¹¢œhô.
ƒn-fx¹éV‰ãÿó@Ä5ÚjÒAÏ@Ú.i85gµ©¶©ÊeuÅgU”Ð}ær]®æâÜX«*2FÏ‹˜= yC*ýø¸qËAëoÉV(˜S¶±©¡FSØóP»J VMP3šzxø)Eå–NJu“EÿóBÄË6Ò$a†UÓt§'‰Ì£Š«¾Ž½Y¶nŽ^Æ>p¼ØÊ1ðôŒ`çÔ2‰bUfuïÄÞKm2Ø¥šW+sÊ‘Nùj§«;’妾Z®_ùý\þXÃd‰Ucglrj‹c*«˜>•ÿó@ÄÒ®ÚT`ÄðB‚Ðÿ1T2²8RÞß«e“
§° †û\Ïdè‹Ö¬Kœ¤*µùž}Yµ[3£JéÏÓ~IEc“y
ý;¼³v™tý)¡…°Ñ×öè‡ä•  ºW ø·ÉÿóBÄÑšÊ,HFôBâ#n(i–Ç^ÂÔ‡Q‘š
Ñ"E䔑"ÀØ0Åñ³9Ì–å/•áG*bYau«©(Hle©cl_KEjÈMëazÕÑBþTAD¨q`
ˆŒ!ÚFÿó@Äð‚Ù–Ì:!iÈ¤¢€R úŠ Ä¢ ƒéÜ½¢¥âšV8¢ ù†Áó_ŸVì>ñ É
y@M¤(€?`©À¬LÖ&(2)gˆlÈÀ@Ly*G((Œêù?ÌŠÿóBÄ"fÑŠF ªú’ÓRÖX\þù/
x$,@ÀK¹hdä4¬&É$2Û ¢ÍAQH‹Ã]¢®òj‰Ÿ¾¡
5œ2 ç<Í©f¹&¼‘gPÆDymÇU2gVŠ¯Ì`ª½à–rwkÿó@Ä-ªÉ”@Äô8²_ë²ÕžÚ*6Ñg)Fã'«IûîwYÕ#ãJÜÊÿÅÂU`BWl
™©‘YB‡ÌÊì|—³m{`Óašz{ìQn   Œ¹§™µU½Î›•ù÷“ý×Óìò÷xP÷ÿóBÄ7êÆ,I†l…-òü±kº
kÑÈkO¨»ÁÉXÊÙšUª`@Ÿý¤§„]Àà‹ýæ‹&»ìlÃÆ2FaL[Féz.&"’è>בGÅbÅB„u„Zö­‡“¿…¹ÿó@ÄCØÊÎ6†‚Ú8¡•¤_óJÓ¢£7)NœÅ&[@…5Ei'(D …•êÑ*ʤ¶…¹I秿ChU_
ÛBêАօ·(`ᵋ¡Õ“@Ú…‘4LSO*¥ÿóBÄN‘"ɶI†ptýj@s1áâÏ’šñ~3kŠ8V‘'5\j‡QÌŠ»\Cã<1QðûHôï—Õ”·Õ{1ªÌé=ÕÆØa`  0ÐUÑ   ¢â3ó¿Ÿè[}…jÁ"Ò­ò©Lÿó@ÄWÑÖµDHDèšzý*´Tå%+a)­(1’Y½‚™5$γ>),¦eUØñï¦uf­yõ¹eEÌ,¨hµíjžTƒ{krÊÙžm¾l¹jBa ª=ÌŒ 8€rHŠÿóBÄb¡
ÆV
F)Èì«/Á…éÊL( [<ÊæJDü»¡@òžÌhb©X‘4LˉEAÖ ‹R·^ÌÛäGïtU
bU·²¾§æ*›r5.^°VZéx)ÔܽDÇsfhëõÿó@Äkq"±ŽI†PÕ´Î
nÀ…«–¢¨!YÑpÓV€˜`ö((°®=aÔ9«T’" 3‚FL]E"§Øõ–‰]QFÊ#lˆIðìµÑúv^¯÷ù”j= K‹S{
Rudloff commented 6 years ago

?> at the end of a PHP file is optional 😉

This is not garbage, it's the beginning of the converted MP3 file! This means the conversion does work when called this way. It also means we are getting closer to the real problem.

Here is another test for you:

<?php
use Alltube\Config;
use Alltube\VideoDownload;
use Slim\Http\Stream;

require_once __DIR__.'/vendor/autoload.php';

header('Content-Type: text/plain');

$config = Config::getInstance();
$downloader = new VideoDownload();

$process = $downloader->getAudioStream('https://www.youtube.com/watch?v=dQw4w9WgXcQ', 'worst');
$stream = new Stream($process);
var_dump($stream->getSize());
var_dump($stream->isPipe());
var_dump($stream->read(3));
var_dump($stream->eof());

(Please copy the whole output.)

seanvree commented 6 years ago

@Rudloff Too big to post here.

See here:

https://gist.githubusercontent.com/seanvree/924574c8960dec9c51262611c3bb5dcb/raw/bd0bd8294e49d0b33282a2287a2dfa7f89617ab4/alltube_test2.php

Rudloff commented 6 years ago

Does your test file only contain the latest test I gave you? It is not supposed to return this many characters.

seanvree commented 6 years ago

they are separate. created two files, test1.php and test2.php. Test1 had the first script you posted, and test2 had only the 2nd script you posted.

you can see here:

sean vree . com /alltube / test1.php seanvree . com / alltube / test2.php

Rudloff commented 6 years ago

Oh sorry, I know what's going on here. You need to revert the changes you made in classes/VideoDownload.php first. I totally forgot to tell you. Then try test1.php and test2.phpagain.

seanvree commented 6 years ago

@Rudloff no problem

test1.php gave me this:

resource(39) of type (stream)
string(138) "ID3     TXXX     major_brand mp42 TXXX     minor_version 0 TXXX     compatible_brands isommp42 TSSE     Lavf57.82.101           "
bool(false)

resource(40) of type (stream)
string(1303) "ffmpeg version N-87353-g183fd30 Copyright (c) 2000-2017 the FFmpeg developers
built with gcc 7.2.0 (GCC)
configuration: --enable-gpl --enable-version3 --enable-cuda --enable-cuvid --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-nvenc --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib
libavutil      55. 76.100 / 55. 76.100
libavcodec     57.106.101 / 57.106.101
libavformat    57. 82.101 / 57. 82.101
libavdevice    57.  8.101 / 57.  8.101
libavfilter     6.105.100 /  6.105.100
libswscale      4.  7.103 /  4.  7.103
libswresample   2.  8.100 /  2.  8.100
libpostproc    54.  6.100 / 54.  6.100
"
bool(false)

Test2.php gave me this:

NULL
bool(true)
string(3) "ID3"
bool(false)
Rudloff commented 6 years ago

The results are all correct. And test2 is almost the same code that Alltube calls when converting a video so I really don't know why it works in the test but not in the app. I'm sorry but I'm out of ideas here.

(It might be a very specific quirk in your PHP config or a bug in the Slim framework.)

seanvree commented 6 years ago

@Rudloff Damn, that sucks. All good brother. I'll keep an eye on this.

Did you by chance test this from my server in your browser just to see the issue I'm having? There's no auth on it right now, so you can test if you'd like.

sean vree . com / all tube

Really appreciate your help.

Rudloff commented 6 years ago

I did some more tests with your server and I finally found the issue. Here are the headers returned by your server:

< HTTP/1.1 200 OK
< Cache-Control: no-store, no-cache, must-revalidate
< Pragma: no-cache
< Content-Type: audio/mpeg
< Expires: Thu, 19 Nov 1981 08:52:00 GMT
< Server: Microsoft-IIS/10.0
< X-Powered-By: PHP/7.1.11
< Set-Cookie: PHPSESSID=xxx; path=/; HttpOnly
< Content-Disposition: attachment; filename="Rick Astley - Never Gonna Give You Up-dQw4w9WgXcQ.mp3"
<  Access-Control-Allow-Origin: Access-Control-Allow-Origin: xxx
< X-Powered-By: ASP.NET
< Date: Mon, 13 Nov 2017 13:50:12 GMT
< Content-Length: 110

The Content-Length header should not be there, as Alltube does not add it when converting. It makes the browser think that the download is only 110 bytes long, so it explains why it stops immediately. The X-Powered-By: ASP.NET is also strange since Alltube does not use ASP at all. So it seems something in your webserver is adding additional headers to the response. This confirms it is not an Alltube bug but rather an issue with your IIS config.

seanvree commented 6 years ago

Ahhh, okay, I think I know how to fix this! Thanks, I'll get back to you!

seanvree commented 6 years ago

@Rudloff . Looks like you're right! I think I got it. Question - do you recommend static and/or dynamic compression enabled?

Rudloff commented 6 years ago

🎉 This must be the longest thread we ever had here. I'm glad we finally found a solution!

I would recommend compressing static content but not dynamic content.

seanvree commented 6 years ago

ha, yeah brotha. sorry about that. I'm the kind of guy who doesn't give up. Appreciate you hanging with me. I'm sure maybe you learned something?

One last question...how did you get that header info for the actual MP3 you tried to DL? I'm using Chrome to debug, and when I download the file, I don't see the header pop up.

image

Rudloff commented 6 years ago

Yes, this issue helped me find the motivation to make our automated tests work on Windows (https://github.com/Rudloff/alltube/commit/2432a06c1bdce7cf05d778f5b114278ce1bb1e61) 😄

I used curl in order to get the headers:

curl -vvv "http://example.com/alltube/?url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DdQw4w9WgXcQ&page=video&audio=on"
seanvree commented 6 years ago

@Rudloff got it. thanks.

seanvree commented 6 years ago

@Rudloff Okay, did some research and some configuring.

Turns out that that value is NOT defining the max size of content that is allowable to transfer, rather the actual bytes transferred.

For IIS and the ALLTUBE sub-applicaiton I have the max content-length set to 30000000 bytes. see screenshot below:

            <requestFiltering>
                <fileExtensions>
                    <add fileExtension=".mp3" allowed="true" />
                    <add fileExtension=".mp4" allowed="true" />
                </fileExtensions>
                <requestLimits maxAllowedContentLength="300000000">
                    <headerLimits>
                        <add header="maxAllowedContentLength" sizeLimit="30000000" />
                        <remove header="Content-Length" />
                    </headerLimits>
                </requestLimits>
            </requestFiltering>

Now, I DO NOT think that this necessarily a problem with YOUR app, maybe curl? Maybe PHP? But also, this ONLY happens when I"m converting from video to audio. If content-length was an issue in IIS, wouldn't I NOT be able to download anything? Also, I am able to convert videos to a smaller size, and change format - MP4 to REMUX, etc. I mean, I download videos and convert videos larger than 1GB all the time, it's ONLY when trying to convert to audio only. So, that's what tells me that this may still be a conversion problem. Maybe a combination of a lot of things - php, curl, conversion, IIS. Also, i do run a FULL front-end to my PLEX server that includes A LOT of php, curl and downloading stuff from the IIS server, (over 20+ web apps) and I've never seen this issue.

Anyway, I'm pretty tired of this issue, as I"m sure you are. Just wanted to put that out there.

Thanks for all your help!

image