BrokenPipeError: [Errno 32] Broken pipe in Ubuntu 20.04

[vigneshsudhakar]

I am trying to add watermark image to a video and upload it to S3, since I was not able to upload a video directly to S3, what I am doing is saving the video in local and then upload the local file to S3

Same code works fine in windows desktop but error happens in Ubuntu server.

video = VideoFileClip(request_video_url)
logo = (ImageClip("logo.png")
        .resize(height=80)  # if you need to resize...
        .set_pos(("left", "bottom")))

final = CompositeVideoClip([video, logo])

final = final.set_duration(video.duration)
final.write_videofile(filename, audio=True)

I am getting an error while the execution reaches final.write_videofile(filename, audio=True)

I'm not sure if this is the whole error message or whether MoviePy is not giving the full error for some reason.

Traceback (most recent call last):

  File "/home/ubuntu/glimpse-flask/venv/lib/python3.8/site-packages/moviepy/video/io/ffmpeg_writer.py", line 136, in write_frame
    self.proc.stdin.write(img_array.tobytes())
BrokenPipeError: [Errno 32] Broken pipe

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ubuntu/glimpse-flask/professionaldetails/routes.py", line 175, in updateProfessionalDetailByProfessional
    status = UpdateProfessionalDetailByProfessional(professionalDetails, professionalId, profile_image, preview_video)
  File "/home/ubuntu/glimpse-flask/professionaldetails/Service/professional_details_service.py", line 292, in UpdateProfessionalDetailByProfessional
    watermark_and_upload_file_to_s3('Girls/videos/', preview_video, str(professionalDetails['url_friendly_name']), preview_video_url, fileName)
  File "/home/ubuntu/glimpse-flask/common/Service/AwsService.py", line 74, in watermark_and_upload_file_to_s3
    final.write_videofile(filename, audio=True)
  File "<decorator-gen-55>", line 2, in write_videofile
  File "/home/ubuntu/glimpse-flask/venv/lib/python3.8/site-packages/moviepy/decorators.py", line 54, in requires_duration
    return f(clip, *a, **k)
  File "<decorator-gen-54>", line 2, in write_videofile
  File "/home/ubuntu/glimpse-flask/venv/lib/python3.8/site-packages/moviepy/decorators.py", line 135, in use_clip_fps_by_default
    return f(clip, *new_a, **new_kw)
  File "<decorator-gen-53>", line 2, in write_videofile
  File "/home/ubuntu/glimpse-flask/venv/lib/python3.8/site-packages/moviepy/decorators.py", line 22, in convert_masks_to_RGB
    return f(clip, *a, **k)
  File "/home/ubuntu/glimpse-flask/venv/lib/python3.8/site-packages/moviepy/video/VideoClip.py", line 300, in write_videofile
    ffmpeg_write_video(self, filename, fps, codec,
  File "/home/ubuntu/glimpse-flask/venv/lib/python3.8/site-packages/moviepy/video/io/ffmpeg_writer.py", line 228, in ffmpeg_write_video
    writer.write_frame(frame)
  File "/home/ubuntu/glimpse-flask/venv/lib/python3.8/site-packages/moviepy/video/io/ffmpeg_writer.py", line 180, in write_frame
    raise IOError(error)
OSError: [Errno 32] Broken pipe

MoviePy error: FFMPEG encountered the following error while writing file liquids.mp4:

 b''

I am looking for any kind of help. 1) Upload directly to S3 without having to save in local 2) Fix this broken pipe error 3) Figure out why I am not getting the full error.

Any one of the above should sort me out, been stuck on this for a while now, and I am kind of new to this, any help would be appreciated.

Thanks in advance.

[mondeja]

request_video_url is an URL instead of a path to a local file, I suppose, right?

[vigneshsudhakar]

request_video_url is an URL instead of a path to a local file, I suppose, right?

Yes it is an URL, at least in my code. Works fine in Windows.

[mondeja]

Could you share the versions of moviepy and FFmpeg that you are using in both platforms?

[vigneshsudhakar]

Could you share the versions of moviepy and FFmpeg that you are using in both platforms?

On windows: imageio==2.9.0 imageio-ffmpeg==0.4.3 moviepy==1.0.3

On Linux server: imageio==2.9.0 imageio-ffmpeg==0.4.3 moviepy==1.0.3 ffmpeg version 4.2.4-1ubuntu0.1

[mondeja]

Could you share the version of FFmpeg in Windows? You should be able to see it executing ffmpeg in the console.

[vigneshsudhakar]

Could you share the version of FFmpeg in Windows? You should be able to see it executing ffmpeg in the console.

I don't have FFmpeg installed in windows. "'ffmpeg' is not recognized as an internal or external command, operable program or batch file."

[mondeja]

Okay, in that case, could you execute this? python3 -c "import imageio_ffmpeg as i;print(i.get_ffmpeg_version())"

[vigneshsudhakar]

python3 -c "import imageio_ffmpeg as i;print(i.get_ffmpeg_version())"

On windows: 4.2.2

Same command on Linux server: 4.2.2-static

[mondeja]

Could you double check if you are sharing the full traceback? The binary message returned by FFmpeg seems hidden:

MoviePy error: FFMPEG encountered the following error while writing file liquids.mp4:

b''

[tburrows13]

Could you double check if you are sharing the full traceback? The binary message returned by FFmpeg seems hidden:

MoviePy error: FFMPEG encountered the following error while writing file liquids.mp4:

b''

That just happens sometimes/for some errors. I'm not sure if it is because ffmpeg is genuinely not returning anything or if moviepy is just failing to parse it.

[vigneshsudhakar]

Could you double check if you are sharing the full traceback? The binary message returned by FFmpeg seems hidden:

MoviePy error: FFMPEG encountered the following error while writing file liquids.mp4:

b''

Yes, this is the full traceback I got, that is why I was wondering if there is any other way to get the full error message.

[mondeja]

The URL that you are using is http or https? If it is https, you should check if your FFmpeg version in Ubuntu server has been built with OpenSSL support.

[vigneshsudhakar]

The URL that you are using is http or https? If it is https, you should check if your FFmpeg version in Ubuntu server has been built with OpenSSL support.

It is https.

[mondeja]

Executing ffmpeg in Ubuntu, check if --enable-openssl appears in the banner.

[vigneshsudhakar]

The URL that you are using is http or https? If it is https, you should check if your FFmpeg version in Ubuntu server has been built with OpenSSL support.

Supported file protocols: Input: async bluray cache concat crypto data ffrtmphttp file ftp gopher hls http httpproxy https mmsh mmst pipe rtmp rtmps rtmpt rtmpts rtp sctp srtp subfile tcp tls udp udplite unix sftp Output: crypto ffrtmphttp file ftp gopher http httpproxy https icecast md5 pipe prompeg rtmp rtmps rtmpt rtmpts rtp sctp srtp tee tcp tls udp udplite unix sftp

Looks like https is supported by the version of ffmpeg I have in my Linux server

[vigneshsudhakar]

--enable-openssl

Nope

ffmpeg version 4.2.4-1ubuntu0.1 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-10ubuntu2)
  configuration: --prefix=/usr --extra-version=1ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-nvenc --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...

Use -h to get full help or, even better, run 'man ffmpeg'

[vigneshsudhakar]

Executing ffmpeg in Ubuntu, check if --enable-openssl appears in the banner.

@mondeja how do I enable openssl for ffmpeg. Tried few options listed in other sites, no luck.

[mondeja]

You need to compile FFmpeg yourself enabling openssl.

[vigneshsudhakar]

You need to compile FFmpeg yourself enabling openssl. @mondeja ffmpeg -i https://xyz.mp4 -i https://LogoTransparent.png -filter_complex "overlay=0:0" output.mp4

This works fine, only when I access through code it is throwing that errno32. If ssl was the issue, it shouldn't be working while I use command line as well right?

[Iamprashanth-1]

use (threads =4,ffmpeg_params =['-f', 'mp4']) these params in write_videofile function worked for me