nadermx / backgroundremover

Background Remover lets you Remove Background from images and video using AI with a simple command line interface that is free and open source.
https://www.backgroundremoverai.com
MIT License
6.46k stars 538 forks source link

-tov/--transparentvideoovervideo: invalid <lambda> value: #28

Closed j2l closed 2 years ago

j2l commented 2 years ago

Hi, I tried multiple way to get a -tv working an example video to work with a background and it always throws the error in the title.

For instance, backgroundremover -i "./stress.mp4" -tov "vert.mp4" -o "output.mp4" results in: backgroundremover: error: argument -tov/--transparentvideoovervideo: invalid <lambda> value: 'vert.mp4'

Weirdly enough backgroundremover -i "./stress.mp4" -tov -o "output.mp4" "works" ... but there's no overlay video input so I get: /<stdin>: No such file or directory Process finished

Any idea?

celikmustafa89 commented 2 years ago

same issue:

backgroundremover -i 15_terminator.mp4 -tov green.mp4 -o g_out.mov usage: backgroundremover [-h] [-m {u2net,u2net_human_seg,u2netp}] [-a [ALPHA_MATTING]] [-af ALPHA_MATTING_FOREGROUND_THRESHOLD] [-ab ALPHA_MATTING_BACKGROUND_THRESHOLD] [-ae ALPHA_MATTING_ERODE_SIZE] [-az ALPHA_MATTING_BASE_SIZE] [-wn WORKERNODES] [-gb GPUBATCHSIZE] [-fr FRAMERATE] [-fl FRAMELIMIT] [-mk [MATTEKEY]] [-tv [TRANSPARENTVIDEO]] [-tov [TRANSPARENTVIDEOOVERVIDEO]] [-toi [TRANSPARENTVIDEOOVERIMAGE]] [-tg [TRANSPARENTGIF]] [-tgwb [TRANSPARENTGIFWITHBACKGROUND]] [-i [INPUT]] [-bi [BACKGROUNDIMAGE]] [-bv [BACKGROUNDVIDEO]] [-o [OUTPUT]] backgroundremover: error: argument -tov/--transparentvideoovervideo: invalid <lambda> value: 'green.mp4'

j2l commented 2 years ago

So far, I didn't find a way to make it work and python code looks the same as -tv so I don't see why the file is not loaded as overlay. Let me know @celikmustafa89 if you find some way :wink:

celikmustafa89 commented 2 years ago

So far, I didn't find a way to make it work and python code looks the same as -tv so I don't see why the file is not loaded as overlay. Let me know @celikmustafa89 if you find some way 😉

Hi,

I found the way of running it.

backgroundremover -i terminator.mp4 -toi -bv green.mp4 -o out_vid.mov

however the output is not good. I was expecting the original video with the green background but it is a grayscale output. :S :S

image
celikmustafa89 commented 2 years ago

may be it is because of the version of FFmpeg ?? I do not know.

j2l commented 2 years ago

I had the same, its the alpha matte (mask for the alpha channel). For an image with the same size as the video it shrinks the video and only keep the matte: image

celikmustafa89 commented 2 years ago

I had the same, its the alpha matte (mask for the alpha channel). For an image with the same size as the video it shrinks the video and only keep the matte: image

how should we run the code to get a proper output? any idea?

j2l commented 2 years ago

I had the same, its the alpha matte (mask for the alpha channel). For an image with the same size as the video it shrinks the video and only keep the matte: image

how should we run the code to get a proper output? any idea?

No idea :confused:

celikmustafa89 commented 2 years ago

hey @j2l,

I have used following command to combine the mate, video and the background video.

ffmpeg -i video.mp4 -i matte.mp4 -i background.mp4 -filter_complex '[1][0]scale2ref[mask][main];[main][mask]alphamerge[vid];[2:v][vid]overlay[out]' -map [out] complete.mp4

It works but has some rescaling issues. But the main point the code should give a proper output. it is working for image but now working for the video. any idea @nadermx ?

by the way here is the output:

image
j2l commented 2 years ago

Your exact command worked for me!

vlcsnap-2022-03-29-15h37m59s453

celikmustafa89 commented 2 years ago

actually i have some rescale problem with original and background video. by the way my green background is not an image. it is a green backgorund video which shows only green. could you do the merge of original video, matte and a green.png (image)? please share the code?

j2l commented 2 years ago

I also used a green background video and applied your code with my videos: ffmpeg -i stress.mp4 -i matte.mp4 -i vert.mp4 -filter_complex '[1][0]scale2ref[mask][main];[main][mask]alphamerge[vid];[2:v][vid]overlay[out]' -map [out] complete.mp4

It doesn't fix the backgroundremove issue so I keep the issue open for @nadermx or someone who finds the bug. It's way above my skills :smile:

If I can help you more, let me know.

j2l commented 2 years ago

Hey,it also works with a PNG image: ffmpeg -i stress.mp4 -i matte.mp4 -i vert.png -filter_complex '[1][0]scale2ref[mask][main];[main][mask]alphamerge[vid];[2:v][vid]overlay[out]' -map [out] complete2.mp4 ("vert" is green in French)

celikmustafa89 commented 2 years ago

The following command generates a video that scale-up the video for the background image. It may help in some cases. At least work for me. I spent some time writing this command, just want to share it to save your time if you need it.

ffmpeg -i test_fightclub.mp4 -i fight_matte_image.mov -i green.png -filter_complex '[1][0]scale2ref[mask][main];[main][mask]alphamerge[vid];[vid][2:v]scale2ref[fg][bg];[bg][fg]overlay[out]' -map [out] fight_image_complete_scale2ref.mp4

my output was like this:

image

now I can generate following output with the code:

image
j2l commented 2 years ago

Cool! Thank you!

nadermx commented 2 years ago

I had this open on superuser as I had gotten stuck. I'll fix the '-tov' ffmpeg command and close this issue.

j2l commented 2 years ago

Thank you!