search

PeterL1n / BackgroundMattingV2-TensorFlow

TensorFlow implementation of Real-Time High-Resolution Background Matting
MIT License
78 stars 13 forks source link

Wrong results in Tensorflow JS #6

Closed FabioRomagnolo closed 2 years ago

FabioRomagnolo commented 2 years ago

Hi! I'm having very strange outputs in TensorflowJS w.r.t. the ones I obtain running locally the MattingRefine model.

Everything is fine if the refine mode is "Full" or I use only the MattingBase network.

But if I use in TensorflowJS the "sampling" and "thresholding" refine modes within the MattingRefine model I obtain wrong results, like showed in the figures. The images are simply the foreground output concatened with the alpha channel. Background and bottle are the same.

Tensorflow JS output (MattingRefine w/ thresholding mode)

Background matting webcam demo - Google Chrome 28_07_2022 11_42_43

Tensorflow output (MattingRefine w/ thresholding mode)

BackgroundMatting – tf_model_refinercustom py 28_07_2022 11_40_24

I think this is related to different implementations of operators between Keras and TensorflowJS. Do you have any idea where are the instructions wrongly interpreted?

Thanks for your help and congrats for your great job!

Notes: I converted the Keras model first to Saved Model format and then to TFJS using TFJS-converter.

FabioRomagnolo commented 2 years ago

I actually discovered that the problem was due to TensorflowJS not supporting the operation TensorScatterUpdate. So the reimplementation of this operation in JS was just wrong.

You can delete this issue if you want.