Open pkubiak opened 5 days ago
.take
@pkubiak yes, great issue and a clear bug. The obvious solution is either to switch to bilinear interpolation, or just to clamp the values to (0, 255) before we do the uint8 conversion. My preference is for clamping - give me a sec and I'll open the PR.
Wow - I had no idea bicubic was this bad. In your example, when we bicubic interpolate a 518x518 input image up to 1500x1500, the range of values goes from (0, 9.48) to (-0.79, 9.6) ! I expected little rounding errors but this is huge, and means that clamping might lose some information.
Talked about this with @rwightman and the conclusion was that clamping the post-interpolation distribution to the original range before rescaling + uint8 conversion is correct. @pkubiak would be you be willing to make a PR for that? If you don't have time, let me know and I'll put it on my list, but thank you for the clear and helpful issue either way!
@Rocketknight1: I see that in the meantime some other PR has fixed this problem, but in a different way than in your proposal: https://github.com/huggingface/transformers/pull/32550 🤷 It also introduced some breaking changes 🤷
@pkubiak understood! If you have any issues after that PR, can you raise them on that PR page and ping the authors?
System Info
transformers
version: 4.45.2Who can help?
Hi, In a
depth-estimation
pipeline, there is a code part that interpolate model predicted depth map to the size of original image and than render it as a pillow image: https://github.com/huggingface/transformers/blob/main/src/transformers/pipelines/depth_estimation.py#L118-L122Unfortunately, it does not take into account the fact that in some cases the value of bicubic interpolation may lie outside the range of the original nodal values. In this case it means that sometimes we can get a negative depth values.
Such situation is not handle properly on line 122, which result in
int8
overflow and strange edge artefacts in generated images.Error is model independent (tested on
depth-anything/Depth-Anything-V2-Small-hf
,depth-anything/Depth-Anything-V2-Base-hf
andIntel/dpt-large
).@amyeroberts @qubvel @Rocketknight1
Information
Tasks
examples
folder (such as GLUE/SQuAD, ...)Reproduction
Noticeable white border:
Expected behavior
I would like to get a depth map image without strange edge artefacts :)
Something like the following picture (effect of changing
bicubic
tobilinear
interpolation):