How to gather all necessary inputs for a custom image before running the model

LucBourrat1 commented 2 years ago

Is it possible to run the model on a RGB custom image without having the depth, or it is necessary to have the depth and thus we must use a predicted one?

In the first case, can you explain how to run the model without the depth.
In the second case, I used DPT model to predict the depth from the image, but it seems that my predicted depth does not have the right scale/format (the command cv2.imread with the arguments "cv2.IMREAD_ANYCOLOR | cv2.IMREAD_ANYDEPTH" does not work and does not remove the 3rd dimension and convert the image into 16bits). Can you explain the preprocess steps to perform on the predicted depth so that it can be used as input? Also the ground truth depth from scanNet has low values, between 0 and 18 for some images. The predicted depth from DPT has values between 0 and 1, can be transformed between 0 and 255, but what are the constraints on the values?

Also, globally, can you explain the preprocess that should be performed on each input before running the model, or things to check on the input to be sure they are valid?

If needed I can send the inputs I'm currently trying to use so you can see why it does not work.

vevenom commented 2 years ago

This issue was discussed per email.

dilpreet7365 commented 1 year ago

How can we run the model without depth images

vevenom / RoomLayout3D_RandC

How to gather all necessary inputs for a custom image before running the model #7