search

rehg-lab / 3DShapeGen

Code for 3D Reconstruction of Novel Object Shapes from Single Images paper
MIT License
123 stars 19 forks source link

A question about image pre-process #14

Open liuzhengzhe opened 2 years ago

liuzhengzhe commented 2 years ago

Thanks for your great work! I have a question: I use "best_model_img_3DOF.pth.tar" to generate 3D shape from an image, and how to preprocess the image? I tried image=image/255.0, the result seems not correct. Can you help me?

Thanks with regards.

2 The result is: image

keerthan2 commented 2 years ago

@liuzhengzhe can you elaborate on what code did you run to get this output ?.

liuzhengzhe commented 2 years ago

Hi, Can you provide a simple script to generate 3D shape from a single image? Thanks very much.

Zhengzhe

RA Keerthan @.***> 于2022年1月17日周一 17:24写道：

Hey. Can you elaborate on what code did you run to get this output ?.

— Reply to this email directly, view it on GitHub https://github.com/rehg-lab/3DShapeGen/issues/14#issuecomment-1014307406, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB7WPDAGU3ET7LYHSO5VW5TUWPN57ANCNFSM5LEVNJTA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you authored the thread.Message ID: @.***>

keerthan2 commented 2 years ago

Well, I'm also stuck in the 3D generation part. The sdf values that I'm getting are not proper. The preprocessing steps mentioned by the author is as below (taken from dataloader.py).

img_transform = transforms.Compose([ transforms.Resize(input_size), transforms.ToTensor(), transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]) ])

You can check my version of a simple test code here which takes in input a image and outputs sdf values. The sdf values are by itself improper wherein the minimum sdf is positive!. I tried changing the dimensions of the cube, but still no luck.

@liuzhengzhe can you send the code that you used to get the result that you just posted ?.

ngailapdi commented 2 years ago

Hi, thank you for your interest in our work.

Shape models are very sensitive to input images (as shown in section 4.6 of the paper) and that’s why images generated with different rendering processes might cause the model to output low-quality results beyond the training set characteristics.

keerthan2 commented 2 years ago

Hi @ngailapdi, thank you for the response.

Do you have a test script that produces the 3D mesh from an image ?. The file I referenced in my earlier comment (which is here) is intended for that purpose, but I'm not getting the desired output. Maybe its because of the variance in rendering/lighting conditions of the image, but to be very sure, I would highly appreciate if you can maybe share a code / instructions on how to test your model to generate 3D mesh from a single image.

ngailapdi commented 2 years ago

Hi @keerthan2,

You should be able to directly run python eval.py after changing the paths in your config.py to point to where your inputs are and specify the test field In your json. This should work with any number of test images.

keerthan2 commented 2 years ago

Hi @ngailapdi,

I already tried that, but the minimum SDF value is still positive (which shouldn't be the case right ?). Since I was trying it out in colab, I got a permission denied error while doing executing computeMarchingCubes, which I dint bother to resolve since my SDF values dint seem to make sense in the first place. I used best_model_img_3DOF.pth.tar weight file.

Do you have any idea on what I might be missing or please let me know what additional information you need.

liuzhengzhe commented 2 years ago

Well, I'm also stuck in the 3D generation part. The sdf values that I'm getting are not proper. The preprocessing steps mentioned by the author is as below (taken from dataloader.py).

img_transform = transforms.Compose([ transforms.Resize(input_size), transforms.ToTensor(), transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]) ])

You can check my version of a simple test code here which takes in input a image and outputs sdf values. The sdf values are by itself improper wherein the minimum sdf is positive!. I tried changing the dimensions of the cube, but still no luck.

@liuzhengzhe can you send the code that you used to get the result that you just posted ?.

Hi, thanks for your script! The difference of my script is just changing your "transforms.Resize(input_size)," line to "transforms.Resize([224, 224]),".

ngailapdi commented 2 years ago

Hi @keerthan2

Are you trying to reconstruct from our released rendered images?