search

ankurhanda / gvnn

gvnn: Geometric Vision with Neural Networks
445 stars 71 forks source link

Demo or example for 3D data #8

Open gxstudy opened 7 years ago

gxstudy commented 7 years ago

Hi, Ankur,

Thank you for proving this awesome gvnn code. I am wondering if you could provide an example of the network setting on using 3D data, like 30x30x30 voxel data. I saw the demo you provided is on 2D image, so I am very confused how to use it directly on 3D voxel data. Thank you very much.

Best, Cindy Guo

ankurhanda commented 7 years ago

Can you tell me exactly what you'd like to do with the voxel data? What sort of operation would you like to do on the voxel grid?

gxstudy commented 7 years ago

Sure, I am working on 3D object recognition, each sample is 30x30x30, and each sample is rotated by a random degree. I learned that spatial transformer networks could automatically learn the rotation parameters and rotate (and re-sample) it back, I want to apply spatial transformer networks during the training and testing to see if it can achieve higher accuracy on the rotated dataset. There are about 9000 training data and 1000 testing data. The input to spatial transformer networks is a batch of 30x30x30 volumetric data, as the setting in torch, I guess it means, kT=30, kW=30, kH=30. After spatial transformer networks, the output will be supplied to torch VolumetricConvolution layer, for example, nn.VolumetricConvolution(1, 32, 6, 6, 6 ,2, 2, 2).

Please let me know if anything is unclear. Best, CIndy

ankurhanda commented 7 years ago

OK --- so I think a bilinearinterpolation3d layer should be needed for that. We only have 2D bilinear interpolation so far. Let me know if you are able to do it yourself otherwise I will take a stab at it when I find time.

gxstudy commented 7 years ago

I'd prefer you do it because I am not familiar with the implementation details. I just want to compare and cite your work. Thank you so much for the help. Best, Cindy

ankurhanda commented 7 years ago

Sorry I can't do it in the immediate future but I will do it at some point. Otherwise all you need to do is to make sure you have the 3rd dimension here in the bilinear interpolation code https://github.com/ankurhanda/gvnn/blob/master/BilinearSamplerBHWD.cu

gxstudy commented 7 years ago

Would you have time to do it in a week or two, if so, I can wait. I don't really have time to go through all the details and I don't know what I need to do to call this bilinearSamplerBHWD. Thanks a lot. Best, Cindy

ankurhanda commented 7 years ago

OK --- I will try to do it in these two weeks and let you know how things go. Not a problem.

gxstudy commented 7 years ago

Thank you so much! I really appreciate it! Best, Cindy

ranahanocka commented 7 years ago

Also looking for this. Was it ever added?

ankurhanda commented 7 years ago

Super sorry not yet but I will add soon. Have been super busy with SceneNet RGB-D lately. Please check back soon!

ranahanocka commented 7 years ago

Cool. Love your work on SceneNet - looking forward to seeing the code from the awesome video that was published not so long ago. But even more excited about the 3D bilinear :)

Ethiral commented 7 years ago

Was looking for this. Any update on the 3D bilinear?

ankurhanda commented 7 years ago

I should do it but I haven't had the time to. If you are able to extend the 2D version - I can take a loo at it! I'm sorry I have been busy of late.