search

charlesq34 / pointnet

PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
Other
4.66k stars 1.44k forks source link

Confusion to create .h5 file for multi dimensional features #181

Closed csitaula closed 3 years ago

csitaula commented 5 years ago

Dear Author, I am trying to use PointNet for my research to combine the random feature of the image obtained from the random slices. I am planning to use 50*600 size features per image. Here, 50 means the number of slices of the image and 600 means the size of features. I want to get a single aggregated feature representing an image. Is it possible from your network? Another confusion is, how to prepare a data .h5 similar to yours? It would be wonderful if you suggested me some ideas. Thanks

TianyangChen357 commented 5 years ago

Hi there,

I am also using this 3D semantic segmentation method in my study.

Based on your question, do you mean your data is 2D image data? This PointNext here is for 3D point cloud data by the way.

Best, Tianyang

TianyangChen357 commented 5 years ago

Hi,

I saw your question about whether it is possible to use PointNet for higher dimensional vectors such as 500-D and perform training.

Sounds like you mean your data has multiple channels? Channels are like x,y,z, R, G, B and they could be more. As stated in the paper, I think the answer to your question is Yes.

csitaula commented 5 years ago

Thanks for your reply. Firstly, I sliced an image into 100 slices and created phog(pyramid of histogram of gradient) features of 768-D. I am planning to create a matrix of 100*768 per image using those features. Can I use pointnet to aggregate such features? Look forward to hearing from you. Thank you.

On Wed, 22 May 2019, 06:16 cagis2019, notifications@github.com wrote:

Hi,

I saw your question about whether it is possible to use PointNet for higher dimensional vectors such as 500-D and perform training.

Sounds like you mean your data has multiple channels? Channels are like x,y,z, R, G, B and they could be more. As stated in the paper, I think the answer to your question is Yes.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/charlesq34/pointnet/issues/181?email_source=notifications&email_token=AIRLZ2RGOEMMBNE4U3GRL5DPWRKAXA5CNFSM4HN2E4TKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODV5BWRQ#issuecomment-494541638, or mute the thread https://github.com/notifications/unsubscribe-auth/AIRLZ2V4S5LEB6AICST7IFLPWRKAXANCNFSM4HN2E4TA .

TianyangChen357 commented 5 years ago

Hi there,

Emmmm what you said are 2D image. Features in 2D image are based on a 2D window moving on the image, where HOG is also based on that window. PointNet is for 3D datasets like point cloud. The moving window (or block) is 3D cubic. So, I do think PointNet could work on the problem you stated.

Hope this helps!

Tianyang

On Tue, May 21, 2019 at 7:09 PM Chiranjibi Sitaula notifications@github.com wrote:

Thanks for your reply. Firstly, I sliced an image into 100 slices and created phog(pyramid of histogram of gradient) features of 768-D. I am planning to create a matrix of 100*768 per image using those features. Can I use pointnet to aggregate such features? Look forward to hearing from you. Thank you.

On Wed, 22 May 2019, 06:16 cagis2019, notifications@github.com wrote:

Hi,

I saw your question about whether it is possible to use PointNet for higher dimensional vectors such as 500-D and perform training.

Sounds like you mean your data has multiple channels? Channels are like x,y,z, R, G, B and they could be more. As stated in the paper, I think the answer to your question is Yes.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub < https://github.com/charlesq34/pointnet/issues/181?email_source=notifications&email_token=AIRLZ2RGOEMMBNE4U3GRL5DPWRKAXA5CNFSM4HN2E4TKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODV5BWRQ#issuecomment-494541638 , or mute the thread < https://github.com/notifications/unsubscribe-auth/AIRLZ2V4S5LEB6AICST7IFLPWRKAXANCNFSM4HN2E4TA

.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/charlesq34/pointnet/issues/181?email_source=notifications&email_token=ALIBLW2PL2UB7JLPGGT5MG3PWR6KFA5CNFSM4HN2E4TKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODV5N5OY#issuecomment-494591675, or mute the thread https://github.com/notifications/unsubscribe-auth/ALIBLW3XIDKBN2I6L4ZUVCTPWR6KFANCNFSM4HN2E4TA .

csitaula commented 5 years ago

Hi Tianyang, Thanks for you reply. Yes, I use 2D grayscale image. I extract the PHOG features from each slices and planning to aggregate these features using PointNet. Just wondering if I can train PointNet with these datasets and aggregate those hoG features. I am not going to use 2D or 3D point clouds for the research. Do you think I can train PointNet with these datasets? If so, I need to design .h5 file using those HoG features?Thanks in advance.

TianyangChen357 commented 5 years ago

Empirically, I suggest you have a try since the shape of your data could potentially feed to such an algorithm. And it does have an aggregation of features in PointNet and works as it was supposed to be (extract global and local features of each point) even though the author stated he was not sure whether such aggregation works as the same way as he explained in such a black box.

Theoretically, You may not use it directly since the spatially related processes may not be appropriate to an image. For example, one important step in PointNet is to calculate the normals of each point. This, obviously, does not make sense to your data since normals are features of the 3D point cloud not for a 2D image. But who knows, maybe the way to calculate normals based on the first three dimensions x, y, z in 3D point cloud dataset could have some other meanings within your 2D image dataset.

Yes, for training with PointNet, you have to convert your data to .h5 file. The author provided this code, refer to ./sem_seg/gen_indoor3d_h5.py. You can follow that code to convert your dataset. I suggest you go through his demo with the demo data, so, you may have a better understanding of his code. I can assist you if you meet any question since I have worked it out.

Good luck, Tianyang

On Tue, May 21, 2019 at 10:49 PM Chiranjibi Sitaula < notifications@github.com> wrote:

Hi Tianyang, Thanks for you reply. Yes, I use 2D grayscale image. I extract the PHOG features from each slices and planning to aggregate these features using PointNet. Just wondering if I can train PointNet with these datasets and aggregate those hoG features. I am not going to use 2D or 3D point clouds for the research. Do you think I can train PointNet with these datasets? If so, I need to design .h5 file using those HoG features?Thanks in advance.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/charlesq34/pointnet/issues/181?email_source=notifications&email_token=ALIBLWYZWFZFEMNQ5IQMZTTPWSYELA5CNFSM4HN2E4TKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODV5XVNI#issuecomment-494631605, or mute the thread https://github.com/notifications/unsubscribe-auth/ALIBLWYFUJM5IUW4SRLZOG3PWSYELANCNFSM4HN2E4TA .

csitaula commented 5 years ago

Hi Tianyang, Thanks a lot for the reply and clarification. I have already run PointNet with the dataset provided on their 3D points. I was simply wondering about the feasibility of my dataset. Yes, I will try to create the .h5 file of my dataset (PHoG features of the slices per image and stack them to form 100*768 size matrix per image) and let you know what happens. Can you give me your email so that I can contact you during the course of my implementation? Thanks

TianyangChen357 commented 5 years ago

Sure！ It is tchen19@uncc.edu

Chiranjibi Sitaula notifications@github.com于2019年5月21日 周二下午11:42写道：

Hi Tianyang, Thanks a lot for the reply and clarification. I have already run PointNet with the dataset provided on their 3D points. I was simply wondering about the feasibility of my dataset. Yes, I will try to create the .h5 file of my dataset (PHoG features of the slices per image and stack them to form 100*768 size matrix per image) and let you know what happens. Can you give me your email so that I can contact you during the course of my implementation? Thanks

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/charlesq34/pointnet/issues/181?email_source=notifications&email_token=ALIBLW32KJ67CSOQ2KLBYVLPWS6L3A5CNFSM4HN2E4TKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODV5Z4BI#issuecomment-494640645, or mute the thread https://github.com/notifications/unsubscribe-auth/ALIBLW47A3QKNJ4ZAPSJRH3PWS6L3ANCNFSM4HN2E4TA .

csitaula commented 5 years ago

Thanks a lot. I will contact you if I face any difficulties and let you know what happens.

Best Regards, Chiranjibi Sitaula Postgraduate Research Fellow School of IT Burwood Campus Deakin University,Melbourne, Australia ORCID:orcid.org/0000-0002-4564-2985 CYA E-RYT 550 Certified Yoga Instructor from Nepal.

On Wed, May 22, 2019 at 2:10 PM cagis2019 notifications@github.com wrote:

Sure！ It is tchen19@uncc.edu

Chiranjibi Sitaula notifications@github.com于2019年5月21日 周二下午11:42写道：

Hi Tianyang, Thanks a lot for the reply and clarification. I have already run PointNet with the dataset provided on their 3D points. I was simply wondering about the feasibility of my dataset. Yes, I will try to create the .h5 file of my dataset (PHoG features of the slices per image and stack them to form 100*768 size matrix per image) and let you know what happens. Can you give me your email so that I can contact you during the course of my implementation? Thanks

— You are receiving this because you commented. Reply to this email directly, view it on GitHub < https://github.com/charlesq34/pointnet/issues/181?email_source=notifications&email_token=ALIBLW32KJ67CSOQ2KLBYVLPWS6L3A5CNFSM4HN2E4TKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODV5Z4BI#issuecomment-494640645 , or mute the thread < https://github.com/notifications/unsubscribe-auth/ALIBLW47A3QKNJ4ZAPSJRH3PWS6L3ANCNFSM4HN2E4TA

.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/charlesq34/pointnet/issues/181?email_source=notifications&email_token=AIRLZ2VZVHEGPZQ5XOBO7QLPWTBSFA5CNFSM4HN2E4TKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODV523YY#issuecomment-494644707, or mute the thread https://github.com/notifications/unsubscribe-auth/AIRLZ2SC2YWBJJVNRD2SUYLPWTBSFANCNFSM4HN2E4TA .

csitaula commented 5 years ago

@cagis2019 I am working with PointNet and using your code from PointNet-keras. I having difficulty in Data generator class of that file. I have already converted my data into .h5 format. That data generator is for point cloud. How to fit my 680-D?

noridayu1998 commented 3 years ago

@TianyangChen357 may I know which data for semantic segmentation?

TianyangChen357 commented 3 years ago

Sorry, what do you mean by which data?

On Wed, Mar 24, 2021, 7:22 AM noridayu1998 @.***> wrote:

[Caution: Email from External Sender. Do not click or open links or attachments unless you know this sender.]

@TianyangChen357 https://github.com/TianyangChen357 may I know which data for semantic segmentation?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/charlesq34/pointnet/issues/181#issuecomment-805742348, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALIBLWZSTFL5LQQOWGMBKITTFHDQLANCNFSM4HN2E4TA .

noridayu1998 commented 3 years ago

Sorry, what do you mean by which data? … On Wed, Mar 24, 2021, 7:22 AM noridayu1998 @.**> wrote: [Caution*: Email from External Sender. Do not click or open links or attachments unless you know this sender.] @TianyangChen357 https://github.com/TianyangChen357 may I know which data for semantic segmentation? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#181 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALIBLWZSTFL5LQQOWGMBKITTFHDQLANCNFSM4HN2E4TA .

I already download the dataset. but, I have a error when run the train.py for sem_seg. File "D:\1. PSM 2020\pointnet\pointnet-master\sem_seg\train.py", line 228, in train_one_epoch log_string('mean loss: %f' % (loss_sum / float(num_batches)))

ZeroDivisionError: float division by zero

do you know how to fix it?