weights for computing the loss in semantic segmentation task

shuluoshu commented 7 years ago

Hi, @charlesq34, Thanks for your nice work, I have two questions as follows:

I modified the train.py in your old "pointnet" version and use it to train the newly-updated sem_seg model, however, I notice that the input data is changed from B x N x 9 to B x N x 3, does that mean you didn't use the RGB information any more ?
What about the 'smpws' ? I guess it is the weights of each point when computing the training loss, but how can I set the weight for each point ? I am still using the "indoor3d_sem_seg_hdf5_data".

Thanks a lot !

triple-tam commented 5 years ago

@shuluoshu did you resolve your #2 question?

I'm running python train.py with the semantic segmentation model on the preprocessed data modelnet40_ply_hdf5_2048 that pointnet2 linked, but this is the error I get related to smpw. My root directory is segmentation which contains my virtualenv and pointnet2 repo.

In short, train() > MODEL.get_loss(pred, labels, weights=end_points=smpw) But 'weights' needs to be a tensor of the same dim / broadcastable to dim of 'labels'. However, 'end_points' (output of MODEL.get_model()) is a dictionary with two tensors, neither of whose shapes fulfill the shape requirement to be 'weights'. Any help or insight on how I can provide a proper 'weights' tensor into MODEL.get_loss() ?

Thank you!

pid: 13809
WARNING:tensorflow:From 
segmentation/03_repos/pointnet2/utils/pointnet_util.py:127: calling reduce_max (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version.
Instructions for updating:
keep_dims is deprecated, use keepdims instead
WARNING:tensorflow:From 
segmentation/03_repos/pointnet2/utils/pointnet_util.py:213: calling reduce_sum (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version.
Instructions for updating:
keep_dims is deprecated, use keepdims instead
WARNING:tensorflow:From segmentation/venv_pointnet/local/lib/python2.7/site-packages/tensorflow/python/util/deprecation.py:553: calling conv1d (from tensorflow.python.ops.nn_ops) with data_format=NHWC is deprecated and will be removed in a future version.
Instructions for updating:
`NHWC` for data_format is deprecated, use `NWC` instead
Traceback (most recent call last):
  File "segmentation/03_repos/pointnet2/train.py", line 284, in <module>
    train()
  File "segmentation/03_repos/pointnet2/train.py", line 122, in train
    MODEL.get_loss(pred, labels_pl, end_points)
  File "segmentation/03_repos/pointnet2/models/pointnet2_sem_seg.py", line 52, in get_loss
    classify_loss = tf.losses.sparse_softmax_cross_entropy(labels=label, logits=pred, weights=smpw)
  File "segmentation/venv_pointnet/local/lib/python2.7/site-packages/tensorflow/python/ops/losses/losses_impl.py", line 912, in sparse_softmax_cross_entropy
    labels, logits, weights, expected_rank_diff=1)
  File "segmentation/venv_pointnet/local/lib/python2.7/site-packages/tensorflow/python/ops/losses/losses_impl.py", line 838, in _remove_squeezable_dimensions
    weights = ops.convert_to_tensor(weights)
  File "segmentation/venv_pointnet/local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1050, in convert_to_tensor
    as_ref=False)
  File "segmentation/venv_pointnet/local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1146, in internal_convert_to_tensor
    ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
  File "segmentation/venv_pointnet/local/lib/python2.7/site-packages/tensorflow/python/framework/constant_op.py", line 229, in _constant_tensor_conversion_function
    return constant(v, dtype=dtype, name=name)
  File "segmentation/venv_pointnet/local/lib/python2.7/site-packages/tensorflow/python/framework/constant_op.py", line 208, in constant
    value, dtype=dtype, shape=shape, verify_shape=verify_shape))
  File "segmentation/venv_pointnet/local/lib/python2.7/site-packages/tensorflow/python/framework/tensor_util.py", line 531, in make_tensor_proto
    "supported type." % (type(values), values))
TypeError: Failed to convert object of type <type 'dict'> to Tensor.

triple-tam commented 5 years ago

Actually nevermind, I just resolved this issue. In train.py, modify the following lines:

Line 109: pointclouds_pl, labels_pl, smpws_pl = MODEL.placeholder_inputs(BATCH_SIZE, NUM_POINT)
Line 121: pred, end_points = MODEL.get_model(pointclouds_pl, is_training_pl, NUM_CLASSES, bn_decay=bn_decay)
Line 122: MODEL.get_loss(pred, labels_pl, smpws_pl)

Although this issue is solved, more are arising. I will post my eventual revisions suited for semantic segmentation to my repos sometime soon.

triple-tam commented 5 years ago

hi @shuluoshu , sorry to bother again, but did you get answers to your two questions? Would greatly help me, as I don't know how to set the smpws weight values either.... The alternative is to just set 1 for everything, but I would prefer a concrete answer. Thanks!

MrXiaoZhen commented 5 years ago

@triple-tam Have you solved the two problems?

lodurality commented 5 years ago

I am interested in the answer to this question as well.

charlesq34 / pointnet2

weights for computing the loss in semantic segmentation task #10