error in creating tensorflow dataset

[buffyhridoy]

def load_data(point_cloud_batch, label_cloud_batch):
    point_cloud_batch.set_shape([NUM_SAMPLE_POINTS, 7])  #NUM_SAMPLE_POINTS=1024
    label_cloud_batch.set_shape([NUM_SAMPLE_POINTS, 9])
    return point_cloud_batch, label_cloud_batch

def augment(point_cloud_batch, label_cloud_batch):
    noise = tf.random.uniform(
        tf.shape(label_cloud_batch), -0.005, 0.005, dtype=tf.float64
    )
    point_cloud_batch += noise[:, :, :3]
    return point_cloud_batch, label_cloud_batch

def generate_dataset(point_clouds, label_clouds, is_training=True):
    dataset = tf.data.Dataset.from_tensor_slices((point_clouds, label_clouds))
    dataset = dataset.shuffle(BATCH_SIZE * 100) if is_training else dataset
    dataset = dataset.map(load_data, num_parallel_calls=tf.data.AUTOTUNE)
    dataset = dataset.batch(batch_size=BATCH_SIZE)
    dataset = (
        dataset.map(augment, num_parallel_calls=tf.data.AUTOTUNE)
        if is_training
        else dataset
    )
    return dataset

split_index = int(len(point_clouds) * (1 - VAL_SPLIT))
train_point_clouds = point_clouds[:split_index]
train_label_cloud = point_cloud_labels[:split_index]
total_training_examples = len(train_point_clouds)

val_point_clouds = point_clouds[split_index:]
val_label_cloud = point_cloud_labels[split_index:]

print("Num train point clouds:", len(train_point_clouds))
print("Num train point cloud labels:", len(train_label_cloud))
print("Num val point clouds:", len(val_point_clouds))
print("Num val point cloud labels:", len(val_label_cloud))

train_dataset = generate_dataset(train_point_clouds, train_label_cloud)
val_dataset = generate_dataset(val_point_clouds, val_label_cloud, is_training=False)

print("Train Dataset:", train_dataset)
print("Validation Dataset:", val_dataset)

############################################################################################## the result should be:

Num train point clouds: 819
Num train point cloud labels: 819
Num val point clouds: 205
Num val point cloud labels: 205

Train Dataset: <ParallelMapDataset shapes: ((None, 1024, 7), (None, 1024, 9)), types: (tf.float64, tf.float32)>
Validation Dataset: <BatchDataset shapes: ((None, 1024, 7), (None, 1024, 9)), types: (tf.float64, tf.float32)>
###############################################################################################
but we are getting error in line 2:

Num train point clouds: 819
Num train point cloud labels: 819
Num val point clouds: 205
Num val point cloud labels: 205
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-60-dd40329ada93> in <module>()
     39 print("Num val point cloud labels:", len(val_label_cloud))
     40 
---> 41 train_dataset = generate_dataset(train_point_clouds, train_label_cloud)
     42 val_dataset = generate_dataset(val_point_clouds, val_label_cloud, is_training=False)
     43 

11 frames
/usr/local/lib/python3.7/dist-packages/tensorflow/python/autograph/impl/api.py in wrapper(*args, **kwargs)
    697       except Exception as e:  # pylint:disable=broad-except
    698         if hasattr(e, 'ag_error_metadata'):
--> 699           raise e.ag_error_metadata.to_exception(e)
    700         else:
    701           raise

ValueError: in user code:

    File "<ipython-input-60-dd40329ada93>", line 2, in load_data  *
        point_cloud_batch.set_shape([NUM_SAMPLE_POINTS, 7])

    ValueError: Shapes must be equal rank, but are 1 and 2

[SuryanarayanaY]

@buffyhridoy ,

I tried to replicate but it seems the dataset URL having issues as mentioned below.

Exception: URL fetch failure on https://git.io/JiY4i: 502 -- Bad Gateway

CC'ing the authors @sayakpaul , @soumik12345 , Could you please update the URL ?

Thanks!

[soumik12345]

Hi @SuryanarayanaY @buffyhridoy You can use this link: https://github.com/soumik12345/point-cloud-segmentation/releases/download/v0.1/shapenet.zip

[SuryanarayanaY]

@soumik12345 ,

Thanks for the link. I will check it and if working I am willing to raise a PR to fix this if you are fine with that. Thanks!

[sachinprasadhs]

The code has been migrated to Keras 3 and it's working fine, please find the latest updated tutorial here https://keras.io/examples/vision/pointnet_segmentation/