lmdb_create_example.py create invalid lmdb

[tpys]

I use lmdb_create_example.py to create my own dataset, then use it in Multi-GPU_Training tutorial, then display_images_and_confidence give me all gray images, which is not right. please help me out, thanks!

the code i used to create my own dataset is as below: def create_db(images, labels, output_file): LMDB_MAP_SIZE = 1 << 40 # MODIFY env = lmdb.open(output_file, map_size=LMDB_MAP_SIZE) num_images = len(images) checksum = 0 print(">>> Write database...") with env.begin(write=True) as txn: for j in range(0, num_images):

MODIFY: add your own data reader / creator

        img_data = skimage.img_as_float(skimage.io.imread(images[j])).astype(np.float32)
        img_data = img_data.swapaxes(1, 2).swapaxes(0, 1)
        img_data = img_data[(2, 1, 0), :, :]
        # img_data = np.random.rand(3, 160, 160)
        label = int(labels[j])
        # Create TensorProtos
        tensor_protos = caffe2_pb2.TensorProtos()
        img_tensor = tensor_protos.protos.add()
        img_tensor.dims.extend(img_data.shape)
        img_tensor.data_type = 1
        flatten_img = img_data.reshape(np.prod(img_data.shape))
        img_tensor.float_data.extend(flatten_img)
        label_tensor = tensor_protos.protos.add()
        label_tensor.data_type = 2
        label_tensor.int32_data.append(label)
        txn.put(
            '{}'.format(j).encode('ascii'),
            tensor_protos.SerializeToString()
        )
        checksum += np.sum(img_data) * label
        if (j % 1000 == 0):
            print("Inserted {} rows".format(j))

[tpys]

Hi, if any body there. i found a instead solution, use old caffe to generate lmdb, then it can be used by brew.input_image with option use_caffe_datum on. But still it’s not the ultimate answer. Hopefully sombody can help me out.

[leovandriel]

skimage.img_as_float returns color values between 0 and 1. You probably need to scale to floats between -128 and 128. Try adding:

img_data = img_data * 256 - 128

[tpys]

thanks a lot, i also used cv2 to load image and doesn't give me right result. Anyway i will try it out and let you know if it work. @leonardvandriel

[tpys]

I tried your suggestion , put img_data = img_data *256 - 128, still get invalid image (all value is -1). I use the following code to load and show image.

    data, label = brew.image_input(
        model,
        reader, ["data", "label"],
        batch_size=batch_size,
        output_type=dtype,
        use_gpu_transform=True if model._device_type == 1 else False,
        use_caffe_datum=True,
        mean=128.,
        std=128.,
        scale=256,
        crop=img_size,
        mirror=1,
        is_test=is_test,
    )

def display_first_image():
    data = workspace.FetchBlob("gpu_0/data")
    bgr = (data[0, ...].swapaxes(0, 1).swapaxes(1, 2) + 1.0) / 2.0
    rgb = bgr[...,::-1]
    plt.imshow(rgb)
    plt.pause(0.001)