About converting coco to lmdb

[AlexofNTU]

I followed the steps to generate lmdb of COCO. The lmdb file can't be read by the following codes but It could work with PASCAL datasets (07 & 12)

import caffe
import lmdb
import PIL.Image
from StringIO import StringIO
import numpy as np

def read_lmdb(lmdb_file):
    cursor = lmdb.open(lmdb_file, readonly=True).begin().cursor()
    datum = caffe.proto.caffe_pb2.Datum()
    for _, value in cursor:
        datum.ParseFromString(value)
        s = StringIO()
        s.write(datum.data)
        s.seek(0)

        yield np.array(PIL.Image.open(s)), datum.label

lmdb_dir = '/raid/shared/Dataset/coco/lmdb/coco_minival_lmdb/'
for im, label in read_lmdb(lmdb_dir):
    print label, im

I got the following error message: IOError: cannot identify image file

I was curious about why the default values of the 4 following parameters are 0. min_dim=0 max_dim=0 width=0 height=0

My understand of the width and height is that the input size of the network. For example for Alexnet, with= 224, height=224, For SSD300, width= 300, height=300, For SSD500, width= 500, height=500

However, I can't even read the lmdb no matter what the setting are. The error messages are almost identical. If someones were successful in making lmdb, please kindly indicate where I should go ^^|

[weiliu89]

Did you try to use script like ssd_pascal.py to train on those lmdb you created?

[madhavajay]

I had this problem and the issue was the wrong Datum class, you need AnnotatedDatum: For example:

def preview_lmdb_records(lmdb_path, limit=20):
    annotated_datum = caffe_pb2.AnnotatedDatum()
    lmdb_cursor = open_lmdb(lmdb_path)

    color = (255, 0, 0)

    images = []

    count = 0
    for key, value in lmdb_cursor:
        count += 1
        if count > limit:
            break
        filename = str(key).split('/')[-1][:-1]
        annotated_datum.ParseFromString(value)
        datum = annotated_datum.datum
        groups = annotated_datum.annotation_group
        boxes = []
        for group in groups:
            label_id = group.group_label
            label_name = get_labelname(label_id)
            for anno in group.annotation:
                xmin = int(anno.bbox.xmin * datum.width)
                ymin = int(anno.bbox.ymin * datum.height)
                xmax = int(anno.bbox.xmax * datum.width)
                ymax = int(anno.bbox.ymax * datum.height)
                box = {'xmax': int(xmax), 'xmin': int(xmin),
                       'ymax': int(ymax), 'ymin': int(ymin),
                        'color': color, 'label_name': label_name}
                boxes.append(box)

        image_data = np.frombuffer(datum.data, dtype=np.uint8)
        image = cv2.imdecode(image_data, -1)

        images.append(((draw_boxes(image, boxes)), filename))

    plot_images(images)