export lmdb to jpg has error

[ratsuki]

I just try to export_data.py export tower_train_lmdb --out_dir data Traceback (most recent call last): File "export_data.py", line 65, in main() File "export_data.py", line 61, in main export_images(lmdb_path, args.out_dir) File "export_data.py", line 39, in export_images image_out_dir = join(out_dir, '/'.join(key[:6])) TypeError: sequence item 0: expected str instance, int found How can I solve this question,btw my export fuction dont use 'flat' parameter

[fyu]

Is this still a problem? Can you print the key on that line?

[shirishr]

@fyu, @ratsuki I can confirm it is still a problem. My environment is python 3.6 & opencv 3.4.1

[zikai1]

I encounter the same problem, my env is python 3.6 . Didyou solve the problem ? @ratsuki @fyu @shirishr

[cippall]

Same issue. They key prints fine : b'000013'. Any solutions ?

[cippall]

The changes below fixes this issue when running the script in a windows environment. Using .decode() for the key and 'wb' for writing in binary mode seems to work in my case. Modify these lines in the data.py script:

LINE 44 : image_out_dir = join(out_dir, '/'.join(key[:6].decode()))

LINE 50 : with open(image_out_path, 'wb') as fp:

[patriciaVitoria]

When i modify both lines below i get the following error:

python data.py export bedroom_val_lmdb --out_dir data Exporting bedroom_val_lmdb to data Traceback (most recent call last): File "data.py", line 89, in main() File "data.py", line 85, in main export_images(lmdb_path, args.out_dir, args.flat) File "data.py", line 49, in export_images image_out_path = join(image_out_dir, key + '.webp') TypeError: can't concat str to bytes

Anyone know how to solve it?

[arthurmchr]

@patriciaVitoria line 49 : image_out_path = join(image_out_dir, key.decode() + '.webp')

[pkuanjie]

The changes below fixes this issue when running the script in a windows environment. Using .decode() for the key and 'wb' for writing in binary mode seems to work in my case. Modify these lines in the data.py script:

LINE 44 : image_out_dir = join(out_dir, '/'.join(key[:6].decode()))

LINE 50 : with open(image_out_path, 'wb') as fp:

It works on Ubuntu16.04 with python3.6, thanks!

[i-chaochen]

it exports the image as webp, how do you convert it as jpg?

@arthurmchr @cippall

[aviadpinis]

Hi, I work with Ubuntu 18.04, and i changed the function to:

def export_images(db_path, out_dir, flat=False, limit=-1):
    print('Exporting', db_path, 'to', out_dir)
    env = lmdb.open(db_path, map_size=1099511627776,
                    max_readers=100, readonly=True)
    count = 0
    with env.begin(write=False) as txn:
        cursor = txn.cursor()
        for key, val in cursor:
            if not flat:
                image_out_dir = join(out_dir, '/'.join(key[:6].decode()))
            else:
                image_out_dir = out_dir
            if not exists(image_out_dir):
                os.makedirs(image_out_dir)
            print('Current key:', key)
            image_out_path = join(image_out_dir, key.decode() + '.jpg')
            img = cv2.imdecode(
                numpy.fromstring(val, dtype=numpy.uint8), 1)
            cv2.imwrite(image_out_path, img)
            count += 1
            if count == limit:
                break
            if count % 1000 == 0:
                print('Finished', count, 'images')

and it works, all images save in image_out_dir. but the problem is each image saved in separately, so i recommend to add in finish --flat to your command, for example:

python data.py export ./bedroom_val_lmdb --out_dir ./bedroom_val --flat

it extracts that all images into one folder, so this is a more neat..

[littlejuyan]

Hi, I work with Ubuntu 18.04, and i changed the function to:

def export_images(db_path, out_dir, flat=False, limit=-1):
    print('Exporting', db_path, 'to', out_dir)
    env = lmdb.open(db_path, map_size=1099511627776,
                    max_readers=100, readonly=True)
    count = 0
    with env.begin(write=False) as txn:
        cursor = txn.cursor()
        for key, val in cursor:
            if not flat:
                image_out_dir = join(out_dir, '/'.join(key[:6].decode()))
            else:
                image_out_dir = out_dir
            if not exists(image_out_dir):
                os.makedirs(image_out_dir)
            print('Current key:', key)
            image_out_path = join(image_out_dir, key.decode() + '.jpg')
            img = cv2.imdecode(
                numpy.fromstring(val, dtype=numpy.uint8), 1)
            cv2.imwrite(image_out_path, img)
            count += 1
            if count == limit:
                break
            if count % 1000 == 0:
                print('Finished', count, 'images')

and it works, all images save in image_out_dir. but the problem is each image saved in separately, so i recommend to add in finish --flat to your command, for example:

python data.py export ./bedroom_val_lmdb --out_dir ./bedroom_val --flat

it extracts that all images into one folder, so this is a more neat..

Hi, thanks for the sharing. When I use this code to extract jpg images from mdb, I got the error: Traceback (most recent call last): File "exportmdb.py", line 88, in main() File "exportmdb.py", line 84, in main export_images(lmdb_path, args.out_dir, args.flat) File "exportmdb.py", line 49, in exportimages img = cv2.imdecode(numpy.fromstring(val, dtype=numpy.uint8), 1) cv2.error: OpenCV(4.5.2) ../modules/imgcodecs/src/loadsave.cpp:736: error: (-215:Assertion failed) !buf.e mpty() in function 'imdecode'

Do you know how to fix this? Thanks so much!