Processing images in batches possible?

[tobbber]

Issue Type

Questions

Source

GitHub (source)

DECIMER Image Transformer Version

2.4.0

OS Platform and Distribution

No response

Python version

No response

Current Behaviour?

Hi! Is there a way of running the SMILES prediction on a batch of images? As of now, the predict_SMILES function only accepts filepaths of single images. However, i also had no luck with loading the DECIMER model manually and trying to infer on a batch.

Is there any way to do batch processing of images with DECIMER? Inference currently takes around 2sec/image (On Tesla T4) and i was hoping to speed up inference by using batch processing.

Thank you for your help!

Which images caused the issue? (This is mandatory for images related issues)

No response

Standalone code to reproduce the issue

# attempt to manually batch images before processing
from DECIMER import predict_SMILES
from DECIMER import utils, config
from DECIMER.decimer import DECIMER_V2
import tensorflow as tf

testfiles = [
  "image1.png",   
  "image2.png"
]

prepped = [config.decode_image(f) for f in testfiles]
batch = tf.stack(prepped, axis=0)
print('batch shape:', batch.shape)  
# >>> shape image batch: (2, 512, 512, 3)

DECIMER_V2(batch)
# ValueError: Could not find matching concrete function to call loaded from the SavedModel. Got:
#   Positional arguments (1 total):
#     * <tf.Tensor 'Decoded_Image:0' shape=(2, 512, 512, 3) dtype=float32>
#   Keyword arguments: {}

#  Expected these arguments to match one of the following 1 option(s):

# Option 1:
#   Positional arguments (1 total):
#     * TensorSpec(shape=(512, 512, 3), dtype=tf.float32, name='Decoded_Image')
#   Keyword arguments: {}

Relevant log output

No response

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[Kohulan]

Hi @tobbber ,

One way to process multiple images is to load several model instances and provide each instance with a list of file paths.

Kind regards, Kohulan

[OBrink]

@tobbber, I would recommend to work with the checkpoint and look into the training code. You can't just call-method of the DECIMER_V2 object that you are using is not configured to work with batches the way you tried it there

[frxchii666]

Hi @tobbber ,

One way to process multiple images is to load several model instances and provide each instance with a list of file paths.

Kind regards, Kohulan

hi, could you give an example of using this method?

[Kohulan]

@frxchii,

I plan to make updates in the future. However, we strongly advise running predictions sequentially. If you have ample resources, loading multiple instances would be more efficient. Please refer to the DECIMER.ai repository for insights into how we have implemented this approach.