Multiple Image Inputs support

[bojiang]

Is your feature request related to a problem? Please describe.

The new ImageInput with micro-batching support does not allow multiple image inputs, comparing to the old ImageHandler

Describe the solution you'd like Add bentoml.adapters.MultiImageInput input adapter type:

class MultiImageInput(BaseInputAdapter):
    """
    Warning: this is a not yet implemented, follow issue #806 for updates
    Transform incoming images data from http request, cli or lambda event into numpy
    array.
    Args:
        input_names (string[]]): A tuple of acceptable input name for HTTP request.
            Default value is (image,)
        accept_image_formats (string[]):  A list of acceptable image formats.
            Default value is loaded from bentoml config
            'apiserver/default_image_input_accept_file_extensions', which is
            set to ['.jpg', '.png', '.jpeg', '.tiff', '.webp', '.bmp'] by default.
            List of all supported format can be found here:
            https://imageio.readthedocs.io/en/stable/formats.html
        pilmode (string): The pilmode to be used for reading image file into numpy
            array. Default value is 'RGB'.  Find more information at:
            https://imageio.readthedocs.io/en/stable/format_png-pil.html
    Raises:
        ImportError: imageio package is required to use ImageInput
    Example usage:

    class MyService(BentoService):
        @bentoml.api(input=MultiImageInput(input_names=('imageX', 'imageY')))
        def predict(self, image_groups):
            for image_group in image_groups:
                image_array_x = image_group['imageX']
                image_array_y = image_group['imageY']
    ```
"""

see placeholder file here: https://github.com/bentoml/BentoML/pull/804

**Describe alternatives you've considered**
`bentoml.adapters.MultiFileInput`

**Additional context**
N/A

[akainth015]

input_names (string[]]): A tuple of acceptable input name for HTTP request.
            Default value is (image,)

Why do we want to use this instead of passing along all of the files and letting the consumer decide which files they will use?

[parano]

@akainth015 great question, I think it makes sense to support inputs of a variable-length list of files/images instead of only fixed-length, named images. The original design allows us to give the user a columnar look of a batch of input, e.g. {'image1': [<image_file>, ...], 'image2': [<image_file>] }, but I think it makes more sense to group input images in rows considering this case, e.g. [ [<image_file>, ..], [<image_file>, ..], ... ] or [ {'image1': <image_file>, 'image2': <image_file>}, ... ], cc @bojiang

[bojiang]

The current design is exactly what you said:

image_array_x = image_group['imageX']

->

 {'image1': <image_file>, 'image2': <image_file>}, ... ]

In most cases, the API developer doesn't need file inputs with dynamic names. The input_names would be useful for API users, for example, shown in the swagger.

[jkuypers93]

Hi @parano @bojiang is it possible currently to post multiple files/images without the input_names. For my use case, the number of images can vary per request, and I don't need to have input names for each image. I would like to just post an array of images e.g [ [<image_file>, ..], [<image_file>, ..], ... ]

My cURL looks something like this: curl -i -X POST "localhost:5000/predict" -H "Content-Type: multipart/form-data" -F "image=@002.jpg" -F "image=@image.jpg"

At the moment with MultiFileInput(input_names=['image']) only the first image file is received

[parano]

hi @jkuypers93 - unfortunately, this is not supported in the current version of BentoML. could you tell me a bit more about your use case? @bojiang and I were just discussing how to support handling requests with a variable number of files in the future version but could not find a good use case for it, would love to learn more.

[jkuypers93]

Hey @parano, thanks for getting back to me!

I have an application where users can upload a variable number of images (sent as an array of images). My model then creates a batch using this array of files and return a prediction for each image. I could query the API one image at the time but that is less efficient.

Previously, I was deploying my model with FastAPI and was using their List[FileUpload] class.

[jkuypers93]

Is it ok if I re-open this issue for this? I would like to contribute :)

[parano]

@jkuypers93 are you in the BentoML slack channel? feel free to ping me and discuss more before you started to make the contribution. The team is actively developing a new version in the 1.0 branch and a lot of things have changed

[parano]

This is now supported in BentoML 1.0, via the Multipart IO descriptor: https://docs.bentoml.org/en/latest/api/api_io_descriptors.html#bentoml-io-multipart

[tomukmatthews]

I am also trying to convert a FastAPI classifying batches of images to bentoml, previously i used the List[FileUpload] as @jkuypers93 did.

Sorry if i'm missing something obvious, but its not clear to me how you would use the Multipart IO Descriptor in this scenario? Is there an example of this anywhere?