CFIALeronB commented 7 months ago

Issue Description

The goal is to implement model switching functionality in the inference_request function, allowing it to handle different models based on the model_name parameter passed in the request. An issue arises when the system tries to access a model function that is not implemented, leading to an error.

Steps to Reproduce

Send a request to the /inf endpoint with model_name set to a value that does not correspond to an implemented model function.
The system attempts to access a function based on the model_name and fails if the function is not implemented.

Expected Behavior

The inference_request function should dynamically select and call the correct model processing function based on the model_name provided in the request. If a model_name is provided that does not have an associated processing function, the system should handle it gracefully.

Current Behavior

The system throws an error indicating that the required model processing function is not found in the model_inference.inference module when a non-implemented model_name is used.

Possible Solution

Implement a check within the inference_request function to ensure that the model_name provided in the request corresponds to an implemented model processing function. If not, the system should return a meaningful error message or default to a general processing function.

Additional Context

This issue is crucial for the scalability of the system, allowing new models to be integrated seamlessly.
The handling of different models should be modular and maintainable, avoiding complex conditional structures for each model.

Tasks

[x] Verify the structure and implementation of the model_inference.inference module.
[x] Implement a robust model selection mechanism in the inference_request function.
[x] Ensure that the system gracefully handles cases where the model_name does not correspond to an implemented model processing function.
[x] Update documentation and comments to reflect the new model switching functionality.

rngadam commented 7 months ago

@MaxenceGui @RussellJimmies

MaxenceGui commented 7 months ago

Inference Request with n pipelines

Glossary

Pipelines

Pipelines are define as a set of models that follow each others where output of one model is used as input for the next models. A pipeline contains from 1 to n models.

flowchart LR

A{Pipeline entry point}-->|user input|SB1-->F{Pipeline exit point}-->|"last model
output"|G>"return
result.json to user"]

subgraph SB1[In pipeline]
    direction TB

    B(model 01)-->|"model output
    send to the next one"|C(model 02)-->|can have n models|D(n models)

end

Models

A model is a AI model who's part of a pipeline. A model accept images as input and return json as output. Generally json contains coordinate of object in the source image, that he may prompt out to feed the next step of the model.

Model from Frontend

On the frontend interface, a pipeline will be call a model, since the user will not be aware of the difference. From the user perspective, he sent data to a model and receive the result.

Suggestion: we could call the pipeline a method, if we don't want to mix term.

Sequence Diagram for inference request

sequenceDiagram

    actor Client
    participant Frontend
    participant Backend
    participant Blob storage
    participant Model

    Note over Backend,Blob storage: initialisation
    Backend-)Backend: before_serving()
    Backend-)Backend: get_pipelines_models()
    alt
    Backend-)Blob storage: HTTP POST req.
    Blob storage--)Backend: return pipelines_models.json
    else
    Backend-)Frontend: error 400 No pipeline found
    end
    Note over Backend,Blob storage: end of initialisation

    Client->>Frontend: applicationStart()
    Frontend-)Backend: HTTP POST req.
    Backend-)Backend: get_pipelines_names()
    Note right of Backend: return pipelines names set(pipelines.keys())
    Backend--)Frontend: Pipelines names res.

    Frontend->>Client: application is ready
    Client-->>Frontend: client ask action from specific pipeline
    Frontend-)Backend: HTTP POST req.
    Backend-)Backend: inference_request(pipeline_name, folder_name, container_name, imageDims, image)
    alt missing argument
        Backend--)Frontend: Error 400 missing arguments
    else no missing argument
        Backend-)Backend: mount_container(connection_string(Environnement Variable, container_name))
        Backend-)Blob storage: HTTP POST req.
        Blob storage--)Backend: container_client

        Backend-)Backend: upload_image(container_client, folder_name, image_bytes, hash_value)
        Backend-)Blob storage: HTTP POST req.
        Blob storage--)Backend: blob_name

        Backend-)Backend: get_blob(container_client, blob_name)
        Backend-)Blob storage: HTTP POST req.
        Blob storage--)Backend: blob

        loop for every model in pipeline
            Note over Backend,Blob storage: Need to build endpoint with pipeline_name
            Backend-)Backend: urllib.request.Request(endpoint_url, body, header)
            Backend-)Model: HTTP POST req.
            Model--)Backend: result.json
        end

        par Backend to Frontend
            Backend--)Frontend: Processed result res.
        and Backend to Blob storage
            Backend-)Backend: upload_inference_result(container_client, folder_name, result_json_string, hash_value)
            Backend-)Blob storage: HTTP POST req.
        end
    end
    Frontend--)Client: display result

footer_for_diagram