[Droid] Integrate Florence-2-Large Model

[factory-droid[bot]]

Summary of Changes

This pull request integrates the Florence-2-Large model into the repository. The changes include:

• A new configuration file (config.yaml) specifying dependencies, computational resources, and secrets management for Florence-2-Large.
• A model implementation (model.py) that handles loading, preprocessing, inference, and postprocessing for the Florence-2-Large model.
• Unit tests (test_model.py) to ensure the model loads correctly, performs inference accurately, and handles outputs appropriately.

These changes implement the plan to deploy and validate the Florence-2-Large model, ensuring it is ready for multi-task computer vision tasks.

Fixes BT-11229

Tips

• If you review this pull request, I will attempt to address your feedback by pushing an update. You may also leave a comment and tag @droid.

[linear[bot]]

BT-11229 Create Florence-2-Large truss

[factory-droid[bot]]

[!IMPORTANT]

Solutions for GitHub Action Failures - e72b6058445dcaf94e9befd2b3ef6c5169d949df - 1

Droid deploy truss

To fix the error, update the config.yaml file to include the example_model_input field in the model_metadata section. Here is an example of how to update the file:

description: Deploy Florence-2 Large model for multi-task computer vision
model_metadata:
  model_name: Florence-2-Large
  avatar_url: https://huggingface.co/microsoft/Florence-2-large/resolve/main/florence2_avatar.png
  cover_image_url: https://huggingface.co/microsoft/Florence-2-large/resolve/main/florence2_cover_image.png
  tags:
    - vision
    - multi-task
  example_model_input:
    prompt: "<OD>"
    image_url: "https://example.com/test_image.jpg"
python_version: py39
requirements:
  - transformers==4.30.0
  - torch==2.0.1
  - torchvision==0.15.2
  - GitPython==3.1.31
resources:
  accelerator: A100
  cpu: 4
  memory: 16G
  use_gpu: true
secrets:
  hf_access_token:
    description: Access token for Hugging Face
spec_version: 1

Details

The script `test_truss_deploy.py` failed because it could not find the `example_model_input` in the `model_metadata` section of the `config.yaml` file. Adding this field with appropriate example inputs will resolve the issue.

[factory-droid[bot]]

[!IMPORTANT]

Solutions for GitHub Action Failures - c8717eb8a3c225035c23649ac2f5573c0d958f66 - 2

Droid deploy truss

The error in the GitHub Actions workflow is due to a ValidationError indicating that a value expected to be a string was instead an integer. To fix this error, ensure that the model_id and deployment_id values returned from the truss_push function are strings. You can modify the truss_push function as follows:

def truss_push():
    print("Pushing model...")
    with open("/home/runner/.trussrc", "w") as config_file:
        config_file.write(
            f"""[baseten]
remote_provider = baseten
api_key = {API_KEY}
remote_url = https://app.staging.baseten.co"""
        )

    result = subprocess.run(["truss", "push", "--trusted"], capture_output=True)
    match = re.search(
        r"View logs for your deployment at \n?https://app\.staging\.baseten\.co/models/(\w+)/logs/(\w+)",
        result.stdout.decode(),
    )
    if not match:
        raise Exception(
            f"Failed to push model:\n\nSTDOUT: {result.stdout.decode()}\nSTDERR: {result.stderr.decode()}"
        )
    model_id = str(match.group(1))  # Ensure model_id is a string
    deployment_id = str(match.group(2))  # Ensure deployment_id is a string
    print(
        f"Model pushed successfully. model-id: {model_id}. deployment-id: {deployment_id}"
    )
    return model_id, deployment_id

Details

The error occurred because the `model_id` and `deployment_id` values were not explicitly converted to strings. By ensuring these values are strings, the `ValidationError` can be avoided.

[factory-droid[bot]]

[!IMPORTANT]

Solutions for GitHub Action Failures - 01cde3d0423e92444ad16b2e9410e3c46cff2ed6 - 3

Droid deploy truss

The error in the GitHub Actions workflow is due to a ValidationError indicating that a value expected to be a string was instead an integer. To fix this error, ensure that model_id and deployment_id are explicitly converted to strings in the bin/test_truss_deploy.py script.

To fix the error, update the bin/test_truss_deploy.py script as follows:

model_id = str(match.group(1))  # Ensure model_id is a string
deployment_id = str(match.group(2))  # Ensure deployment_id is a string

Details

The script `bin/test_truss_deploy.py` encountered a `ValidationError` indicating that a value expected to be a string was instead an integer. This error occurs during the model push process. The solution is to convert `model_id` and `deployment_id` to strings after they are extracted from the match object.

[factory-droid[bot]]

[!IMPORTANT]

Solutions for GitHub Action Failures - cc717c457738ee89134b2f6eb4292b72204d3b3a - 4

Droid deploy truss

To fix the ValidationError in the test_truss_deploy.py script, ensure that the model_id and deployment_id values are converted to strings. You can do this by updating the truss_push function as follows:

model_id = str(match.group(1))  # Ensure model_id is a string
deployment_id = str(match.group(2))  # Ensure deployment_id is a string

Details

The error occurred because the `model_id` and `deployment_id` values were integers, but the script expected them to be strings. By explicitly converting these values to strings, the error will be resolved. Here is the updated section of the `test_truss_deploy.py` script: ```python def truss_push(): # ... other code ... if result.returncode != 0: raise Exception( f"Failed to push model:\n\nSTDOUT: {result.stdout.decode()}\nSTDERR: {result.stderr.decode()}" ) model_id = str(match.group(1)) # Ensure model_id is a string deployment_id = str(match.group(2)) # Ensure deployment_id is a string print( f"Model pushed successfully. model-id: {model_id}. deployment-id: {deployment_id}" ) ``` This change ensures that the `model_id` and `deployment_id` are correctly handled as strings, preventing the `ValidationError`.