MadcowD / ell

A language model programming library.
http://docs.ell.so/
MIT License
5.37k stars 316 forks source link

Image generation #330

Open neil-armstrong-instil opened 1 month ago

neil-armstrong-instil commented 1 month ago

I'm a little new to ell (and python in general, so bare with me a bit šŸ˜…), just trying to get an experiment working with image generation. So given a simple example:

bedrockClient = boto3.client("bedrock-runtime", region_name="us-east-1")

@ell.simple(model="amazon.titan-image-generator-v2:0", temperature=1.5, client=bedrockClient)
def execute(userMessage: str) -> PIL.Image:
    return userMessage

if __name__ == "__main__":
    ell.init(verbose=True)
    result = execute("A picture of a bicycle")
    print(result.text)
    if result.images:
        result.images[0].show()

I am getting a result of

botocore.errorfactory.ValidationException: An error occurred (ValidationException) when calling the Converse operation: This action doesn't support the model that you provided. Try again with a supported text or chat model.

I'm not sure if there is some configuration I need to do in order for it to not be a "Converse operation" if that be in ell or the bedrockClient? Is there an example like this I could use as a reference?

MadcowD commented 1 month ago

Hey, this is a good question for @brnaba-aws he is currently maitnaing our bedrock support. I'm not sure he implemented image generation.

4t8dd commented 1 month ago

@neil-armstrong-instil I guess you had an error function definition here:

def execute(userMessage: str) -> PIL.Image:
    return userMessage

This prompt function may return nothing or just string instead of PIL.Image object.