PrefectHQ / marvin

✨ Build AI interfaces that spark joy
https://askmarvin.ai
Apache License 2.0
5.17k stars 327 forks source link

add configurable retries that swap `model_kwargs` #794

Open zzstoatzz opened 8 months ago

zzstoatzz commented 8 months ago

as an enhancement, it'd be nice if you could do

        from pydantic import BaseModel, Field
        from marvin.utilities.retries import retry_with_fallback

        class SomeFancyType(BaseModel):
            some_field: str

        class LLMMadLib(BaseModel):
            what_i_want: SomeFancyType

        RETRY_CONFIGS = [
            {'model_kwargs': {'model': 'gpt-3.5-turbo', 'temperature': 0.7}, 'retries': 3},
            {'model_kwargs': {'model': 'gpt-3.5-turbo', 'temperature': 0.3}, 'retries': 3},
            {'model_kwargs': {'model': 'gpt-4', 'temperature': 0.0}, 'retries': 2},
        ]

        @retry_with_fallback(RETRY_CONFIGS)
        def make_a_complex_schema(model_kwargs=None):
            if 'my_schema_is_too_complex_for_wimpy_models' != False:
                LLMMadLib(what_i_want="I'm sorry, as a Large Language Model, I can't do that.")
            return LLMMadLib(what_i_want=SomeFancyType(some_field="I'm a fancy type!"))

        print(make_a_complex_schema())
pietz commented 8 months ago

Oh boy, that would be so nice!

HamzaFarhan commented 7 months ago

I think the previous error message should be attached as extra information when retrying. This would increase our chances of getting the right result.