Have a consistent way to distinguish model parsers from models

A single ModelParser can be used to run inference for multiple models. For example, the AnyscaleEndpoint model parser (https://github.com/lastmile-ai/aiconfig/blob/main/python/src/aiconfig/default_parsers/anyscale_endpoint.py) supports inference for LLaMA, Mistral, CodeLLaMA, LLaMAGuard, and other models.

Similarly, Hugging Face task model parsers can each support inference for thousands of models.

In aiconfig today, it's hard to distinguish between models and model parsers. A "model parser" is really a "model inference provider" for all models that can be run in the same way.

For example, the following prompt is valid:

  {
      "name": "gen_itinerary",
      "input": "Generate an itinerary ordered by {{order_by}} for these activities: {{get_activities.output}}.",
      "metadata": {
        "model": {
          "name": "AnyscaleEndpoint",
          "settings": {
            "model": "mistralai/Mixtral-8x7B-Instruct-v0.1",
            "max_tokens": 3000,
            "system_prompt": "You are an expert travel coordinator with exquisite taste."
          }
        },
        "parameters": {
          "order_by": "geographic location"
        }
      }
    }

But, if we add a model_parsers mapping to the global metadata

    "model_parsers": {
      "meta-llama/Llama-2-7b-chat-hf": "AnyscaleEndpoint",
   }

then the same prompt can be expressed as:

    {
      "name": "gen_itinerary",
      "input": "Generate an itinerary ordered by {{order_by}} for these activities: {{get_activities.output}}.",
      "metadata": {
        "model": {
          "name": "mistralai/Mixtral-8x7B-Instruct-v0.1",
          "settings": {
            "model": "mistralai/Mixtral-8x7B-Instruct-v0.1",
            "max_tokens": 3000,
            "system_prompt": "You are an expert travel coordinator with exquisite taste."
          }
        },
        "parameters": {
          "order_by": "geographic location"
        }
      }
    }

Some things we need to clean up and simplify:

confusion between model.name and model.settings.model -- they are often duplicated for no good reason.
having cleaner distinguishability between the concept of "model parser" and the "model id"

A couple more thoughts here:

Ideally we could also just express as

{
  "name": "gen_itinerary",
  "input": "Generate an itinerary ordered by {{order_by}} for these activities: {{get_activities.output}}.",
  "metadata": {
    "model": {
      "name": "mistralai/Mixtral-8x7B-Instruct-v0.1",
      "settings": {
        "max_tokens": 3000,
        "system_prompt": "You are an expert travel coordinator with exquisite taste."
      }
    },
    "parameters": {
      "order_by": "geographic location"
    }
  }
}

if the model parser registry has a registered parser for mistralai/Mixtral-8x7B-Instruct-v0.1.

We currently also support registering model parsers for different models in 2 ways:

gpt_models = [
    "gpt-4",
    "gpt-4-0314",
    "gpt-4-0613",
    "gpt-4-32k",
    "gpt-4-32k-0314",
    "gpt-4-32k-0613",
    "gpt-3.5-turbo",
    "gpt-3.5-turbo-16k",
    "gpt-3.5-turbo-0301",
    "gpt-3.5-turbo-0613",
    "gpt-3.5-turbo-16k-0613",
]
for model in gpt_models:
    ModelParserRegistry.register_model_parser(DefaultOpenAIParser(model))

and

ModelParserRegistry.register_model_parser(DefaultOpenAIParser(), gpt_models)

with the former instantiating a new DefaultOpenAIParser class for each model ID (with the parser id being set to model id), and the latter reusing the same class instance (with default parser id) and associating it to multiple model ids.

We probably don't want to support both cases

lastmile-ai / aiconfig

Have a consistent way to distinguish model parsers from models #782