Both the utils for the Anthropic's MM-LLM and OpenAI's MM-LLM rely on logic that both duplicates that found in generic_utils (specifically for encoding) and can also be made more concise.
The new generic_utils.set_base64_and_mimetype_for_image_docs provides a utility which both encodes and detect the mime type more concisely than the existing MM implementations. A future PR will then update the OpenAI and Anthropic implementations to utilize this, thereby making their code more concise and legible.
Anthropic LLM integration had incorrect max context windows of 180k rather than the 200k supported by Anthropic. Updated to 200k.
Added missing Bedrock models and added support for US & EU inference profiles.
Type of Change
[x] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
Version Bump?
Did I bump the version in the pyproject.toml file of the package I am updating? (Except for the llama-index-core package)
[x] Yes - llama-index-llms-anthropic
[ ] No
How Has This Been Tested?
[x] I added new unit tests to cover this change
Suggested Checklist:
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] My changes generate no new warnings.
[x] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes
[x] I ran make format; make lint to appease the lint gods
Description
Both the utils for the Anthropic's MM-LLM and OpenAI's MM-LLM rely on logic that both duplicates that found in
generic_utils(specifically for encoding) and can also be made more concise.The new
generic_utils.set_base64_and_mimetype_for_image_docsprovides a utility which both encodes and detect the mime type more concisely than the existing MM implementations. A future PR will then update the OpenAI and Anthropic implementations to utilize this, thereby making their code more concise and legible.Type of Change
Version Bump?
Did I bump the version in the
pyproject.tomlfile of the package I am updating? (Except for thellama-index-corepackage)llama-index-llms-anthropicHow Has This Been Tested?
Suggested Checklist:
make format; make lintto appease the lint gods