Research Cloud Service Pricing

[VeriHeppi]

Check pricing for the most popular cloud services, also available free credits to be used for testing.

[janivanhala]

Google Cloud (Vertex AI):

• Offers free tier(no LLM service?) and up-to $300 in free credits for new users.
• https://cloud.google.com/vertex-ai/pricing
• Text input is charged by every 1,000 characters of input (prompt) and every 1,000 characters of output (response). Characters are counted by UTF-8 code points and white space is excluded from the count, resulting in approximately 4 characters per token. Prediction requests that lead to filtered responses are charged for the input only.
• If a query context is longer than 128K, all tokens are charged at long context rates.
• PDFs are billed as image input, with one PDF page equivalent to one image.
• Gemini models are available in batch mode at 50% discount.
• Gemini 1.5 Flash (low latency):
  • Text input: $0.00001875 / 1k characters (=< 128K context window) to $0.0000375 / 1k characters ( > 128K context window).
  • Text output: $0.000075 / 1k characters to $0.00015 / 1k characters.
• Gemini 1.5 Pro:
  • Text input: $0.00125 / 1k characters to $0.0025 / 1k characters
  • Text output: $0.00375 / 1k characters to $0.0075 / 1k characters
• Gemini 1.0 Pro:
  • only support up to 32K context window.
  • Text input: $0.000125 / 1k characters
  • Text output: $0.000375 / 1k characters
• Context caching, you can reduce the cost of Gemini input token processing by 75%. https://cloud.google.com/vertex-ai/generative-ai/pricing#context-caching
• Partner models:
  • Mistral AI’s models:
  • Mistral Large (2407):
    • Input: $2 / million tokens
    • Output: $6 / million tokens
  • AI21 Lab's models (chatbot):
  • Jamba 1.5 Large:
    • Input: $2 / million tokens
    • Output: $8 / million tokens
  • Jamba 1.5 Mini:
    • Input: $0.20 / million tokens
    • Output: $0.40 / million tokens
  • Anthropic’s Claude models (writing helper):
  • Claude 3 Opus (complex understanding and response):
    • Input: $15 / million tokens
    • Output: $75 / million tokens

Microsoft Azure (OpenAI API):

• Provides $200 in free credits for the first 30 days.
• https://azure.microsoft.com/en-us/pricing/details/cognitive-services/openai-service/
• Language models are also now available in the Batch API that returns completions within 24 hours for a 50% discount on Global Standard Pricing. (available in Sweden central, possibly good for making the initial model for the data-structure) https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/batch?tabs=standard-input&pivots=programming-language-python
• EU West:
  • gpt-4o-2024-08-06 Global Deployment: 128K (max context) per 1000 tokens: $0.0025 (input) $0.01 (output)
  • GPT-4o Global Deployment: 128K $0.005 $0.015
  • GPT-4o-mini Global Deployment: 128K $0.00015 $0.0006
• Sweden Central, same as above plus:
  • gpt-4o-2024-08-06 Regional API 128K $0.00275 $0.011
  • GPT-4o Regional API 128K $0.005 $0.015
  • GPT-4o-mini Regional API 128K $0.000165 $0.00066
  • GPT-4 8K $0.03 $0.06
  • GPT-4 32K $0.06 $0.12
• Also has Price per hour, monthly and yearly.
• Has legacy options for GPT-3.5-Turbo

Amazon Web Services (AWS) (Bedrock)

• Free tier doesn't seem to have LLM service

• AWS Rapid Ramp Credits:

  • $300 credits for 6 months
  • Specific qualifications: https://aws.amazon.com/free/offers/
  • Has approval process

• https://aws.amazon.com/bedrock/pricing/

• Meta Llama:

  • Located in US region
  • Llama 3.1:
    • Instruct (8B): $0.00022 $0.00022
    • Instruct (70B): $0.00099 $0.00099
    • Instruct (405B): $0.00532 $0.016
  • Llama 3 (London):
    • Instruct (8B): $0.00039 $0.00078
    • Instruct (70B): $0.00345 $0.00455
  • Mistral AI:
  • Mistral 7B: $0.0002 $0.00026
  • Mixtral 8*7B: $0.00059 $0.00091
  • Mistral Large (24.02): $0.0052 $0.0156

• Possible Ireland option in Europe with slightly less cost.

• Also has it's own LLM (Amazon Titan Text)

• Amazon Bedrock Guardrails for free sensitive information filter and word filters.

• Batch support located in US region (-50% price).

IBM Cloud (Watson NLP):

• Offers $200 in free credits, which can be used for Watson NLP services that include pre-trained models and APIs for natural language processing.
• Has certain amount of usage per month for free.
• Does not have direct support for Llama, Mistral or GPT.

Summary table:

	Google Cloud (Vertex AI)	Microsoft Azure (OpenAI API)	Amazon Web Services (Bedrock)
Free credits	$300	$200	$300 with approval process
Batch mode (-50% price)	Only Gemini	Only GPT	Yes
LLM's	Gemini, Mistral(partner)	GTP	Llama, Mistral
Input (<128k):	Gemini 1.5 Pro: $0.00125 per 1k chars ($0.005 per 1k tokens) Mistral Large (2407) $2 per million tokens ($0.002 per 1k tokens)	gpt-4o-2024-08-06: $0.0025 per 1k tokens	Llama 3.1 (70B) $0.00099 Llama 3.1 (405B) $0.00532 Mistral 7B $0.0002 Mistral 8*7B $0.00059 Mistral Large (24.02) $0.0052 per 1k tokens
Output:	Gemini 1.5 Pro: $0.00375 per 1k chars ($0.015 per 1k tokens) Mistral Large (2407) $6 per million tokens ($0.006 per 1k tokens)	gpt-4o-2024-08-06: $0.011 per 1k tokens	Llama 3.1 (70B) $0.00099 Llama 3.1 (405B) $0.016 Mistral 7B $0.00026 Mistral 8*7B $0.00091 Mistral Large (24.02) $0.0156 per 1k tokens

[janivanhala]

Fixed pricing table (above).

Notes:

• AWS requiring approval process for free credits. Most likely means we would only get $300 once.
• Google supports over 128k context inputs with double the price (Gemini only).
• From #7 , Mistral possibly being locally available.
• GPT-4o (Microsoft) is cheaper than Gemini 1.5 Pro (Google). Even when accounting free credits.

Possible use-case:

• Getting classification result: Large input but small output. ("training" part)
• Processing use-data with resulted classifications: Equally large input and output.

Result: Microsoft Azure.