OpenRouterTeam / openrouter-runner

Inference engine powering open source models on OpenRouter
https://openrouter.ai
MIT License
517 stars 49 forks source link

Support for the parameter `n` in Requests #99

Open ZachL1 opened 3 weeks ago

ZachL1 commented 3 weeks ago

Describe the bug Currently OpenRouter does not seem to support the use of parameter n in post requests. The OpenAI API is supported, see https://platform.openai.com/docs/api-reference/chat/create#chat-create-n

To Reproduce Using OpenRouter to chat with GPT4o, set the parameter n=5 but only get a chat completion choices. code:

import openai

# request gpt by openrouter
client = openai.OpenAI(
    base_url="https://openrouter.ai/api/v1",
    api_key="xxxxx",
)
model = "openai/gpt-4o-2024-05-13"

completion = client.chat.completions.create(
    model=model,
    messages=[
        {
            "role": "user",
            "content": "Are cats cute?",
        },
    ],
    n=5,
)

print(f'Response count: {len(completion.choices)}')
for i, choice in enumerate(completion.choices):
    print(f'Response {i}: {choice.message.content}')

Output:

Response count: 1
Response 0: The perception of cuteness can be quite subjective and varies from person to person. However, many people find cats cute due to their playful behavior, expressive eyes, soft fur, and intriguing personalities. Their quirky antics and affectionate nature often endear them to their owners. Whether a cat is considered cute or not largely depends on individual preferences and experiences with these animals.

Expected behavior Use OpenAI API to chat with GPT4o, set parameter n=5 and get 5 chat completion choices. code:

import openai

# request gpt by openai
client = openai.OpenAI(
    api_key="xxxxxx",
)
model = "gpt-4o-2024-05-13"

completion = client.chat.completions.create(
    model=model,
    messages=[
        {
            "role": "user",
            "content": "Are cats cute?",
        },
    ],
    n=5,
)

print(f'Response count: {len(completion.choices)}')
for i, choice in enumerate(completion.choices):
    print(f'Response {i}: {choice.message.content}')

Output:

Response count: 5
Response 0: Whether cats are considered cute is largely subjective and can vary from person to person. Many people find cats adorable due to their playful behavior, affectionate nature, and physical features such as big eyes, soft fur, and small size. On the other hand, some individuals might not find them as appealing for various reasons, such as allergies or a preference for other types of pets.

In general, cats are widely regarded as cute by a large segment of the population, which is why they are popular pets and frequent stars of internet memes and videos. Their quirky personalities and wide range of behaviors also contribute to their charm for many people.
Response 1: Many people find cats to be incredibly cute due to their various features and behaviors. Their large eyes, soft fur, playful antics, and often mysterious demeanor can make them endearing to a wide audience. Additionally, cats often exhibit behaviors like purring, kneading, and snuggling, which can enhance their cuteness. Of course, perceptions of cuteness are subjective, so while many find cats adorable, others may not feel the same way. Ultimately, whether or not you find cats cute depends on your personal preferences and experiences.
Response 2: Many people find cats to be cute due to their expressive eyes, soft fur, playful behavior, and often quirky personalities. Their small size and delicate features can evoke a nurturing response in humans, which adds to their appeal. Of course, perceptions of cuteness are subjective and can vary from person to person. What makes cats endearing to some might be different for others who might not share the same affection for them.
Response 3: Yes, many people find cats to be extremely cute! Their playful behavior, soft fur, expressive eyes, and curious nature often endear them to humans. Additionally, cats have a variety of endearing traits, such as purring, kneading, and their unique personalities, that many people find irresistible. Of course, perceptions of cuteness are subjective and can vary from person to person.
Response 4: Many people find cats to be incredibly cute! Their playful behavior, expressive eyes, and often affectionate nature can endear them to their owners. The appeal of cats can vary from person to person, but traits like their soft fur, the way they purr, and their unique personalities contribute to their widespread popularity as pets. Do you have a specific aspect of cats' appearance or behavior you're curious about?

Additional context Add any other context about the problem here.