API call fails with a medium-sized system prompt (270 tokens) while working with smaller ones

[turgor111]

Hi there, I know that the APIConnectionError relates to network issues, however I wanted to clarify this particular situation.

I have a system prompt BASE_SYSTEM that's 270 tokens. It times out after a minute when I run basic completion:

client = Groq(
    api_key=os.environ.get("GROQ_API_KEY"),
)

chat_completion = client.chat.completions.create(
    messages=[
        {
            "role": "system",
            "content": BASE_SYSTEM,
        },
                {
            "role": "user",
            "content": 'hi',
        }
    ],
    model="llama3-8b-8192",
    temperature=0
)

However, if I include no system prompt or make it one sentence, the server responds within 5 seconds. Where might the problem lie in this case? I don't seem to be having network issues that would indicate this is the problem on my end.

[turgor111]

What can't a good ol' system reboot solve? ;)