support for async dspy.LM

[JacksonCakes]

Hi, does Dspy currently support the asynchronous version of dspy.LM? which might be equivalent to the following:

async def call_llm(prompt: str,file:str, cur_type: str):
    headers = {"Content-Type": "application/json"}
    res_args = {
        "model": LLM_NAME,
        "messages": [
            {
                "role": "user",
                "content": prompt,
            },
        ],
        "temperature": 0,
    }
    async with httpx.AsyncClient(timeout=httpx.Timeout(LLM_TIMEOUT)) as client:
        response = await client.post(LLM_URL, json=res_args, headers=headers)
    response = response.json()["choices"][0]["message"]["content"].strip()

task = [call_llm(x) for x in item] 
result = asyncio.gather(*task)

[okhat]

Hey @JacksonCakes ! See #1734 and let us know what you think!

[JacksonCakes]

Thanks!! That's exact what I am looking for!

[okhat]

Wonderful. May I ask what you need async for? We're considering the right form of async to merge.

[JacksonCakes]

Any LLM call that can be run in parallel would be super useful here! A great use case from my side is finding the right SQL table based on user queries from a huge database. With so many tables, compress every table metadata into single prompt or doing this sequentially just isn't practical

[okhat]

Would you find this #1690 useful? We're probably going to go with that.

[JacksonCakes]

Yup, I think that would work too! But I am just curious why multi-threaded is preferred over #1734 ?