cloudflare / workerd

The JavaScript / Wasm runtime that powers Cloudflare Workers
https://blog.cloudflare.com/workerd-open-source-workers-runtime/
Apache License 2.0
6.02k stars 285 forks source link

Python worker can't access env.AI methods like run after binding #1992

Open neeravkumar opened 5 months ago

neeravkumar commented 5 months ago

Can't create a python worker that can access the AI bindings

from js import Response

async def on_fetch(request, env):
    simple = "{'prompt': 'Tell me a random quote'}"
    response = await env.AI.run("@cf/meta/llama-2-7b-chat-int8", simple)
    return Response.new(response)

returns

Traceback (most recent call last):
  File \"/session/metadata/entry.py\", line 5, in on_fetch
    response = await env.AI.run(\"@cf/meta/llama-2-7b-chat-int8\", simple)
                     ^^^^^^^^^^
AttributeError: run
hoodmane commented 5 months ago

For context, in JavaScript you would write:

import { Ai } from '@cloudflare/ai'

export interface Env {
  // If you set another name in wrangler.toml as the value for 'binding',
  // replace "AI" with the variable name you defined.
  AI: any;
}

async function fetch(request: Request, env: Env) {
  const ai = new Ai(env.AI);
  const response = await ai.run('@cf/meta/llama-2-7b-chat-int8', {
      prompt: "What is the origin of the phrase Hello, World"
    }
  );
}

So the run method is on the Ai wrapper and not on env.AI itself. We need some plan for how to allow Python workers to access equivalent functionality.