Right now, the gen servers load an API key from the environment, and use that key to call out to the model backends.
This will stop scaling as soon as the server gets hosted publicly.
Going forward, each request to the http endpoints ie generate_signature and generate_code should include an API key as part of the payload (just like they currently take a model). This key then gets fed into whatever model will do the generation.
The servers should not load any api keys from the env.
This means that each step in the generation pipeline must include its own model/api-key pairs, provided not by the server but by the request.
The demos will need to be updated to use this new format. The demos of course can load the keys from .env files.
Right now, the
genservers load an API key from the environment, and use that key to call out to the model backends.This will stop scaling as soon as the server gets hosted publicly.
Going forward, each request to the http endpoints ie
generate_signatureandgenerate_codeshould include an API key as part of the payload (just like they currently take amodel). This key then gets fed into whatever model will do the generation.The servers should not load any api keys from the env.
This means that each step in the generation pipeline must include its own model/api-key pairs, provided not by the server but by the request.
The demos will need to be updated to use this new format. The demos of course can load the keys from .env files.