Open olsonadr opened 2 years ago
Now that I'm thinking about it, it might be nice to have a single config file in the inference
directory like inference/lambda_config.json
that describes each lambda. For the lambdas in #11 this might look like:
{
"lambdas": [
{
"name": "text-to-db-similar",
"handler_script": "text-to-db-similar.py",
"required_resources": [
"model/*",
"helpers/longformer_helpers.py",
"helpers/cosine_helpers.py"
]
},
{
"name": "embed-to-db-similar",
"handler_script": "embed-to-db-similar.py",
"required_resources": [
"model/*",
"helpers/longformer_helpers.py",
"helpers/cosine_helpers.py"
]
}
]
}
where each path is a relative path starting from the inference
directory
A lot of code will be re-used between lambdas so might as well factor out some of this code for future development. If it makes sense I might do this before starting on #15 so that they can use these.
I'm imagining some amount of config could be done per-lambda (maybe in a config file adjacent to each lambda handler script) so you can tell the Dockerfile which files to parcel into their containers, as not all future lambdas would need a copy of the model, all the helper functions, etc.
Common theme: I haven't done this yet to make sure we're feature-complete by Fri, but it pained me a little to ignore it