JWT validation. When enabled (jwt_sign.pem file is present – ecdsa pubkey), it will force the limit of models per user to 1
Removed cbor on the wire, refactor inputs and outputs into message TensorData struct, as well as tensor facts for RunModel and SendModel
Make optimize optional. Not in protobuf yet, but implemented server side. Can be used in startup models (from config.toml file)
Make loading models from config.toml file possible
Make loading huge models from config.toml file possible
Model can be sealed, defaults to false.
"send_model = false" in config.toml
Tensors may have names. Not yet used in blindai client, but it's here on the wire.
Max model store (number) for blindai as a service. Defaults to 0, which does not change anything. If set, older models will be removed.
Update tract, because it was needed for big model support from filesystem.
Minor changes like adding a lot of context to existing errors (anyhow .context("xx"))
Add DatumTypes U8, I8, U16, I16, Bool
Added support for pytorch / numpy tensors directly on RunModel (in client!)
Client: Support using Mithril Cloud Services
Config entry: send_model_requires_auth (bool)
Config entry: allow_sendmodel (bool)
Config entry: max_model_store (int). When set, models that are pushed beyond this limit will remove old models.
Left to do
Edit the tests, make them pass, write more.
Documentation for each of these features.
Update examples.
Update readme.
Related Issue
None
Type of change
Warning: The changes to the python API are not backward compatible.
[x] This change requires a documentation update
[x] This change affects the client
[x] This change affects the server
[x] This change affects the API
[x] This change only concerns the documentation
How Has This Been Tested?
Please describe in detail how you tested your changes. Include details of your testing environment, and the tests you ran to see how your change affects other areas of the code, etc.
Checklist:
[ ] My code follows the style guidelines of this project
[ ] I have performed a self-review of my code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] My changes generate no new warnings
[ ] I have updated the documentation according to my changes
Description
Lots of things.
send_model_requires_auth(bool)allow_sendmodel(bool)max_model_store(int). When set, models that are pushed beyond this limit will remove old models.Left to do
Related Issue
None
Type of change
Warning: The changes to the python API are not backward compatible.
How Has This Been Tested?
Please describe in detail how you tested your changes. Include details of your testing environment, and the tests you ran to see how your change affects other areas of the code, etc.
Checklist: