-
Dispatch with model ID to a machine learning online service with a CLI.
- The model will first be downloaded or already downloaded by model hub manager
- The model will be dispatched to as a service …
-
### Willingness to contribute
Yes. I can contribute this feature independently.
### Proposal Summary
The inferred requirements.txt is immensely valuable for future reference. However, it may not en…
-
**Prerequisites**
> Please fill in by replacing `[ ]` with `[x]`.
* [x] Are you running the latest `bert-as-service`?
* [x] Did you follow [the installation](https://github.com/hanxiao/bert-as-…
-
Full-sized images used for training are split between the EOL images on Hugging Face and the iNat21 and BIOSCAN image sets through their own distribution sources.
However, the images were resized t…
-
failed to create predictor: Log_id: 0 Raise_msg: (NotFound) Cannot open file serving_server/__model__, please confirm whether the file is normal.
[Hint: Expected static_cast(fin.is_open()) == true…
-
I'm looking into this package as an option for serving data science models at my company. We have a security compliance policy that dictates we can only use http server frameworks that support/ are co…
-
we are converting our RESTier server to a Multi-tenant server and I was wondering if RESTier has support for that.
is there a built-in way to register an EDM model per tenant?
and if not what would…
-
In ODH Dashboard Model Serving, there is the possibility to define access (url, credentials and so on) to S3-compatible storage buckets for model files (described in ODH Dashboard as Data Connecti…
-
when use flask deploy a fine tuned- classification task online, each post will reload the model. this takes about 4 seconds to respond, is there any solutions to avoid reloading?
-
Co-authored with @SolitaryThinker @Yard1 @rkooo567
We are landing multi-step scheduling (#7000) to amortize scheduling overhead for better ITL and throughput. Since the first version of multi-step…