Open 00zahra000 opened 1 year ago
@00zahra000 in order to serve models using Torchserve, you will need to create a model archive. More information about creating model archives can be found here. Model archives can includes the checkpoint file using the --serialized-file
option and is used at model loading time. Any additional data/files required by the model for initialization or during inference can be included using the --extra-files
option.
For configuration options such as batch_size
and num_worker
for a given model, a configuration file can be used to specify these. More details can be found here.
In order to use further options/arguments that will need to specified at the time of inference, you can consider implementing a custom handler and providing these options as input data to the predictions
endpoint. For example:
Hello everyone. In my code in my local system, this is how I get the arguments when I run my python file.
def get_opt():
I was wondering how I can write this in my handler file
This is the rest of the code, if it's helpful:
def test(opt, seg, gmm, alias):
def main():
if name == 'main': main()