I am using the pytorch and ResNet model serving with bentoml in production. Using this model, I developed an API to extract features of two images and compare similarities.
However, sometimes ClientOSError occurs, so I wonder if it is possible to correct the cause on the code.
ClientOSErroraiohttp.streams in read
To reproduce
[1] save_model.py
weights = ResNet50_Weights.DEFAULT
model = resnet50(weights=weights)
model.eval()
return_nodes = {'avgpool': 'avgpool'} # extract feature to save it to variable
new_model = create_feature_extractor(model, return_nodes=return_nodes)
bentoml.pytorch.save_model("model", new_model)
[4] Outout
It is working well, but sometimes return the error.
Expected behavior
I wonder if the model and service code are well-organized to distribute to the production environment.
And, I wonder what kind of defense code will be needed to prevent the error from occurring.
Describe the bug
I am using the pytorch and ResNet model serving with bentoml in production. Using this model, I developed an API to extract features of two images and compare similarities. However, sometimes ClientOSError occurs, so I wonder if it is possible to correct the cause on the code.
ClientOSErroraiohttp.streams in read
To reproduce
[1] save_model.py
[2] service.py
[3] save model and run
[4] Outout It is working well, but sometimes return the error.
Expected behavior
I wonder if the model and service code are well-organized to distribute to the production environment. And, I wonder what kind of defense code will be needed to prevent the error from occurring.
Environment
[1] python: python3.9-slim-buster docker image [2] requirements.txt