Config option to increase or disable model load timeout

[svenkata9]

Hi,

For test purposes, I need to increase the model load timeout (or rather disable). Is there a way to do that via config.properties? I am using the prebuilt docker image, so an option via config.properties would be nice.

I did take a look into the code: https://github.com/awslabs/multi-model-server/blob/1fee69499137522aca6946275b298a2bf1c01baf/frontend/server/src/main/java/com/amazonaws/ml/mms/ModelServer.java#L219

https://github.com/awslabs/multi-model-server/blob/1fee69499137522aca6946275b298a2bf1c01baf/frontend/server/src/main/java/com/amazonaws/ml/mms/util/ConfigManager.java#L522

But, I am a bit confused - because in my environment the model load timeout happens in 60 seconds whereas this code seems to indicate the timeout is 120 seconds. So, perhaps I am not looking at the right code.

Thanks.

[svenkata9]

Some more info - I set the default_response_timeout=360 in the custom config, but still it fails. Can someone clarify if there is a way to extend the timeout for model load.

2023-02-07 04:59:27,869 [WARN ] main com.amazonaws.ml.mms.ModelServer - Failed to load model: onnx-mobilenet.mar
java.util.concurrent.TimeoutException
        at java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1784)
        at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1928)
        at com.amazonaws.ml.mms.wlm.WorkLoadManager.addServerThread(WorkLoadManager.java:182)
        at com.amazonaws.ml.mms.wlm.ModelManager.startBackendServer(ModelManager.java:180)
        at com.amazonaws.ml.mms.wlm.ModelManager.registerModel(ModelManager.java:134)
        at com.amazonaws.ml.mms.ModelServer.initModelStore(ModelServer.java:212)
        at com.amazonaws.ml.mms.ModelServer.start(ModelServer.java:315)
        at com.amazonaws.ml.mms.ModelServer.startAndWait(ModelServer.java:103)
        at com.amazonaws.ml.mms.ModelServer.main(ModelServer.java:86)