Updating the behavior of LmiConfigRecommender to match the behavior of the default properties when using a serving.properties for NeuronX containers. The recommender currently defaults all models to rolling batch, but this is not the desired behavior for LMI NeuronX due to the limited number of supported models for rolling batch.
This undoes a change that was a altered the default as aa side effect of the recommenders implementation but is not consistent in defaulting between launching with environmental variables and serving.properties for LMI NeuronX.
If this change is a backward incompatible change, why must this change be made?
Description
Updating the behavior of LmiConfigRecommender to match the behavior of the default properties when using a serving.properties for NeuronX containers. The recommender currently defaults all models to rolling batch, but this is not the desired behavior for LMI NeuronX due to the limited number of supported models for rolling batch.
This undoes a change that was a altered the default as aa side effect of the recommenders implementation but is not consistent in defaulting between launching with environmental variables and serving.properties for LMI NeuronX.