Maximum batch size set by tf_serving pod instead of consumer

[elaubsch]

Describe the bug The batch size used for each consumer is determined by the batch size of the model run by the consumer. The GrpcModelWrapper.get_batch_size in grpc_clients.py to determine this. The maximum batch size for a consumer is determined with the environment variable TF_MAX_BATCH_SIZE in the helmfile for each consumer. If this value is larger than the maximum batch size set by the environment variable MAX_BATCH_SIZE in the tf-serving helmfile, the prediction will fail.

To Reproduce Steps to reproduce the behavior:

1. Set TF_MAX_BATCH_SIZE in the helmfile for the consumer to a number larger than MAX_BATCH_SIZE in the tf-serving helmfile. For example, for the segmentation consumer, set TF_MAX_BATCH_SIZE in the segmentation consumer helmfile to 64 and ensure that MAX_BATCH_SIZE in the tf-serving helmfile is 32.
2. Upload an image to the kiosk for prediction.
3. See error: "Task size 64 is larger than maximum input batch size 32"

Expected behavior Enforce that the maximum batch size of the consumer is smaller than the maximum batch size of tf-serving before prediction.

[msschwartz21]

An additional complication with this issue is that the maximum number of frames that can be tracked is directly limited by the maximum batch size for tensorflow serving.