Configure larger disks for bigger models

[mathemakitten]

• If one of the models to be evaluated is listed in DISK_NEEDED_FOR_LARGE_MODELS, sum up the approximate disk space needed for all of the big models
• Pass the total amount of disk space needed to the payload
• Else we assume that the models are probably all small enough to fit within 150GB of disk
• Downside: we need to manually add large models to the dictionary, along with their estimated size. I didn't see a simple way to estimate how big a model would be on disk for an arbitrary model uploaded to the Hub, but please let me know if you know a better way to do this!

Minimal example test:

DISK_NEEDED_FOR_LARGE_MODELS = {"opt-66b": 200}
selected_models = ['opt-66b', 'opt-13b']
size_of_models_on_disk = sum(filter(None, [DISK_NEEDED_FOR_LARGE_MODELS.get(model) for model in selected_models]))
max(size_of_models_on_disk, 150)

200

[mathemakitten]

I'm in favor of just making the disk space large enough for all zero shot submissions, so we don't need the large model dict. Disk space shouldn't cost that much anyway unless I'm missing something.

OK! In that case I've just bumped up the default to 200 GB, which accounts for 145GB for 66B and then some for the system, saving preds, etc. It also assumes that we'll route requests for models > 66B elsewhere (for now; we can change it when we start supporting 175B+ inference)—let me know if that works.