More Model API support is needed

[wangchen615]

To fetch the models, we should support COS more than IBM COS (We probably only need a diagram update, as our APIs are S3 APIs) and allow models to be fetched from HuggingFace directly. In addition, we can allow users to fetch models downloaded locally on their disks.

[rohanarora]

Chen (@wangchen615), Thanks for bringing this up. My experience has been the opposite i.e. no support for COS / IBM COS.

At this time, fmperf does seem to support:

1. Loading models from Hugging Face. For TGHISModelSpec this can be achieved by setting download_weights=True. When the model needs an agreement to be accepted (e.g. LLAMA family of models), one needs to introduce a variable key value pair in get_vars of TGISModelSpec here:

   {
   "name": HUGGING_FACE_HUB_TOKEN",
   "value": "${HUGGING_FACE_TOKEN}"
   }

   Though we may want to consider surfacing this to the top. That way a user won't have the need to tinker around with ModelSpecs.py.

2. Loading models from local disk is enabled via extraMounts in a way