kserve / modelmesh-minio-examples

ModelMesh example models packaged into a MinIO container for user exploration and functional verification testing
Apache License 2.0
2 stars 18 forks source link

[WIP] Don't upload unused files #1

Closed ckadner closed 1 year ago

ckadner commented 1 year ago

Don't merge this PR!

This is just for the record to keep a "copy" of the unused model files in the v0.10.0 modelmesh-mino-examples and modelmesh-minio-dev-examples images

The files proposed (NOT) to be added in this PR are from @tedhtchang 's PR here https://github.com/kserve/modelmesh-serving/pull/300

The repository does not contain those large files as they are either duplicates of other files or because they are not used for the ModelMesh FVT purposes (anymore).

The files which did not make it from Ted PR into this repository are the following:

This helped shaving off about 315 MB from the size of this repo -- before any of those files were ever committed, since Git would keep a history of the files and the repo would stay larger even if we deleted those file subsequently.

For the Tensorflow models in the first commit, there are more duplicates that are excluded here:

ckadner commented 1 year ago

Hint to future self, pushing those large files failed repeatedly until setting a few git flags:

# increase some buffer sizes
git config http.postBuffer 50M
git config http.maxRequestBuffer 50M
git config core.compression 0

# cleanup some git internal caches
git gc

# enable some tracing
export GIT_TRACE_PACKET=1
export GIT_TRACE=1
export GIT_CURL_VERBOSE=1

# repeat push with extra verbose debug output
git push -vvv