Currently we are using the same model & server endpoint for sdg teacher model and eval judge model. In real scenarios, there will be a separate teacher model server and judge model server. Will need to update the pipeline and standalone scripts to accommodate separate models:
For example, standalone and kfp pipeline should expect:
configmaps:
judge-server
teacher-server
secrets:
judge-server
teacher-server
Currently expected:
standalone
only 1 secret, judge-server-details secret is used (sdg isn't fully implemented in standalone yet) for all server detail
kfp pipeline
sdg uses kfp-model-server secret for sdg and judge-server secret for eval
sdg uses kfp-model-server configmap and eval uses judge-server configmap to find the server endpoint & model-name
Currently we are using the same model & server endpoint for sdg teacher model and eval judge model. In real scenarios, there will be a separate teacher model server and judge model server. Will need to update the pipeline and standalone scripts to accommodate separate models:
For example, standalone and kfp pipeline should expect:
configmaps:
secrets:
Currently expected:
standalone
judge-server-detailssecret is used (sdg isn't fully implemented in standalone yet) for all server detailkfp pipeline
kfp-model-serversecret for sdg andjudge-serversecret for evalkfp-model-serverconfigmap and eval usesjudge-serverconfigmap to find the server endpoint & model-name