search

opendatahub-io / caikit-nlp

Apache License 2.0
0 stars 5 forks source link

[P0] Create CAIKit-nlp community images/CI for Open Data Hub #2

Closed heyselbi closed 1 year ago

Jooho commented 1 year ago

@vaibhavjainwiz @xianli123 I think we don't need image/ci for caikit-nlp because we will use caikit-nlp from pypi library. Am I right?

xianli123 commented 1 year ago

Hi, @Jooho I am sorry, but I don't have the answer. I am a UX designer and not so familiar with the question. We may need to wait for @vaibhavjainwiz 's reply.

vaibhavjainwiz commented 1 year ago

@Jooho I have same understanding we don't need image/ci for caikit-nlp. @danielezonca @Xaenalt Could you please confirm.

danielezonca commented 1 year ago

We need to decide how we prefer to proceed here: caikit-nlp is a python project so we can publish it to pypi but at the same time we need to build the actual runtime image with the entry point. At the moment @Xaenalt created a custom Dockerfile that has a simple start-serving.sh entry point We can keep the image creation part in another repo but I think it is easier to add the dockerfile + scripts (if necessary) directly in the caikit-nlp (Note: there is this entry-point script already in the repo that can be used)

Long story short: I expect we will publish caikit-nlp to pypi but we will also create the image to be used for the serving. I think we can focus directly on the image part

@Xaenalt Have you already discussed with caikit team if/when they plan to publish caikit-nlp to pypi?

Xaenalt commented 1 year ago

We do need a CI image for the caikit runtime, which includes caikit-nlp

Separately, caikit-nlp will eventually be pushed to pypi (there are some ongoing efforts around this work, check the IBM slack)

Xaenalt commented 1 year ago

As for the entrypoint script, I only took the smaller part for two reasons: one is that it was a test image, and two is that they are actively refactoring the handling of connections, https://github.com/caikit/caikit/pull/268 is the issue for that. Based on that work, the start script will need some reworking, for now it's just an example of how to launch it in a minimal configuration

Xaenalt commented 1 year ago

So, current working images that we use for KServe testing are:

TGIS: quay.io/opendatahub/text-generation-inference@sha256:04e3a0d84b17614853990955a6372b7647ad66ee663fd5d162fae406efadd42b Caikit-nlp: quay.io/opendatahub/caikit-tgis-serving@sha256:20cbc7b312e0dc63903d4a2c7d70b741135ffea31113af63dd8691da4baae0f9