search

kserve / website

User documentation for KServe.
https://kserve.github.io/website/
Apache License 2.0
103 stars 109 forks source link

Add Huggingface Serving Runtime example with Llama2 #345

Closed gavrissh closed 2 months ago

gavrissh commented 4 months ago

Proposed Changes

netlify[bot] commented 4 months ago

Deploy Preview for elastic-nobel-0aef7a ready!

Name Link
Latest commit e699490a82701a8e63009161c027f5fd06bba064
Latest deploy log https://app.netlify.com/sites/elastic-nobel-0aef7a/deploys/6612fac7c24e7f00089b8b94
Deploy Preview https://deploy-preview-345--elastic-nobel-0aef7a.netlify.app
Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

gavrissh commented 4 months ago

Consider adding example using out-of-box runtime kserve/kserve#3395

I'll take this up next, Thanks!

lizzzcai commented 3 months ago

I think it is not so good to use Llama2 as an example, as the model has license and need GPU hardware, not so friendly for people who just want to try it out. Maybe a model that can be run in CPU will be better like bigscience/bloom-560m.

yuzisun commented 2 months ago

I think it is not so good to use Llama2 as an example, as the model has license and need GPU hardware, not so friendly for people who just want to try it out. Maybe a model that can be run in CPU will be better like bigscience/bloom-560m.

@lizzzcai Could you help add the bloom CPU example after merging this? still worth showing the llama2 example I think.

lizzzcai commented 2 months ago

@lizzzcai Could you help add the bloom CPU example after merging this? still worth showing the llama2 example I think.

Sure I can help to add CPU example after this is merged.

yuzisun commented 2 months ago

/lgtm /approve

oss-prow-bot[bot] commented 2 months ago

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: gavrishp, yuzisun

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[OWNERS](https://github.com/kserve/website/blob/main/OWNERS)~~ [yuzisun] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment