search

aws-samples / aws-samples-for-ray

MIT No Attribution
64 stars 23 forks source link

Add Ray serve example using AWS neuron-cores #15

Closed chappidim closed 1 year ago

chappidim commented 1 year ago

Description of changes:

This example helps customers to launch a cluster for serving inference model using Ray serve. Apart from Ray documentation, this example provides

  1. Ray cluster with trainium instance launch and appropriate neuron-sdk commands
  2. Ray Serve example using Open LLAMA to compile and deploy the model.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

I confirm