Closed jchauhan closed 10 months ago
You are trying to access the frozen model and that's empty, ill create an example for you in next commit.
i have tested all of the examples and all of them are working fine use this as a mindset example
from EasyDel import (
JAXServer,
JAXServerConfig
)
server_config = JAXServerConfig(
top_k=40,
top_p=0.9,
temperature=0.8,
max_length=2048,
max_new_tokens=4096,
max_compile_tokens=128
)
model_id = "TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T"
server = JAXServer.from_torch_pretrained(
server_config,
model_id
)
tkns = 0
for response, tokens_generated in server.process(
"```python\ndef make_pytorch_linear("
):
print(response[tkns:], end="")
tkns += len(response)
Describe the bug Error while serving a toy model using EasyDel with the following exception.
To Reproduce
Context