search

SkywardAI / kirin

APIs aggregator for inference, fine-tuning and build models.
https://skywardai.github.io/skywardai.io/
Apache License 2.0
5 stars 7 forks source link

[Feature]: Inference function should be in more stable way. #186

Closed Aisuko closed 1 week ago

Aisuko commented 2 weeks ago

Contact Details(optional)

No response

What feature are you requesting?

Currently, we implement the inference through RESTful API. We should have more test-well functional methods to do this rather than only writing it in code directly. And also, tokenise and inference can be done in parallel