search

deepjavalibrary / djl

An Engine-Agnostic Deep Learning Framework in Java
https://djl.ai
Apache License 2.0
4.17k stars 660 forks source link

[llamacpp] Removing llamacpp support in DJL #3312

Closed frankfliu closed 4 months ago

frankfliu commented 4 months ago

Description

Brief description of what this PR is about

vaiju1981 commented 4 months ago

Just trying to understand why we are removing llamacpp engine? is there a replacement planned to support LLM's via DJL?

frankfliu commented 4 months ago

Just trying to understand why we are removing llamacpp engine? is there a replacement planned to support LLM's via DJL?

  1. We don't have maintainer can actively maintain this module.
  2. We we focused on supporting LLM via Python engine in DJLServing repo
  3. We working on rust engine and will support rust based LLM support in future.