Everything OK? Abandoned?

TheBloke commented 8 months ago

Hi @marella

Hope all is OK with you. I'm getting various reports on my repos with issues with CTransformers, and I see it's not been updated for nearly 2 months now.

Is everything OK? Should I consider this project no longer being updated? That'd be a shame!

All the best

Tom

gerred commented 8 months ago

@TheBloke we're fairly dependent on this on LeapfrogAI, have been bringing it up to date locally. If this is unfortunately abandoned, we may fork it and keep it going because it's a great tool.

gphorvath commented 8 months ago

Also interested! @gerred, my team, and I have been developing a backend module for LeapfrogAI based on ctransformers! https://github.com/defenseunicorns/leapfrogai-backend-ctransformers

marella commented 8 months ago

Hi @TheBloke, I got busy with some personal stuff so wasn't able to release updates. I already have some changes in my local that I was planning to release last week but didn't get time to finish. I will try to release them this weekend. Next update will include some major cleanups which should make future updates easier.

phoenixthinker commented 7 months ago

@TheBloke 🥇 💯 👍 @marella 🥇 💯 👍

Both of you are my Hero! I learned LLM application design and integration with your LLMs and the ctransformers lib!

TheBloke commented 7 months ago

Great to hear you're still around marella - looking forward to the updates! It'd be great if ctransformers could support the recent changes to llama.cpp, eg BPE vocab, StableLM support, etc.

luohao123 commented 7 months ago

Any updates for now? I got some old libc error on my server, is there FAQ can refer to resolve it?

davidearlyoung commented 6 months ago

I hope that everything is ok. I like this library.

luohao123 commented 6 months ago

I moved to AWQ quantization which integrated in transofmers, it fast and easy to use.

davidearlyoung commented 6 months ago

I moved to AWQ quantization which integrated in transofmers, it fast and easy to use.

@luahao123 I'm wondering how else that compares to GGUF's loaded through llama-cpp? I've yet to try working with AWQ. I've only ran LLM"S with GPTQ and GGUF. And a bit of GGML. (Old school GGUF.) I understand a bit of the concept of how it's different from doing a bit of light reading about it. I'm currently testing with llama-cpp through the llama-cpp-python c/c++ bindings to see how it goes for my needs. Pretty happy with it so far. I originally avoided llama-cpp and started with ctransforms because it was easier to get started with GPU. Fun fact, I think llama-cpp may support AWQ now with it's latest release. Still early stages though.

DrewWalkup commented 6 months ago

I moved to llama-cpp as well. I'm developing cross-platform and it allows me to leverage metal and cuda acceleration with one code base.

On Fri, Jan 5, 2024 at 3:40 PM David Young @.***> wrote:

I moved to AWQ quantization which integrated in transofmers, it fast and easy to use.

@luahao123 I'm wondering how else that compares to GGUF's loaded through llama-cpp? I've yet to try working with AWQ. I've only ran LLM"S with GPTQ and GGUF. And a bit of GGML. (Old school GGUF.) I understand a bit of the concept of how it's different from doing a bit of light reading about it. I'm currently testing with llama-cpp through the llama-cpp-python c/c++ bindings to see how it goes for my needs. Pretty happy with it so far. I originally avoided llama-cpp and started with ctransforms because it was easier to get started with GPU. Fun fact, I think llama-cpp may support AWQ now with it's latest release. Still early stages though.

— Reply to this email directly, view it on GitHub https://github.com/marella/ctransformers/issues/175#issuecomment-1879383849, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKVLQKP42333BJZ3CHHW2W3YNCFQDAVCNFSM6AAAAAA627GUQOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNZZGM4DGOBUHE . You are receiving this because you commented.Message ID: @.***>

marella / ctransformers

Everything OK? Abandoned? #175