search

arakoodev / EdgeChains

EdgeChains.js is Full-Stack GenAI library. Front-end, backend, apis, prompt management, distributed computing. All core prompts & chains are managed declaratively in jsonnet (and not hidden in classes)
https://www.arakoo.ai/
GNU Affero General Public License v3.0
350 stars 70 forks source link

Text Classification via Large Language Models #95

Open sandys opened 1 year ago

sandys commented 1 year ago

https://arxiv.org/abs/2305.08377

shrey-a-gupta commented 1 year ago

Hi @sandys , I am willing to work on this issue. Please guide me to get started with this.

sandys commented 1 year ago

@shrey-a-gupta we are holding a training session on discord on thursday. please attend it and after that we can decide to assign it to you.

sandys commented 1 year ago

https://twitter.com/_ScottCondron/status/1670827747684364288

shrey-a-gupta commented 1 year ago

@sandys I will surely be attending the session. Please let me know about the timings, so that i may update my calendar and be available for the same.

NtaylorOX commented 1 year ago

Any updates on this?

sandys commented 1 year ago

hi @NtaylorOX - are you looking for implementation of classification here ? it was not super high on our priority list, but if you are actively looking to use it...ill try to bump it up.

can you describe your usecase ? it will help us structure it properly.

NtaylorOX commented 1 year ago

Hello, thanks for the reply. In all honesty, I stumbled upon this after failing to find the implementation from the authors of the paper - it appears the github they link with the paper is no longer existing. I'm also very willing to try help implement this, although admit it may be beyond my skillset to do so within EdgeChains codebase.

I'm sorry if this isn;'t at all helpful. I'll try to think of a more concrete usecase and come back here

sandys commented 1 year ago

https://www.trygloo.com/blog/classify-text-llms-learnings

Create an LLM-classifier with prompt engineering, that is guaranteed to only output your specified classes. Quantifiably measure how prompt changes impacts production (latency, accuracy, biases in class selection) Train + deploy a traditional BERT based classifier based on your LLM data Build a classifier that combines both the trained model + LLM for any new classes you didn’t train on

NtaylorOX commented 1 year ago

That is very helpful - and kind of what I have found in my playing around in this space. I guess the CARP approach, or any prompt based approach is relatively straightfoward but involves some amount of "fishing" for the right prompt. Glad to see smaller LLMs are still remaining relevant in this space.

sandys commented 1 year ago

https://arxiv.org/abs/2306.05816