search

meshachaderele / ddsc-llm

1 stars 0 forks source link

Data on HF? #2

Open KasperGroesLudvigsen opened 9 hours ago

KasperGroesLudvigsen commented 9 hours ago

Hey, do you want to put the data that you create with this notebook on HuggingFace? :) That way we can easily download it (and you get to share it with others :) ) https://github.com/meshachaderele/ddsc-llm/blob/main/process.ipynb

meshachaderele commented 9 hours ago

Yes. I will put it on HugginFace. Let me know how to best to name it when I do so.

KasperGroesLudvigsen commented 9 hours ago

Hm, good question. Maybe something that indicates the original name of the dataset?

meshachaderele commented 8 hours ago

Hey, do you want to put the data that you create with this notebook on HuggingFace? :) That way we can easily download it (and you get to share it with others :) ) https://github.com/meshachaderele/ddsc-llm/blob/main/process.ipynb

I have the dataset on HF now. https://huggingface.co/datasets/meshachaderele/negative-positive-wikipedia-2023-11-da

meshachaderele commented 8 hours ago

I will try the vllm you suggested on the dataset to generate the query.

KasperGroesLudvigsen commented 8 hours ago

That's actually what I am going to commit in a sec :)

KasperGroesLudvigsen commented 7 hours ago

I just pushed to the branch "kasper". It works nicely with vllm and even the 8b model produces reasonable responses. Am going to do a vibe check with expanse 32b

https://github.com/meshachaderele/ddsc-llm/tree/kasper

KasperGroesLudvigsen commented 7 hours ago

It's still WiP tho..