search

wilkox / GPTscreenR

📚GPT-assisted source screening for scoping reviews
Other
2 stars 1 forks source link

Can it support local large language models? #1

Open Otoliths opened 2 months ago

Otoliths commented 2 months ago

@wilkox

Thanks for creating GPTscreenR! It’s a super useful tool for scoping reviews. I noticed that it currently supports GPT-4 through the OpenAI API, which works great, but I was wondering if you’ve considered adding support for local large language models in future updates. This could make the tool more flexible, especially for users who want to cut down on API costs or work in environments where internet access isn’t reliable.

I recently came across a paper called “Evaluating the effectiveness of large language models in abstract screening: a comparative analysis,” and it got me thinking about how adding this capability could really broaden GPTscreenR’s appeal.

Just a thought—thanks for all your hard work on this!

Cheers, Liuyong

wilkox commented 2 months ago

Thanks for your kind words Liuyong. Do you have a particular local LLM in mind? It wouldn't be too hard to adapt GPTscreenR to use a local model.

Otoliths commented 2 months ago

Thanks for your kind words Liuyong. Do you have a particular local LLM in mind? It wouldn't be too hard to adapt GPTscreenR to use a local model.

Thanks. One local LLM platform that I recommend is Ollama. It allows users to run various large language models(e.g., llama3.1) directly on their local machines, which could be a great fit for GPTscreenR.

Otoliths commented 2 months ago

@wilkox Hi, I wanted to recommend the SYNERGY dataset (26 systematic reviews) as a potential resource for testing and evaluating GPTscreenR’s performance. It’s a free dataset that might offer valuable insights into how the tool works in real-world scenarios, which could help guide future updates and improvements to the package.

wilkox commented 1 month ago

Thanks for these suggestions. I do think it would be a good idea to add support for local models, to do this I'll need to make some modifications to my lemur package. It might take me some time to get around to this though.

wilkox commented 2 weeks ago

@Otoliths I've just updated GPTscreenR to support local LLMs with ollama. You'll need to install the new lemur 0.2.0 first with install_github("wilkox/lemur"). You can try it with screen(..., service = "ollama", model = "llama3.2").

Otoliths commented 2 weeks ago

@wilkox Awesome! I can’t wait to try it out. I’ll install the new version of lemur right away and test ollama with the llama3.2model in GPTscreenR.