search

mlhubber / mlflutter

GNU General Public License v3.0
1 stars 0 forks source link

MLHUB: llm model for CHAT #3

Open gjwgit opened 1 month ago

gjwgit commented 1 month ago

Description

The ollama package (or llama2) could be interesting to demonstrate a CHAT command in mlhub.

Once we a re ready for this task, Dawei Chen may be able to give us a quick start with ollama. Then we package it in mlhub, then add a mlflutter interface for it.

Some old notes re llama2:

Graham to start a new llama2 repository: https://github.com/gjwgit/llama2

It needs a pip install ctransformers and then the script from Dr Dawei Chen will be a guide to writing the chat.py in the package.

$ python chat.py
loading model "TheBloke/Llama-2-7b-Chat-GGUF/llama-2-7b-chat.Q4_K_M.gguf" ...
config.json: 100%|█████████████████████████████| 29.0/29.0 [00:00<00:00, 103kB/s]
Fetching 1 files: 100%|████████████████████████████| 1/1 [00:00<00:00,  1.39it/s]
llama-2-7b-chat.Q4_K_M.gguf: 100%|██████████| 4.08G/4.08G [10:40<00:00, 6.37MB/s]
Fetching 1 files: 100%|███████████████████████████| 1/1 [10:41<00:00, 641.22s/it]

Answer the question below in text using about 50 words, your answer should be in bullet points.

Question:
Is nuclear power plant eco-friendly?

Answer:
• No, nuclear power plants produce radioactive waste that can harm the environment and human health.
• The process of generating electricity through nuclear reactions also releases greenhouse gases.
• Nuclear power plants require large amounts of water for cooling, which can impact local ecosystems.
• Accidents at nuclear power plants, such as Chernobyl and Fukushima, have had severe environmental and health consequences.

Answer the question below in text using about 50 words, your answer should be in bullet points.

Question:
How to stay safe during severe weather?

Answer:
• Stay informed through local news and weather alerts
• Create a family emergency plan
• Stock up on essential supplies
• Stay indoors and away from windows
• Avoid driving during severe weather
• Keep your phone charged and have a backup power source.

This will become:

$ ml chat llama2 Is nuclear power plant eco-friendly?

and 

$ ml chat llama2 How to stay safe during severe weather?
gjwgit commented 1 month ago

Check comments from Dawei https://github.com/ollama/ollama