taozhiyuai
commented
7 months ago ollama serve用的是QWEN 14B
Closed taozhiyuai closed 7 months ago
taozhiyuai
commented
7 months ago ollama serve用的是QWEN 14B
taozhiyuai
commented
7 months ago ollama serve的log
Last login: Tue Mar 26 12:08:06 on ttys000 taozhiyu@TAOZHIYUs-MBP ~ % ollama serve time=2024-03-26T12:12:37.435+08:00 level=INFO source=images.go:806 msg="total blobs: 5" time=2024-03-26T12:12:37.435+08:00 level=INFO source=images.go:813 msg="total unused blobs removed: 0" time=2024-03-26T12:12:37.436+08:00 level=INFO source=routes.go:1110 msg="Listening on 127.0.0.1:11434 (version 0.1.29)" time=2024-03-26T12:12:37.436+08:00 level=INFO source=payload_common.go:112 msg="Extracting dynamic libraries to /var/folders/rd/0_kt6h0545g1v749gjmryqqr0000gn/T/ollama197146676/runners ..." time=2024-03-26T12:12:37.462+08:00 level=INFO source=payload_common.go:139 msg="Dynamic LLM libraries [metal]" [GIN] 2024/03/26 - 12:20:09 | 404 | 192.917µs | 127.0.0.1 | POST "/v1/chat/completions"
kaina404
commented
7 months ago 你可以修改某一个bot的实现方式，例如：你选择了Gemini模型，然后 gemini/googel_gemini_bot.py 文件中更下为如下代码即可： import ollama
kaina404
commented
7 months ago
try:
if context.type != ContextType.TEXT:
logger.warn(f"[Ollama] Unsupported message type, type={context.type}")
return Reply(ReplyType.TEXT, None)
logger.info(f"[Ollama] query={query}")
session_id = context["session_id"]
session = self.sessions.session_query(query, session_id)
# 这里直接调用本地的Ollama服务
response = ollama.chat(
model= 'gemma:7b'
messages=self.filter_messages(session.messages))
reply_text = response['message']['content']
self.sessions.session_reply(reply_text, session_id)
logger.info(f"[Ollama] reply={reply_text}")
return Reply(ReplyType.TEXT, reply_text)
except Exception as e:
logger.error("[Ollama] fetch reply error, may contain unsafe content")
logger.error(e)
return Reply(ReplyType.ERROR, "invoke [Ollama] api failed!")/> ```python
try: if context.type != ContextType.TEXT: logger.warn(f"[Ollama] Unsupported message type, type={context.type}") return Reply(ReplyType.TEXT, None) logger.info(f"[Ollama] query={query}") session_id = context["session_id"] session = self.sessions.session_query(query, session_id) # 这里直接调用本地的Ollama服务 response = ollama.chat( model= 'gemma:7b' messages=self.filter_messages(session.messages)) reply_text = response['message']['content'] self.sessions.session_reply(reply_text, session_id) logger.info(f"[Ollama] reply={reply_text}") return Reply(ReplyType.TEXT, reply_text) except Exception as e: logger.error("[Ollama] fetch reply error, may contain unsafe content") logger.error(e) return Reply(ReplyType.ERROR, "invoke [Ollama] api failed!")
感谢帮助.更改如下
from bot.bot import Bot import ollama from bot.session_manager import SessionManager from bridge.context import ContextType, Context from bridge.reply import Reply, ReplyType from common.log import logger from config import conf from bot.baidu.baidu_wenxin_session import BaiduWenxinSession
class GoogleGeminiBot(Bot):
def __init__(self):
super().__init__()
self.api_key = conf().get("gemini_api_key")
# 复用文心的token计算方式
self.sessions = SessionManager(BaiduWenxinSession, model=conf().get("model") or "gpt-3.5-turbo")
def reply(self, query, context: Context = None) -> Reply:
try:
if context.type != ContextType.TEXT:
logger.warn(f"[Ollama] Unsupported message type, type={context.type}")
return Reply(ReplyType.TEXT, None)
logger.info(f"[Ollama] query={query}")
session_id = context["session_id"]
session = self.sessions.session_query(query, session_id)
# 这里直接调用本地的Ollama服务
response = ollama.chat(
model = 'gemma:7b',
messages=self.filter_messages(session.messages))
reply_text = response['message']['content']
self.sessions.session_reply(reply_text, session_id)
logger.info(f"[Ollama] reply={reply_text}")
return Reply(ReplyType.TEXT, reply_text)
except Exception as e:
logger.error("[Ollama] fetch reply error, may contain unsafe content")
logger.error(e)
return Reply(ReplyType.ERROR, "invoke [Ollama] api failed!")
def _convert_to_gemini_messages(self, messages: list):
res = []
for msg in messages:
if msg.get("role") == "user":
role = "user"
elif msg.get("role") == "assistant":
role = "model"
else:
continue
res.append({
"role": role,
"parts": [{"text": msg.get("content")}]
})
return res
@staticmethod
def filter_messages(messages: list):
res = []
turn = "user"
if not messages:
return res
for i in range(len(messages) - 1, -1, -1):
message = messages[i]
if message.get("role") != turn:
continue
res.insert(0, message)
if turn == "user":
turn = "assistant"
elif turn == "assistant":
turn = "user"
return res错误信息如下
config.json配置如下 "model": "gemini",
"gemini_api_key": "lm-studio",
"open_ai_api_key": "lm-studio",
"open_ai_api_base": "http://localhost:1234/v1",
@kaina404
taozhiyuai
commented
7 months ago 用lm studio做SERVER就没那么复杂.改api 和base就可以了.
kaina404
commented
7 months ago 我给出了我本地的修改方案，以支持Ollama服务。https://github.com/zhayujie/chatgpt-on-wechat/issues/1845 @taozhiyuai
taozhiyuai
commented
7 months ago [WARNING][2024-03-29 07:08:37][plugin_manager.py:150] - Failed to init SUMMARY, diabled. [Summary] init failed, not supported bot type
@kaina404
这个插件不支持bot type.麻烦处理下.谢谢
前置确认
⚠️ 搜索issues中是否已存在类似问题
操作系统类型?
MacOS
运行的python版本是?
other
使用的chatgpt-on-wechat版本是?
Latest Release
运行的
channel类型是?wx(个人微信, itchat)
复现步骤 🕹
python=3.11
config.json中 "model": "get-3.5-turbo", "open_ai_api_key": "ollama", "open_ai_api_base": "http://127.0.0.1:11434/v1",
正常启动
问题描述 😯
终端日志 📒