파이썬 로컬 LLM 시작 하기

eunja511005 commented 2 months ago

https://github.com/ollama/ollama/tree/main/examples

https://github.com/ollama/ollama/tree/main/examples/langchain-python-rag-websummary

올라마 이용해서 gemma 실행된 상태에서만 파이썬 프로그램 이용해서 gemma 로컬 LLM 이용 가능
파워쉘 관라지 권한으로 실행
cd C:\Users\y##es\AppData\Local\Programs\Ollama
dir
& '.\ollama app.exe' (olla 까지만 입력하고 tab 버튼 클릭하면 자동 완성 됨)
ollama run llama2
ollama run gemma (한국어 지원 잘 함)
종료는 노트북 우측 하단에 더 보기 화살표 클릭 > 라마 아이콘 우클릭 > Quit Ollama 클릭
```
![image](https://github.com/eunja511005/AutoCoding/assets/118089135/c3d4188d-d2a1-44c4-a31d-7a855be39228)
```
작업 공간 :
```
D:/y#####/AI
```

가상 환경 세팅 :

터미널 실행 : Ctrl + `
python -m venv aienv

가상 환경 실행 :

[중요] 파워쉘에서는 가상환경 실행시 오류 나서 command prompt 열어서 실행 
cd aienv/Scripts
actvate

랭체인 인스톨
```
pip install langchain
```

테스트 파이썬 파일 생성


from langchain_community.llms import Ollama

if name == "main": llm_model = Ollama(model="gemma")

messages=[
        {"role": "system", "content": "너는 자동차영업사원이야"},
        {"role": "user", "content": "차에 대해서 설명해주세요"}
    ]
print(llm_model.invoke(input=messages))

6. 결론 :

답변이 시간이 오래 걸리고 작업 관리자를 보면 ollama.exe가 메모리를 엄청 사용하지만 잘 수행 됨 올라마와 랭체인 이용해서 성공적으로 프롬프트 수행 완료

eunja511005 commented 1 month ago

랭서 이용하여 간단한 챗봇 서비스 하기

from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
from fastapi.responses import RedirectResponse
from langserve import add_routes
from fastapi.staticfiles import StaticFiles
import sys
import os

# 프로젝트 루트 디렉토리를 sys.path에 추가합니다.
sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), '..', '..')))
from llm_module import LLMService

app = FastAPI()
llm_service = LLMService()

class UserInput(BaseModel):
    text: str

@app.post("/chat")
async def chat_with_user(input: UserInput):
    user_input = input.text
    if not user_input:
        raise HTTPException(status_code=400, detail="Input text is empty.")
    response = llm_service.chat(user_input)
    return {"response": response}

# 정적 파일 제공을 위한 설정
app.mount("/static", StaticFiles(directory=os.path.join(os.path.dirname(__file__), 'static')), name="static")

@app.get("/")
async def read_root():
    return RedirectResponse(url="/static/index.html")

# FastAPI 애플리케이션 인스턴스를 export
api = app

if __name__ == "__main__":
    import uvicorn
    # serve(FastAPIServer(app), host="0.0.0.0", port=8001)
    uvicorn.run(app, host="0.0.0.0", port=8000)

from langchain_community.llms import Ollama

class LLMService:
    def __init__(self):
        self.llm = Ollama(model="gemma:2b")

    def chat(self, user_input: str) -> str:
        if not user_input:
            return "Input text is empty."
        return self.llm.invoke(user_input)

eunja511005 commented 1 month ago

server.py

from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
from fastapi.responses import RedirectResponse
from langserve import add_routes
from fastapi.staticfiles import StaticFiles
import sys
import os
from fastapi.middleware.cors import CORSMiddleware

# 프로젝트 루트 디렉토리를 sys.path에 추가합니다.
sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), '..', '..')))
from llm_module import chain

app = FastAPI()

# Set all CORS enabled origins
app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
    expose_headers=["*"],
)

@app.get("/")
async def redirect_root_to_docs():
    return RedirectResponse("/chat/playground")

# Edit this to add the chain you want to add
add_routes(app, chain, path="/chat")

if __name__ == "__main__":
    import uvicorn
    # serve(FastAPIServer(app), host="0.0.0.0", port=8001)
    uvicorn.run(app, host="0.0.0.0", port=8000)

llm_module.py

from langchain_community.llms import Ollama
from langchain_core.prompts import PromptTemplate
from langchain_core.output_parsers import StrOutputParser

template = """You are helpful assistant. 
Please answer user's question.
Question: {input}
Answer: """
prompt = PromptTemplate.from_template(template)

llm = Ollama(model="gemma:2b")

chain = prompt | llm | StrOutputParser()

if __name__ == "__main__":
    res = chain.invoke({"input":"What is the capital city of South Korea?"})
    print(res)

ywbestPark commented 1 month ago

index.html

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Chat Service</title>
    <!-- Bootstrap CSS -->
    <link href="https://stackpath.bootstrapcdn.com/bootstrap/4.5.2/css/bootstrap.min.css" rel="stylesheet">
    <style>
        body {
            background-color: #f4f4f4;
            display: flex;
            justify-content: center;
            align-items: center;
            height: 100vh;
            margin: 0;
        }
        #chat-container {
            width: 100%;
            max-width: 400px;
            box-shadow: 0 0 10px rgba(0, 0, 0, 0.1);
            border-radius: 8px;
            background-color: #fff;
            overflow: hidden;
            display: flex;
            flex-direction: column;
            height: 80vh;
        }
        #chat-log {
            flex: 1;
            overflow-y: auto;
            padding: 10px;
            border-bottom: 1px solid #ccc;
        }
        #chat-log .message {
            margin-bottom: 10px;
            padding: 10px;
            border-radius: 4px;
        }
        #chat-log .message.user {
            background-color: #d1e7dd;
            text-align: right;
        }
        #chat-log .message.bot {
            background-color: #fff3cd;
        }
        #chat-form {
            display: flex;
            padding: 10px;
        }
        #chat-form input {
            flex: 1;
            margin-right: 10px;
        }
    </style>
</head>
<body>
    <div id="chat-container">
        <div class="bg-primary text-white text-center py-2">
            <h1 class="h4 m-0">Chat Service</h1>
        </div>
        <div id="chat-log" class="p-3"></div>
        <form id="chat-form" class="form-inline">
            <input type="text" id="text" name="text" class="form-control mr-2" required placeholder="Enter your message...">
            <button type="submit" class="btn btn-primary">Send</button>
        </form>
    </div>

    <!-- Bootstrap JS and dependencies -->
    <script src="https://code.jquery.com/jquery-3.5.1.slim.min.js"></script>
    <script src="https://cdn.jsdelivr.net/npm/@popperjs/core@2.5.2/dist/umd/popper.min.js"></script>
    <script src="https://stackpath.bootstrapcdn.com/bootstrap/4.5.2/js/bootstrap.min.js"></script>

    <script>
        document.getElementById('chat-form').addEventListener('submit', async function(event) {
            event.preventDefault();
            const textInput = document.getElementById('text');
            const text = textInput.value;
            const chatLog = document.getElementById('chat-log');
            const responseDiv = document.createElement('div');
            responseDiv.className = 'message user';
            responseDiv.innerText = text;
            chatLog.appendChild(responseDiv);
            textInput.value = '';
            textInput.disabled = true;
            const button = document.querySelector('button');
            button.disabled = true;

            try {
                const response = await fetch('/chat', {
                    method: 'POST',
                    headers: {
                        'Content-Type': 'application/json',
                    },
                    body: JSON.stringify({ text: text }),
                });
                const result = await response.json();
                const botResponseDiv = document.createElement('div');
                botResponseDiv.className = 'message bot';
                botResponseDiv.innerText = result.response;
                chatLog.appendChild(botResponseDiv);
                chatLog.scrollTop = chatLog.scrollHeight;
            } catch (error) {
                const errorDiv = document.createElement('div');
                errorDiv.className = 'message bot';
                errorDiv.innerText = 'Error: ' + error.message;
                chatLog.appendChild(errorDiv);
            } finally {
                textInput.disabled = false;
                button.disabled = false;
                textInput.focus();
            }
        });
    </script>
</body>
</html>

eunja511005 / AutoCoding

파이썬 로컬 LLM 시작 하기 #186

랭서 이용하여 간단한 챗봇 서비스 하기

server.py

llm_module.py

index.html