search

lobehub / lobe-chat

🤯 Lobe Chat - an open-source, modern-design AI chat framework. Supports Multi AI Providers( OpenAI / Claude 3 / Gemini / Ollama / Azure / DeepSeek), Knowledge Base (file upload / knowledge management / RAG ), Multi-Modals (Vision/TTS) and plugin system. One-click FREE deployment of your private ChatGPT/ Claude application.
https://chat-preview.lobehub.com
Other
44.36k stars 9.94k forks source link

Server DB Docker Image Feedback | 服务端 Database Docker 镜像问题反馈 #3391

Closed arvinxx closed 2 months ago

arvinxx commented 3 months ago

首先非常感谢 @cy948 的努力，完成了 Next Auth 和 LobeChat 现有 Database 系统的集成，使得我们可以构建起 LobeChat 的服务端数据库的 Docker 镜像。

服务端数据库的 docker 镜像已正式可用，镜像名为 lobehub/lobe-chat-database。

关于服务端数据库的部署文档，详见： https://lobehub.com/docs/self-hosting/server-database

在此也列出一个简版的部署指南，供大家参考。如有其他问题，欢迎在下方留言。

  1. 创建 pg 实例，需要包含 pgvector 插件（注意：以下命令仅用于演示，因为这个 pg 实例并没有包含持久化部分，请自行构建一个符合你诉求的 生产级 pg 实例）
docker run --name my-postgres --network pg -e POSTGRES_PASSWORD=mysecretpassword -p 5432:5432 -d pgvector/pgvector:pg16
  1. 创建一个 lobe-chat.env 文件用于存放环境变量：
#网站域名
APP_URL=http://localhost:3210

# DB 必须
KEY_VAULTS_SECRET=jgwsK28dspyVQoIf8/M3IIHl1h6LYYceSYNXeLpy6uk=
DATABASE_URL=postgres://postgres:mysecretpassword@my-postgres:5432/postgres

# NEXT_AUTH 相关，可以使用 auth0 等，如有其他接入诉求欢迎提 PR
NEXT_AUTH_SECRET=3904039cd41ea1bdf6c93db0db96e250
NEXT_AUTH_SSO_PROVIDERS=auth0
NEXTAUTH_URL=https://your-prod-domain.com/api/auth
AUTH0_CLIENT_ID=xxxxxx
AUTH0_CLIENT_SECRET=cSX_xxxxx
AUTH0_ISSUER=https://lobe-chat-demo.us.auth0.com

# S3 相关
S3_ACCESS_KEY_ID=xxxxxxxxxx
S3_SECRET_ACCESS_KEY=xxxxxxxxxx
S3_ENDPOINT=https://xxxxxxxxxx.r2.cloudflarestorage.com
S3_BUCKET=lobechat
NEXT_PUBLIC_S3_DOMAIN=https://s3-for-lobechat.your-domain.com
  1. 启动 lobe-chat-database docker 镜像
docker run -it -d -p 3210:3210 --network pg --env-file lobe-chat.env --name lobe-chat-database lobehub/lobe-chat-database

注意事项：确保数据库和 LobeChat DB 镜像在同一网络中

lobehubbot commented 3 months ago

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

Finally, I changed to authing and logged in successfully. However, when importing the previous global settings and all assistant data, it prompts: Import failed Error reason: Upload Error

bentwnghk commented 3 months ago

成功部署，但无法点击登陆注册账户后出现错误。部署命令如下（用xxxx模糊了敏感信息）： docker run -it -d --name lobe-chat-database --restart always -p 3210:3210 -e DATABASE_URL=postgresql://root:xxxxx@192.168.xx.xx:5432/mydb -e KEY_VAULTS_SECRET=xxxx -e NEXT_AUTH_SECRET=xxxxx -e NEXT_AUTH_SSO_PROVIDERS=auth0 -e AUTH0_CLIENT_ID=xxxxxxxxxxWPWAS0GpxxxxxxxxxxGxxx -e AUTH0_CLIENT_SECRET=xxxxxxxxxBUlFzVMxxxxxxxxxxxxxxxxxxxxxxxTkU5Dc6xxxxxxx- -e AUTH0_ISSUER=https://dev-oldbiuln0bipyiqh.us.auth0.com -e NEXTAUTH_URL=http://192.168.31.33:3210/api/auth/callback/auth0 -e S3_ACCESS_KEY_ID=xxxxxx -e S3_SECRET_ACCESS_KEY=xxxxx -e S3_ENDPOINT=https://xxxxx.r2.cloudflarestorage.com -e S3_BUCKET=lobechat -e NEXT_PUBLIC_S3_DOMAIN=https://lober2.xxxxx.top lobehub/lobe-chat-database:latest 点击登陆注册后，反馈（因为这会儿不方便，没法截图）： 页面遇到一点问题，待会儿来试试，或回到已知的世界

NEXTAUTH_URL 配置为 http://192.168.31.33:3210/api/auth 试试

不再提示页面遇到问题。但无论是否挂梯子，都还是不行，进入一个新的页面，提示： dev-oldbiuln0bipyiqh Oops!, something went wrong Callback URL mismatch. The provided redirect_uri is not in the list of allowed callback URLs. Please go to the Application Settings page and make sure you are sending a valid callback url from your application.

image 看看AUTH0里面的设置？

没有问题，我看了AUTH0和NEXTAUTH_URL如果都是http://192.168.31.33:3210/api/auth/callback/auth0，就提示页面遇到一点问题，待会儿来试试，或回到已知的世界，如果AUTH0和NEXTAUTH_URL是http://192.168.31.33:3210/api/auth，就转到新页面提示dev-oldbiuln0bipyiqh Oops!, something went wrong Callback URL mismatch. The provided redirect_uri is not in the list of allowed callback URLs. Please go to the Application Settings page and make sure you are sending a valid callback url from your application.

NEXTAUTH_URL 填寫 'http://192.168.31.33:3210/api/auth' Auth0 裏頭 Allowed Callback URLs 填寫 'https://\<your-hosted-domain>/api/auth/callback/auth0'

darkelfzdy commented 3 months ago

成功部署，但无法点击登陆注册账户后出现错误。部署命令如下（用xxxx模糊了敏感信息）： docker run -it -d --name lobe-chat-database --restart always -p 3210:3210 -e DATABASE_URL=postgresql://root:xxxxx@192.168.xx.xx:5432/mydb -e KEY_VAULTS_SECRET=xxxx -e NEXT_AUTH_SECRET=xxxxx -e NEXT_AUTH_SSO_PROVIDERS=auth0 -e AUTH0_CLIENT_ID=xxxxxxxxxxWPWAS0GpxxxxxxxxxxGxxx -e AUTH0_CLIENT_SECRET=xxxxxxxxxBUlFzVMxxxxxxxxxxxxxxxxxxxxxxxTkU5Dc6xxxxxxx- -e AUTH0_ISSUER=https://dev-oldbiuln0bipyiqh.us.auth0.com -e NEXTAUTH_URL=http://192.168.31.33:3210/api/auth/callback/auth0 -e S3_ACCESS_KEY_ID=xxxxxx -e S3_SECRET_ACCESS_KEY=xxxxx -e S3_ENDPOINT=https://xxxxx.r2.cloudflarestorage.com -e S3_BUCKET=lobechat -e NEXT_PUBLIC_S3_DOMAIN=https://lober2.xxxxx.top lobehub/lobe-chat-database:latest 点击登陆注册后，反馈（因为这会儿不方便，没法截图）： 页面遇到一点问题，待会儿来试试，或回到已知的世界

NEXTAUTH_URL 配置为 http://192.168.31.33:3210/api/auth 试试

不再提示页面遇到问题。但无论是否挂梯子，都还是不行，进入一个新的页面，提示： dev-oldbiuln0bipyiqh Oops!, something went wrong Callback URL mismatch. The provided redirect_uri is not in the list of allowed callback URLs. Please go to the Application Settings page and make sure you are sending a valid callback url from your application.

image 看看AUTH0里面的设置？

没有问题，我看了AUTH0和NEXTAUTH_URL如果都是http://192.168.31.33:3210/api/auth/callback/auth0，就提示页面遇到一点问题，待会儿来试试，或回到已知的世界，如果AUTH0和NEXTAUTH_URL是http://192.168.31.33:3210/api/auth，就转到新页面提示dev-oldbiuln0bipyiqh Oops!, something went wrong Callback URL mismatch. The provided redirect_uri is not in the list of allowed callback URLs. Please go to the Application Settings page and make sure you are sending a valid callback url from your application.

NEXTAUTH_URL 填寫 'http://192.168.31.33:3210/api/auth' Auth0 裏頭 Allowed Callback URLs 填寫 'https:///api/auth/callback/auth0'

对，开始就是这里搞错了，以为两个地方要填一样，按这个就好了。现在我用的authing，但是导入的时候提示 Upload Error

bentwnghk commented 3 months ago

成功部署，但无法点击登陆注册账户后出现错误。部署命令如下（用xxxx模糊了敏感信息）： docker run -it -d --name lobe-chat-database --restart always -p 3210:3210 -e DATABASE_URL=postgresql://root:xxxxx@192.168.xx.xx:5432/mydb -e KEY_VAULTS_SECRET=xxxx -e NEXT_AUTH_SECRET=xxxxx -e NEXT_AUTH_SSO_PROVIDERS=auth0 -e AUTH0_CLIENT_ID=xxxxxxxxxxWPWAS0GpxxxxxxxxxxGxxx -e AUTH0_CLIENT_SECRET=xxxxxxxxxBUlFzVMxxxxxxxxxxxxxxxxxxxxxxxTkU5Dc6xxxxxxx- -e AUTH0_ISSUER=https://dev-oldbiuln0bipyiqh.us.auth0.com -e NEXTAUTH_URL=http://192.168.31.33:3210/api/auth/callback/auth0 -e S3_ACCESS_KEY_ID=xxxxxx -e S3_SECRET_ACCESS_KEY=xxxxx -e S3_ENDPOINT=https://xxxxx.r2.cloudflarestorage.com -e S3_BUCKET=lobechat -e NEXT_PUBLIC_S3_DOMAIN=https://lober2.xxxxx.top lobehub/lobe-chat-database:latest 点击登陆注册后，反馈（因为这会儿不方便，没法截图）： 页面遇到一点问题，待会儿来试试，或回到已知的世界

NEXTAUTH_URL 配置为 http://192.168.31.33:3210/api/auth 试试

不再提示页面遇到问题。但无论是否挂梯子，都还是不行，进入一个新的页面，提示： dev-oldbiuln0bipyiqh Oops!, something went wrong Callback URL mismatch. The provided redirect_uri is not in the list of allowed callback URLs. Please go to the Application Settings page and make sure you are sending a valid callback url from your application.

image 看看AUTH0里面的设置？

没有问题，我看了AUTH0和NEXTAUTH_URL如果都是http://192.168.31.33:3210/api/auth/callback/auth0，就提示页面遇到一点问题，待会儿来试试，或回到已知的世界，如果AUTH0和NEXTAUTH_URL是http://192.168.31.33:3210/api/auth，就转到新页面提示dev-oldbiuln0bipyiqh Oops!, something went wrong Callback URL mismatch. The provided redirect_uri is not in the list of allowed callback URLs. Please go to the Application Settings page and make sure you are sending a valid callback url from your application.

NEXTAUTH_URL 填寫 'http://192.168.31.33:3210/api/auth' Auth0 裏頭 Allowed Callback URLs 填寫 'https:///api/auth/callback/auth0'

对，开始就是这里搞错了，以为两个地方要填一样，按这个就好了。现在我用的authing，但是导入的时候提示 Upload Error

您是否嘗試過是否可以上傳照片進行聊天？

nvveex commented 3 months ago

`# DB 必须 KEY_VAULTS_SECRET=6R7JiMQwmxxxxxxcW5/36Z1GprCiQLeAGBxai1iJQ0= DATABASE_URL=postgresql://postgres:mysecretpassword@my-postgres:5432/postgres

NEXT_AUTH 相关

NEXT_AUTH_SECRET=gt5rh0ILxxxxxxxxxxxxxprWoHwTl4d4MsdDBw2U= NEXT_AUTH_SSO_PROVIDERS=auth0 NEXTAUTH_URL=https://ai.vcck.cn/api/auth AUTH0_CLIENT_ID=66b09xxxxxxxxxx6ef2fb1cede9 AUTH0_CLIENT_SECRET=dbe2b9a41xxxxxxxxxxd78aa0001a43ff AUTH0_ISSUER=https://ktzpzryvogzb-demo.authing.cn/oidc

S3 相关

S3_ACCESS_KEY_ID=XY71C7FPxxxxxxxxI3STOU S3_SECRET_ACCESS_KEY=9mAlxlzXQhxxxxxxxxxHdlQ0eMzmIST07fDyLr S3_ENDPOINT=https://obs.cn-north-4.myhuaweicloud.com S3_BUCKET=lobechat NEXT_PUBLIC_S3_DOMAIN=https://lobechat.obs.cn-north-4.myhuaweicloud.com` image 这是我的env，现在点击登录就会错误，可以帮我看一下哪里出问题了吗， 这是我的链接ai.vcck.cn ![Uploading aac50fa546a226f392e8f3ee20a358f.png…]()

lobehubbot commented 3 months ago

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

`# DB required KEY_VAULTS_SECRET=6R7JiMQwmxxxxxxcW5/36Z1GprCiQLeAGBxai1iJQ0= DATABASE_URL=postgresql://postgres:mysecretpassword@my-postgres:5432/postgres

NEXT_AUTH related

NEXT_AUTH_SECRET=gt5rh0ILxxxxxxxxxxxxxprWoHwTl4d4MsdDBw2U= NEXT_AUTH_SSO_PROVIDERS=auth0 NEXTAUTH_URL=https://ai.vcck.cn/api/auth AUTH0_CLIENT_ID=66b09xxxxxxxxxx6ef2fb1cede9 AUTH0_CLIENT_SECRET=dbe2b9a41xxxxxxxxxxd78aa0001a43ff AUTH0_ISSUER=https://ktzpzryvogzb-demo.authing.cn/oidc

S3 related

S3_ACCESS_KEY_ID=XY71C7FPxxxxxxxxI3STOU S3_SECRET_ACCESS_KEY=9mAlxlzXQhxxxxxxxxxHdlQ0eMzmIST07fDyLr S3_ENDPOINT=https://obs.cn-north-4.myhuaweicloud.com S3_BUCKET=lobechat NEXT_PUBLIC_S3_DOMAIN=https://lobechat.obs.cn-north-4.myhuaweicloud.com` image This is my env. Now when I click to log in, there will be an error. Can you help me see where the problem is? This is my link ai.vcck.cn ![Uploading aac50fa546a226f392e8f3ee20a358f.png…]()

darkelfzdy commented 3 months ago

成功部署，但无法点击登陆注册账户后出现错误。部署命令如下（用xxxx模糊了敏感信息）： docker run -it -d --name lobe-chat-database --restart always -p 3210:3210 -e DATABASE_URL=postgresql://root:xxxxx@192.168.xx.xx:5432/mydb -e KEY_VAULTS_SECRET=xxxx -e NEXT_AUTH_SECRET=xxxxx -e NEXT_AUTH_SSO_PROVIDERS=auth0 -e AUTH0_CLIENT_ID=xxxxxxxxxxWPWAS0GpxxxxxxxxxxGxxx -e AUTH0_CLIENT_SECRET=xxxxxxxxxBUlFzVMxxxxxxxxxxxxxxxxxxxxxxxTkU5Dc6xxxxxxx- -e AUTH0_ISSUER=https://dev-oldbiuln0bipyiqh.us.auth0.com -e NEXTAUTH_URL=http://192.168.31.33:3210/api/auth/callback/auth0 -e S3_ACCESS_KEY_ID=xxxxxx -e S3_SECRET_ACCESS_KEY=xxxxx -e S3_ENDPOINT=https://xxxxx.r2.cloudflarestorage.com -e S3_BUCKET=lobechat -e NEXT_PUBLIC_S3_DOMAIN=https://lober2.xxxxx.top lobehub/lobe-chat-database:latest 点击登陆注册后，反馈（因为这会儿不方便，没法截图）： 页面遇到一点问题，待会儿来试试，或回到已知的世界

NEXTAUTH_URL 配置为 http://192.168.31.33:3210/api/auth 试试

不再提示页面遇到问题。但无论是否挂梯子，都还是不行，进入一个新的页面，提示： dev-oldbiuln0bipyiqh Oops!, something went wrong Callback URL mismatch. The provided redirect_uri is not in the list of allowed callback URLs. Please go to the Application Settings page and make sure you are sending a valid callback url from your application.

image 看看AUTH0里面的设置？

没有问题，我看了AUTH0和NEXTAUTH_URL如果都是http://192.168.31.33:3210/api/auth/callback/auth0，就提示页面遇到一点问题，待会儿来试试，或回到已知的世界，如果AUTH0和NEXTAUTH_URL是http://192.168.31.33:3210/api/auth，就转到新页面提示dev-oldbiuln0bipyiqh Oops!, something went wrong Callback URL mismatch. The provided redirect_uri is not in the list of allowed callback URLs. Please go to the Application Settings page and make sure you are sending a valid callback url from your application.

NEXTAUTH_URL 填寫 'http://192.168.31.33:3210/api/auth' Auth0 裏頭 Allowed Callback URLs 填寫 'https:///api/auth/callback/auth0'

对，开始就是这里搞错了，以为两个地方要填一样，按这个就好了。现在我用的authing，但是导入的时候提示 Upload Error

您是否嘗試過是否可以上傳照片進行聊天？

搞清楚问题了，是Cloudflare的跨域引起的，因为我在NEXTAUTH_URL和Auth0使用了IP而不是域名，登录后会转到ip访问而不是域名访问，但在Cloudflare跨域设置的时候我使用的是域名，造成了没法用R2，所以上传始终失败。现在全部调整为域名就好了

darkelfzdy commented 3 months ago

数据库版本只有导入没有导出吗？对数据库不熟悉，完全不能导出感觉对数据好没有安全感

lobehubbot commented 3 months ago

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

Is the database version only importing but not exporting? I am not familiar with the database and cannot export it at all. I feel insecure about the data.

bentwnghk commented 3 months ago

数据库版本只有导入没有导出吗？对数据库不熟悉，完全不能导出感觉对数据好没有安全感

是的，官方版本lobechat cloud也沒有導出功能。

lobehubbot commented 3 months ago

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

Is the database version only imported but not exported? I am not familiar with the database and cannot export it at all. I feel insecure about the data.

Yes, the official version of lobechat cloud does not have an export function.

darkelfzdy commented 3 months ago

数据库版本只有导入没有导出吗？对数据库不熟悉，完全不能导出感觉对数据好没有安全感

是的，官方版本lobechat cloud也沒有導出功能。

要是我的数据库崩了，咋办？

lobehubbot commented 3 months ago

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

Is the database version only imported but not exported? I am not familiar with the database and cannot export it at all. I feel insecure about the data.

Yes, the official version of lobechat cloud does not have an export function.

What if my database crashes?

bentwnghk commented 3 months ago

数据库版本只有导入没有导出吗？对数据库不熟悉，完全不能导出感觉对数据好没有安全感

是的，官方版本lobechat cloud也沒有導出功能。

要是我的数据库崩了，咋办？

你是用本地node postgres實例？可以嘗試backup整個postgres數據庫。

lobehubbot commented 3 months ago

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

Is the database version only imported but not exported? I am not familiar with the database and cannot export it at all. I feel insecure about the data.

Yes, the official version of lobechat cloud does not have an export function.

What if my database crashes?

Are you using a local node postgres instance? You can try to backup the entire postgres database.

maxage commented 3 months ago

要是能用又拍云的 云储存就好了，免费空间也大。 数据库：PostgreSQL https://cloud.memfiredb.com 1G容量 100W读 10W写/月

lobehubbot commented 3 months ago

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

It would be great if you could use Fupaiyun's cloud storage, and the free space would be large. Database: PostgreSQL https://cloud.memfiredb.com 1G capacity 100W read 10W write/month

arvinxx commented 3 months ago

@darkelfzdy 目前不支持的原因是如果服务端数据库版本支持了导出，那客户端 db 导入的话会造成数据格式的混乱。

而要做好客户端 db 的导入服务端 db 数据的兼容不是一件容易的事情。估计要等 Q4 完成整个 客户端 db 和 服务端 db 统一后才能支持服务端数据导出。

lobehubbot commented 3 months ago

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

@darkelfzdy The reason why it is not currently supported is that if the server database version supports export, then importing the client db will cause confusion in the data format. It is not an easy task to ensure import compatibility of client db. It is estimated that we will have to wait until Q4 to complete the unification of the entire client db and server db before we can support server-side data export.

dalefengs commented 3 months ago

ACCESS_CODE=xxx，失效，用户只要登录Auth0，不用填写密码就能请求所有模型接口

lobehubbot commented 3 months ago

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

ACCESS_CODE=xxx, invalid. As long as the user logs in to Auth0, he can request all model interfaces without filling in a password.

Sun-drenched commented 3 months ago

ACCESS_CODE=xxx，失效，用户只要登录Auth0，不用填写密码就能请求所有模型接口

Auth0可以部署阻止策略，控制注册和登录。同时服务器DB模式下可以随账号同步模型KEY/接口，以ACCESS_CODE控制访问+以环境变量形式预配KEY+接口的模式应该是被淘汰了。

lobehubbot commented 3 months ago

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

ACCESS_CODE=xxx, invalid. As long as the user logs in to Auth0, he can request all model interfaces without filling in a password.

Auth0 can deploy blocking policies to control registration and login. At the same time, in server DB mode, model KEY/interface can be synchronized with the account. The mode of controlling access with ACCESS_CODE + preconfiguring KEY + interface in the form of environment variables should be eliminated.

dalefengs commented 3 months ago

ACCESS_CODE=xxx，失效，用户只要登录Auth0，不用填写密码就能请求所有模型接口

Auth0可以部署阻止策略，控制注册和登录。同时服务器DB模式下可以随账号同步模型KEY/接口，以ACCESS_CODE控制访问+以环境变量形式预配KEY+接口的模式应该是被淘汰了。

好的，我看这里是必填 ACCESS_CODE 以为可以进行访问控制

image

lobehubbot commented 3 months ago

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

ACCESS_CODE=xxx, invalid. As long as the user logs in to Auth0, he can request all model interfaces without filling in a password.

Auth0 can deploy blocking policies to control registration and login. At the same time, in server DB mode, model KEY/interface can be synchronized with the account. The mode of controlling access with ACCESS_CODE + preconfiguring KEY + interface in the form of environment variables should be eliminated.

Okay, I see that ACCESS_CODE is required here and I think it can be used for access control.

image

dalefengs commented 3 months ago

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

ACCESS_CODE=xxx, invalid. As long as the user logs in to Auth0, he can request all model interfaces without filling in a password.

Auth0 can deploy blocking policies to control registration and login. At the same time, in server DB mode, model KEY/interface can be synchronized with the account. The mode of controlling access with ACCESS_CODE + preconfiguring KEY + interface in the form of environment variables should be eliminated.

Okay, I see that ACCESS_CODE is required here and I think it can be used for access control.

image

设置 ACCESS_CODE 如果不填写，不能上传图片？但能聊天是否合理？ image

dalefengs commented 3 months ago 
- S3_ACCESS_KEY_ID=xxxx
- S3_SECRET_ACCESS_KEY=xxx

- S3_BUCKET=lobechat
- S3_ENDPOINT=https://minio.xxx.do:19000
- NEXT_PUBLIC_S3_DOMAIN=https://minio.xxx.do
API：https://cloud.kl.do/trpc/edge/upload.createS3PreSignedUrl?batch=1
响应
[
    {
        "result": {
            "data": {
                "json": "https://picture.minio.xxx.do:19000/files/478613/3852d162-73eb-4fa6-a95b-1903790b3721.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=ZCAOhiajoxzBXIs7z4MK%2F20240807%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240807T051634Z&X-Amz-Expires=3600&X-Amz-Signature=658bb0a38c314628283ee5a65e7812e422ac53c0450925d1196efe8d0e1d9778&X-Amz-SignedHeaders=host&x-amz-acl=public-read&x-id=PutObject"
            }
        }
    }

请问下 Minio 支持S3 对象存储，但为什么 S3_BUCKET 会拼接在域名前方？不应该是这样的么？ https://minio.xxx.do:19000/picture/files/478613/3852d162-73eb-4fa6-a95b-1903790b3721.png

lobehubbot commented 3 months ago

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿 


- S3_ACCESS_KEY_ID=xxxx
- S3_SECRET_ACCESS_KEY=xxx
API: https://cloud.kl.do/trpc/edge/upload.createS3PreSignedUrl?batch=1
response
[
    {
        "result": {
            "data": {
                "json": "https://picture.minio.xxx.do:19000/files/478613/3852d162-73eb-4fa6-a95b-1903790b3721.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content -Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=ZCAOhiajoxzBXIs7z4MK%2F20240807%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240807T051634Z&X-Amz-Expires=3600&X-Amz-Signature =658bb0a38c314628283ee5a65e7812e422ac53c0450925d1196efe8d0e1d9778&X-Amz-SignedHeaders=host&x-amz -acl=public-read&x-id=PutObject"
            }
        }
    }

Minio supports S3 object storage, but why is S3_BUCKET spliced ​​in front of the domain name? Shouldn't it be like this? https://minio.xxx.do:19000/picture/files/478613/3852d162-73eb-4fa6-a95b-1903790b3721.png

arvinxx commented 3 months ago

以ACCESS_CODE控制访问+以环境变量形式预配KEY+接口的模式应该是被淘汰了。

@Sun-drenched 并不是淘汰，而是在 server db 模式下不适用。 ACCESS_CODE + 预配 key 在 client db 下还是适用的

lobehubbot commented 3 months ago

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

The mode of controlling access with ACCESS_CODE + provisioning KEY + interface in the form of environment variables should be eliminated.

@Sun-drenched is not eliminated, but is not applicable in server db mode. ACCESS_CODE + provisioning key is still applicable under client db

jedy commented 3 months ago

@dalefengs 用的aws的s3 sdk，默认是用域名的方式加bucket。你可以在启动minio时加环境变量MINIO_DOMAIN让minio也支持域名的方式。

lobehubbot commented 3 months ago

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

@dalefengs uses the s3 sdk of AWS. By default, it uses the domain name to add the bucket. You can add the environment variable MINIO_DOMAIN when starting minio so that minio also supports domain names.

mujiannan commented 3 months ago

@dalefengs 用的aws的s3 sdk，默认是用域名的方式加bucket。你可以在启动minio时加环境变量MINIO_DOMAIN让minio也支持域名的方式。

这很离谱，刚开始看见这个版本的S3配置项时就觉得非常奇怪，现在看来确实蛮奇怪的。直接让用户自定义url就好了。

lobehubbot commented 3 months ago

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

@dalefengs uses the s3 sdk of AWS, which uses the domain name to add buckets by default. You can add the environment variable MINIO_DOMAIN when starting minio so that minio also supports domain names.

This is outrageous. When I first saw this version of S3 configuration items, I thought it was very strange. Now it seems really strange. Just let the user customize the URL.

dalefengs commented 3 months ago

@dalefengs 用的aws的s3 sdk，默认是用域名的方式加bucket。你可以在启动minio时加环境变量MINIO_DOMAIN让minio也支持域名的方式。

能正常上传文件到 s3， 但url附带了一些参数无法读取

lobehubbot commented 3 months ago

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

@dalefengs uses the s3 sdk of AWS, which uses the domain name to add buckets by default. You can add the environment variable MINIO_DOMAIN when starting minio so that minio also supports domain names.

Can upload files to s3 normally but cannot read them

jueinin commented 3 months ago

发现用auth0登录后，退出再重新登录，直接用登录上了，压根都不给换账号的机会啊 是不是忘记调auth0的退出登录接口了

lobehubbot commented 3 months ago

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

I found that after logging in with auth0, I logged out and logged in again. I logged in directly without giving me the chance to change my account. Did I forget to adjust the logout interface of auth0?

arvinxx commented 3 months ago

发现用auth0登录后，退出再重新登录，直接用登录上了，压根都不给换账号的机会啊 是不是忘记调auth0的退出登录接口了

@cy948 这个来看看？

lobehubbot commented 3 months ago

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

I found that after logging in with auth0, I logged out and logged in again. I logged in directly without giving me the chance to change my account. Did I forget to adjust the logout interface of auth0?

@cy948 Let’s take a look at this?

coulsontl commented 3 months ago

@arvinxx 不会自动创建表，登录完成之后就报错了，pg也是用docker部署的

[auth][cause]: error: relation "nextauth_accounts" does not exist
    at /app/node_modules/.pnpm/pg-pool@3.6.2_pg@8.12.0/node_modules/pg-pool/index.js:45:11
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async /app/.next/server/chunks/76514.js:27:8152
    at async getUserByAccount (/app/.next/server/app/api/auth/[...nextauth]/route.js:1:3596)
    at async r.<computed> (/app/.next/server/chunks/7247.js:6:41304)
    at async ir (/app/.next/server/chunks/7247.js:362:34258)
    at async ip (/app/.next/server/chunks/7247.js:362:45365)
    at async im (/app/.next/server/chunks/7247.js:362:49738)
    at async /app/node_modules/.pnpm/next@14.2.5_@babel+core@7.23.6_@opentelemetry+api@1.9.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/next/dist/compiled/next-server/app-route.runtime.prod.js:6:36258
    at async eR.execute (/app/node_modules/.pnpm/next@14.2.5_@babel+core@7.23.6_@opentelemetry+api@1.9.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/next/dist/compiled/next-server/app-route.runtime.prod.js:6:26874)
[auth][details]: {}
[auth][error] AdapterError: Read more at https://errors.authjs.dev#adaptererror
[auth][cause]: error: relation "nextauth_accounts" does not exist
    at /app/node_modules/.pnpm/pg-pool@3.6.2_pg@8.12.0/node_modules/pg-pool/index.js:45:11
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async /app/.next/server/chunks/76514.js:27:8152
    at async getUserByAccount (/app/.next/server/app/api/auth/[...nextauth]/route.js:1:3596)
    at async r.<computed> (/app/.next/server/chunks/7247.js:6:41304)
    at async ir (/app/.next/server/chunks/7247.js:362:34258)
    at async ip (/app/.next/server/chunks/7247.js:362:45365)
    at async im (/app/.next/server/chunks/7247.js:362:49738)
    at async /app/node_modules/.pnpm/next@14.2.5_@babel+core@7.23.6_@opentelemetry+api@1.9.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/next/dist/compiled/next-server/app-route.runtime.prod.js:6:36258
    at async eR.execute (/app/node_modules/.pnpm/next@14.2.5_@babel+core@7.23.6_@opentelemetry+api@1.9.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/next/dist/compiled/next-server/app-route.runtime.prod.js:6:26874)
[auth][details]: {}
[NextAuth] Error: {
  cause: 'Configuration',
  message: 'Wrong configuration, make sure you have the correct environment variables set. Visit https://lobehub.com/docs/self-hosting/advanced/authentication for more details.',
  name: 'NextAuth Error'
}

最开始提示lobechat数据库不存在，于是我就自己创建了一个，还是报错，数据库的日志如下

2024-08-11 13:53:49.744 UTC [1] LOG:  database system is ready to accept connections
2024-08-11 13:54:32.021 UTC [72] ERROR:  database "lobechat" does not exist
2024-08-11 13:54:32.021 UTC [72] STATEMENT:  DROP DATABASE "lobechat"
2024-08-11 13:54:32.022 UTC [72] ERROR:  role "lobechat" does not exist
2024-08-11 13:54:32.022 UTC [72] STATEMENT:  DROP USER "lobechat"
2024-08-11 13:55:36.501 UTC [76] FATAL:  database "lobechat" does not exist
2024-08-11 13:57:42.408 UTC [83] ERROR:  relation "nextauth_accounts" does not exist at character 679
2024-08-11 13:57:42.408 UTC [83] STATEMENT:  select "nextauth_accounts"."access_token", "nextauth_accounts"."expires_at", "nextauth_accounts"."id_token", "nextauth_accounts"."provider", "nextauth_accounts"."providerAccountId", "nextauth_accounts"."refresh_token", "nextauth_accounts"."scope", "nextauth_accounts"."session_state", "nextauth_accounts"."token_type", "nextauth_accounts"."type", "nextauth_accounts"."userId", "users"."id", "users"."username", "users"."email", "users"."avatar", "users"."phone", "users"."first_name", "users"."last_name", "users"."full_name", "users"."is_onboarded", "users"."clerk_created_at", "users"."email_verified_at", "users"."preference", "users"."created_at", "users"."updated_at" from "nextauth_accounts" inner join "users" on "nextauth_accounts"."userId" = "users"."id" where ("nextauth_accounts"."provider" = $1 and "nextauth_accounts"."providerAccountId" = $2)
2024-08-11 13:58:49.834 UTC [61] LOG:  checkpoint starting: time
lobehubbot commented 3 months ago

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

@arvinxx The table will not be created automatically. An error will be reported after the login is completed. pg is also deployed with docker.

[auth][cause]: error: relation "nextauth_accounts" does not exist
at /app/node_modules/.pnpm/pg-pool@3.6.2_pg@8.12.0/node_modules/pg-pool/index.js:45:11
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /app/.next/server/chunks/76514.js:27:8152
at async getUserByAccount (/app/.next/server/app/api/auth/[...nextauth]/route.js:1:3596)
at async r.<computed> (/app/.next/server/chunks/7247.js:6:41304)
at async ir (/app/.next/server/chunks/7247.js:362:34258)
at async ip (/app/.next/server/chunks/7247.js:362:45365)
at async im (/app/.next/server/chunks/7247.js:362:49738)
at async /app/node_modules/.pnpm/next@14.2.5_@babel+core@7.23.6_@opentelemetry+api@1.9.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/next/ dist/compiled/next-server/app-route.runtime.prod.js:6:36258
at async eR.execute (/app/node_modules/.pnpm/next@14.2.5_@babel+core@7.23.6_@opentelemetry+api@1.9.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/ node_modules/next/dist/compiled/next-server/app-route.runtime.prod.js:6:26874)
[auth][details]: {}
[auth][error] AdapterError: Read more at https://errors.authjs.dev#adaptererror
[auth][cause]: error: relation "nextauth_accounts" does not exist
at /app/node_modules/.pnpm/pg-pool@3.6.2_pg@8.12.0/node_modules/pg-pool/index.js:45:11
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async /app/.next/server/chunks/76514.js:27:8152
at async getUserByAccount (/app/.next/server/app/api/auth/[...nextauth]/route.js:1:3596)
at async r.<computed> (/app/.next/server/chunks/7247.js:6:41304)
at async ir (/app/.next/server/chunks/7247.js:362:34258)
at async ip (/app/.next/server/chunks/7247.js:362:45365)
at async im (/app/.next/server/chunks/7247.js:362:49738)
at async /app/node_modules/.pnpm/next@14.2.5_@babel+core@7.23.6_@opentelemetry+api@1.9.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/next/ dist/compiled/next-server/app-route.runtime.prod.js:6:36258
at async eR.execute (/app/node_modules/.pnpm/next@14.2.5_@babel+core@7.23.6_@opentelemetry+api@1.9.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/ node_modules/next/dist/compiled/next-server/app-route.runtime.prod.js:6:26874)
[auth][details]: {}
[NextAuth] Error: {
cause: 'Configuration',
message: 'Wrong configuration, make sure you have the correct environment variables set. Visit https://lobehub.com/docs/self-hosting/advanced/authentication for more details.',
name: 'NextAuth Error'
}
arvinxx commented 3 months ago

@coulsontl 不要自己建表，否则出错了自己解决，帮不了你。默认的 db 镜像里会自动创建表结构和自动完成数据库迁移的。如果不行建议检查你的db url写的对不对，不行就删掉 db 重来。

如果你是数据库专家当我没说，自己看下 src/database/server/migration 里的sql ，自行执行表结构迁移应该也可以的。

lobehubbot commented 3 months ago

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

@coulsontl Don't create the table yourself, otherwise you will solve it yourself if something goes wrong, and I won't be able to help you. The default db image will automatically create a table structure and automatically complete database migration. If it doesn't work, it is recommended to check whether your db url is written correctly.

cy948 commented 3 months ago

@jueinin 请单开一个issue提供下背景信息以帮助我们解决问题，谢谢。

lobehubbot commented 3 months ago

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

@jueinin Please open a separate issue and provide background information to help us solve the problem, thank you.

morningtzh commented 3 months ago
image 
/ $ node /app/docker.cjs
[Database] Start to migration...
❌ Database migrate failed. Please check your database is valid and DATABASE_URL is set correctly. The error detail is below:
error: column "client_id" of relation "messages" already exists
    at /app/node_modules/.pnpm/pg@8.12.0/node_modules/pg/lib/client.js:526:17
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async /app/node_modules/drizzle-orm/pg-core/dialect.cjs:60:13
    at async NodePgSession.transaction (/app/node_modules/drizzle-orm/node-postgres/session.cjs:141:22)
    at async PgDialect.migrate (/app/node_modules/drizzle-orm/pg-core/dialect.cjs:56:5)
    at async Object.migrate (/app/node_modules/drizzle-orm/node-postgres/migrator.cjs:27:3)
    at async runMigrations (/app/docker.cjs:16:3) {
  length: 136,
  severity: 'ERROR',
  code: '42701',
  detail: undefined,
  hint: undefined,
  position: undefined,
  internalPosition: undefined,
  internalQuery: undefined,
  where: undefined,
  schema: undefined,
  table: undefined,
  column: undefined,
  dataType: undefined,
  constraint: undefined,
  file: 'tablecmds.c',
  line: '7134',
  routine: 'check_for_column_name_collision'
}

之前一直使用 Dockerfile 构建的镜像，传入了clark参数用于登录。今天保留原有构建环境变量（clark / s3 / db），尝试使用 Dockerfile.database 构建镜像，发现起不了。bash进入镜像后，export DATABASE_URL 再运行 node /app/docker.cjs，发现有数据库字段冲突的问题，不太清楚是什么原因导致的冲突。

也看了上面自己建表的问题，我这个是早期通过 db:push 建立的。

麻烦帮忙看一下是否需要重新建立数据库，另外 Dockerfile.database 镜像是否还兼容构建时传入clark参数用于登录。

arvinxx commented 3 months ago

发现有数据库字段冲突的问题，不太清楚是什么原因导致的冲突。

@morningtzh 可以看下这段迁移 sql ，https://github.com/lobehub/lobe-chat/blob/main/src/database/server/migrations/0001_add_client_id.sql

这里的步骤是都会执行一遍的，如果你是用的db:push ，那么数据库里已经有 client_id 了，这里自然会冲突。

解决方案有两种：

  1. 如果是完全从头自己建镜像，那么可以直接注释这一个文件中的所有 sql ，这样迁移脚本虽然会执行这个 sql 文件，但是由于里面是空的，所以会认为执行成功，这样就能进到下一个 sql 迁移脚本。如果还报错就继续用类似方法，直到全部执行完毕。
  2. 删掉数据库重新部署一遍，这样应该不需要做什么操作直接就可以初始化完毕。

Dockerfile.database 镜像是否还兼容构建时传入clark参数用于登录

理论上应该是还支持的。你可以试下。

lobehubbot commented 3 months ago

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

Found a database field conflict problem, not sure what caused the conflict.

@morningtzh You can take a look at this migration sql, https://github.com/lobehub/lobe-chat/blob/main/src/database/server/migrations/0001_add_client_id.sql

The steps here are all executed once. If you use db:push, then there is already client_id in the database, and there will naturally be a conflict here.

There are two solutions:

  1. If you build the image completely from scratch, you can directly comment all the sql in this file. In this way, although the migration script will execute the sql file, it will be considered successful because it is empty, so you can enter Next sql migration script. If an error is still reported, continue using similar methods until all executions are completed.
  2. Delete the database and redeploy it, so that the initialization can be completed without any operations.

Is the Dockerfile.database image still compatible with the clark parameter passed in when building for login?

Theoretically it should be supported. You can try it.

morningtzh commented 2 months ago

发现有数据库字段冲突的问题，不太清楚是什么原因导致的冲突。

@morningtzh 可以看下这段迁移 sql ，https://github.com/lobehub/lobe-chat/blob/main/src/database/server/migrations/0001_add_client_id.sql

这里的步骤是都会执行一遍的，如果你是用的db:push ，那么数据库里已经有 client_id 了，这里自然会冲突。

解决方案有两种：

  1. 如果是完全从头自己建镜像，那么可以直接注释这一个文件中的所有 sql ，这样迁移脚本虽然会执行这个 sql 文件，但是由于里面是空的，所以会认为执行成功，这样就能进到下一个 sql 迁移脚本。如果还报错就继续用类似方法，直到全部执行完毕。
  2. 删掉数据库重新部署一遍，这样应该不需要做什么操作直接就可以初始化完毕。

Dockerfile.database 镜像是否还兼容构建时传入clark参数用于登录

理论上应该是还支持的。你可以试下。

谢谢 确实可以，我把0001和0002都注释掉了，运行时环境变量补全S3和DB的就能使用，auth0并不是强制依赖的，clerk也能结合新的 Dockerfile.database 镜像运行。

使用手机登录（电脑也在线），切换助手后端会报错 session not found，偶尔前端会报一下，但不影响使用。接下来持续观察一下~

2024-08-14T15:09:36.977934650Z {"level":30,"time":1723648176977,"pid":1,"hostname":"lobe-chat--7tv2x58-7d46986c4f-6zxk9","msg":"Error in tRPC handler (lambda) on path: session.getSessionConfig, type: query"}
2024-08-14T15:09:36.978873296Z Error: Session not found
2024-08-14T15:09:36.978898995Z     at /app/.next/server/app/trpc/lambda/[trpc]/route.js:4:22323
2024-08-14T15:09:36.978909034Z     ... 8 lines matching cause stack trace ...
2024-08-14T15:09:36.978917249Z     at async Promise.all (index 1) {
2024-08-14T15:09:36.978925515Z   code: 'INTERNAL_SERVER_ERROR',
2024-08-14T15:09:36.978934953Z   name: 'TRPCError',
2024-08-14T15:09:36.978943358Z   [cause]: Error: Session not found
2024-08-14T15:09:36.978951373Z       at /app/.next/server/app/trpc/lambda/[trpc]/route.js:4:22323
2024-08-14T15:09:36.978960430Z       at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
2024-08-14T15:09:36.978968566Z       at async E.f.middlewares (/app/.next/server/chunks/54430.js:1:1900)
2024-08-14T15:09:36.978978424Z       at async t (/app/.next/server/chunks/54430.js:1:2174)
2024-08-14T15:09:36.978986429Z       at async t (/app/.next/server/chunks/54430.js:1:2174)
2024-08-14T15:09:36.978994554Z       at async t (/app/.next/server/chunks/54430.js:1:2174)
2024-08-14T15:09:36.979002319Z       at async t (/app/.next/server/chunks/54430.js:1:2174)
2024-08-14T15:09:36.979009853Z       at async r (/app/.next/server/chunks/54430.js:1:2513)
2024-08-14T15:09:36.979017548Z       at async /app/.next/server/chunks/50893.js:1:7091
2024-08-14T15:09:36.979025503Z       at async Promise.all (index 1)
2024-08-14T15:09:36.979033047Z }