search

didi / xiaoju-survey

XIAOJUSURVEY is an enterprises form builder and analytics platform that allows users to create questionnaires, exams, polls, quizzes, and analyze data online.
https://xiaojusurvey.didi.cn
Apache License 2.0
2.23k stars 291 forks source link

💡 MAJOR CHANGELOG #48

Open sudoooooo opened 10 months ago

sudoooooo commented 10 months ago

2024.10

1、新增：数据上报

我们已有非常多的功能迭代，但是我们并不知道这些迭代的实际使用情况。XIAOJUSURVEY 是一个产品化的项目，我们需要定期进行项目规划和评估，希望通过版本数据的收集来观测实际使用情况，用以指导我们迭代的频率和发展计划。

1、数据上报协议：新增了 DATA_COLLECTION.md 用以协议声明。

2、数据上报声明以及如何关闭：数据上报声明

2、变更：优化表字段

对应docker tag1.3.0

⚠️ 本次是非兼容变更，若您已经使用本项目一段时间，数据库里已有数据，本次升级（或者使用Docker 1.3.0），需要按照以下方式更新存量数据：

1、查看升级变更原因和内容

2、如何同步本次升级？

请按照 升级步骤 进行操作。

2024.5.20

对应docker tag1.1.2

web部署由托管在server变更为nginx启动,docker启动的端口从3000变为80: image

2024.5.10

对应docker tag1.1.0

web升级Vue3，具体查看升级文档

2024.1.30

变更

对应commit tag1.2.0

对应docker tag1.0.4

服务端架构应用koa升级成nestjs，变化点如下： 1、服务端架构升级到nestjs 2、安全插件化方案 3、传输加密使用rsa加密 4、服务端单测 5、日志 6、环境变量放到配置文件维护，同时删除无效变量，具体查看.env文件 特别注意DB地XIAOJU_SURVEY_MONGO_AUTH_SOURCE需要单独配置

XIAOJU_SURVEY_MONGO_DB_NAME=xiaojuSurvey
XIAOJU_SURVEY_MONGO_URL=mongodb://localhost:27017
XIAOJU_SURVEY_MONGO_AUTH_SOURCE=

XIAOJU_SURVEY_RESPONSE_AES_ENCRYPT_SECRET_KEY=dataAesEncryptSecretKey
XIAOJU_SURVEY_HTTP_DATA_ENCRYPT_TYPE=rsa

XIAOJU_SURVEY_JWT_SECRET=xiaojuSurveyJwtSecret
XIAOJU_SURVEY_JWT_EXPIRES_IN=8h

XIAOJU_SURVEY_LOGGER_FILENAME=./logs/app.log

7、升级前版本：https://github.com/didi/xiaoju-survey/releases/tag/v1.1.0

2024.1.17

变更

对应commit tag1.2.0

对应docker tag1.0.3

优化服务端环境变量引入方式，主要优化了配置引用方式和命名规范： 1、xiaojuSurveyMongoUrl 规范化成 XIAOJU_SURVEY_MONGO_URL 2、xiaojuSurveyJwtSecret 规范化成 XIAOJU_SURVEY_JWT_SECRET 3、xiaojuSurveyJwtExpiresIn 规范化成 XIAOJU_SURVEY_JWT_EXPIRES_IN 4、xiaojuSurveySessionExpireTime 使用 XIAOJU_SURVEY_JWT_EXPIRES_IN 5、xiaojuSurveyEncryptType 规范化成 XIAOJU_SURVEY_ENCRYPT_TYPE 6、xiaojuSurveyDataAesEncryptSecretKey 规范化成 XIAOJU_SURVEY_ENCRYPT_SECRET_KEY 7、xiaojuSurveyAesCodelength 使用 XIAOJU_SURVEY_ENCRYPT_TYPE_LEN

来源

@alwayrun PR 🎉

影响

⚠️ 服务重新部署前，特别注意您的环境配置，需重新配置环境变量，尤其是MongoDB和启动，如下提示，最新内容看文档

coder-xiaomo commented 9 months ago

@sudoooooo 是否考虑将配置从环境变量转移到配置文件呢，这样配置起来会方便一些，也方便后续扩展

sudoooooo commented 9 months ago

@coder-xiaomo 你好，server架构升级后统一在配置文件维护，可查看.env文件

linux-y commented 8 months ago

崩溃啦，为什么我用docker启动一直报连不上数据库？ [root@redis survey]# docker -v Docker version 19.03.5, build 633a0ea [root@redis survey]# docker-compose -v docker-compose version 1.24.1, build 4667896b [root@redis survey]# cat /etc/redhat-release CentOS Linux release 8.3.2011 [root@redis survey]# uname -a Linux redis 4.18.0-240.el8.x86_64 #1 SMP Fri Sep 25 19:48:47 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

docker-compose.yml如下： `[root@redis survey]# docker logs -f xiaoju-survey

server@0.0.1 start:prod NODE_ENV=production node dist/main

[Nest] 19 - 03/25/2024, 10:03:02 AM LOG [NestFactory] Starting Nest application... [Nest] 19 - 03/25/2024, 10:03:02 AM LOG [InstanceLoader] TypeOrmModule dependencies initialized +34ms [Nest] 19 - 03/25/2024, 10:03:02 AM LOG [InstanceLoader] ConfigHostModule dependencies initialized +1ms [Nest] 19 - 03/25/2024, 10:03:02 AM LOG [InstanceLoader] ServeStaticModule dependencies initialized +0ms [Nest] 19 - 03/25/2024, 10:03:02 AM LOG [InstanceLoader] ConfigModule dependencies initialized +0ms [Nest] 19 - 03/25/2024, 10:03:02 AM LOG [InstanceLoader] ConfigModule dependencies initialized +0ms [Nest] 19 - 03/25/2024, 10:03:02 AM LOG [InstanceLoader] AppModule dependencies initialized +25ms [Nest] 19 - 03/25/2024, 10:03:02 AM ERROR [TypeOrmModule] Unable to connect to the database. Retrying (1)... MongoServerError: Invalid database name: '' at Connection.onMessage (/xiaoju-survey/server/node_modules/mongodb/lib/cmap/connection.js:202:26) at MessageStream. (/xiaoju-survey/server/node_modules/mongodb/lib/cmap/connection.js:61:60) at MessageStream.emit (node:events:513:28) at processIncomingData (/xiaoju-survey/server/node_modules/mongodb/lib/cmap/message_stream.js:124:16) at MessageStream._write (/xiaoju-survey/server/node_modules/mongodb/lib/cmap/message_stream.js:33:9) at writeOrBuffer (node:internal/streams/writable:391:12) at _write (node:internal/streams/writable:332:10) at MessageStream.Writable.write (node:internal/streams/writable:336:10) at Socket.ondata (node:internal/streams/readable:754:22) at Socket.emit (node:events:513:28) [Nest] 19 - 03/25/2024, 10:03:05 AM ERROR [TypeOrmModule] Unable to connect to the database. Retrying (2)... MongoServerError: Invalid database name: '' at Connection.onMessage (/xiaoju-survey/server/node_modules/mongodb/lib/cmap/connection.js:202:26) at MessageStream. (/xiaoju-survey/server/node_modules/mongodb/lib/cmap/connection.js:61:60) at MessageStream.emit (node:events:513:28) at processIncomingData (/xiaoju-survey/server/node_modules/mongodb/lib/cmap/message_stream.js:124:16) at MessageStream._write (/xiaoju-survey/server/node_modules/mongodb/lib/cmap/message_stream.js:33:9) at writeOrBuffer (node:internal/streams/writable:391:12) at _write (node:internal/streams/writable:332:10) at MessageStream.Writable.write (node:internal/streams/writable:336:10) at Socket.ondata (node:internal/streams/readable:754:22) at Socket.emit (node:events:513:28) ^C [root@redis survey]# cat docker-compose.yml .env version: "3.6" services: mongo: image: mongo:4 container_name: xiaoju-survey-mongo restart: always environment: MONGO_INITDB_ROOT_USERNAME: syy# 默认使用系统的环境变量 MONGO_INITDB_ROOT_PASSWORD: syy123 # 默认使用系统的环境变量 ports:

volumes: mongo-volume:

networks: xiaoju-survey: name: xiaoju-survey driver: bridge MONGO_INITDB_ROOT_USERNAME=syy MONGO_INITDB_ROOT_PASSWORD=syy123 XIAOJU_SURVEY_MONGO_DB_NAME=xiaojuSurvey XIAOJU_SURVEY_MONGO_URL=mongodb://localhost:27017 XIAOJU_SURVEY_MONGO_AUTH_SOURCE=

XIAOJU_SURVEY_RESPONSE_AES_ENCRYPT_SECRET_KEY=dataAesEncryptSecretKey XIAOJU_SURVEY_HTTP_DATA_ENCRYPT_TYPE=rsa

XIAOJU_SURVEY_JWT_SECRET=xiaojuSurveyJwtSecret XIAOJU_SURVEY_JWT_EXPIRES_IN=8h

XIAOJU_SURVEY_LOGGER_FILENAME=./logs/app.log `

sudoooooo commented 8 months ago

崩溃啦，为什么我用docker启动一直报连不上数据库？ [root@redis survey]# docker -v Docker version 19.03.5, build 633a0ea [root@redis survey]# docker-compose -v docker-compose version 1.24.1, build 4667896b [root@redis survey]# cat /etc/redhat-release CentOS Linux release 8.3.2011 [root@redis survey]# uname -a Linux redis 4.18.0-240.el8.x86_64 #1 SMP Fri Sep 25 19:48:47 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

docker-compose.yml如下： `[root@redis survey]# docker logs -f xiaoju-survey

server@0.0.1 start:prod NODE_ENV=production node dist/main

[Nest] 19 - 03/25/2024, 10:03:02 AM LOG [NestFactory] Starting Nest application... [Nest] 19 - 03/25/2024, 10:03:02 AM LOG [InstanceLoader] TypeOrmModule dependencies initialized +34ms [Nest] 19 - 03/25/2024, 10:03:02 AM LOG [InstanceLoader] ConfigHostModule dependencies initialized +1ms [Nest] 19 - 03/25/2024, 10:03:02 AM LOG [InstanceLoader] ServeStaticModule dependencies initialized +0ms [Nest] 19 - 03/25/2024, 10:03:02 AM LOG [InstanceLoader] ConfigModule dependencies initialized +0ms [Nest] 19 - 03/25/2024, 10:03:02 AM LOG [InstanceLoader] ConfigModule dependencies initialized +0ms [Nest] 19 - 03/25/2024, 10:03:02 AM LOG [InstanceLoader] AppModule dependencies initialized +25ms [Nest] 19 - 03/25/2024, 10:03:02 AM ERROR [TypeOrmModule] Unable to connect to the database. Retrying (1)... MongoServerError: Invalid database name: '' at Connection.onMessage (/xiaoju-survey/server/node_modules/mongodb/lib/cmap/connection.js:202:26) at MessageStream. (/xiaoju-survey/server/node_modules/mongodb/lib/cmap/connection.js:61:60) at MessageStream.emit (node:events:513:28) at processIncomingData (/xiaoju-survey/server/node_modules/mongodb/lib/cmap/message_stream.js:124:16) at MessageStream._write (/xiaoju-survey/server/node_modules/mongodb/lib/cmap/message_stream.js:33:9) at writeOrBuffer (node:internal/streams/writable:391:12) at _write (node:internal/streams/writable:332:10) at MessageStream.Writable.write (node:internal/streams/writable:336:10) at Socket.ondata (node:internal/streams/readable:754:22) at Socket.emit (node:events:513:28) [Nest] 19 - 03/25/2024, 10:03:05 AM ERROR [TypeOrmModule] Unable to connect to the database. Retrying (2)... MongoServerError: Invalid database name: '' at Connection.onMessage (/xiaoju-survey/server/node_modules/mongodb/lib/cmap/connection.js:202:26) at MessageStream. (/xiaoju-survey/server/node_modules/mongodb/lib/cmap/connection.js:61:60) at MessageStream.emit (node:events:513:28) at processIncomingData (/xiaoju-survey/server/node_modules/mongodb/lib/cmap/message_stream.js:124:16) at MessageStream._write (/xiaoju-survey/server/node_modules/mongodb/lib/cmap/message_stream.js:33:9) at writeOrBuffer (node:internal/streams/writable:391:12) at _write (node:internal/streams/writable:332:10) at MessageStream.Writable.write (node:internal/streams/writable:336:10) at Socket.ondata (node:internal/streams/readable:754:22) at Socket.emit (node:events:513:28) ^C [root@redis survey]# cat docker-compose.yml .env version: "3.6" services: mongo: image: mongo:4 container_name: xiaoju-survey-mongo restart: always environment: MONGO_INITDB_ROOT_USERNAME: syy# 默认使用系统的环境变量 MONGO_INITDB_ROOT_PASSWORD: syy123 # 默认使用系统的环境变量 ports: - "27017:27017" # 数据库端口 volumes: - mongo-volume:/data/db # xiaoju-survey-data/db/data:/data/db networks: - xiaoju-survey

xiaoju-survey: image: "xiaojusurvey/xiaoju-survey:1.0.4" container_name: xiaoju-survey restart: always ports: - "8080:3000" # API端口 environment: MONGO_INITDB_ROOT_USERNAME: syy # 默认使用系统的环境变量 MONGO_INITDB_ROOT_PASSWORD: syy123 # 默认使用系统的环境变量 XIAOJU_SURVEY_MONGO_DB_NAME: xiaojuSurvey XIAOJU_SURVEY_MONGO_AUTH_SOURCE: XIAOJU_SURVEY_MONGO_URL: mongodb://syy:syy123@192.168.2.246:27017/xiaojuSurvey #XIAOJU_SURVEY_MONGO_URL: mongodb://syy:syy123@xiaoju-survey-mongo:27017 # docker-compose 会根据容器名称自动处理 links: - mongo:mongo depends_on: - mongo networks: - xiaoju-survey

volumes: mongo-volume:

networks: xiaoju-survey: name: xiaoju-survey driver: bridge MONGO_INITDB_ROOT_USERNAME=syy MONGO_INITDB_ROOT_PASSWORD=syy123 XIAOJU_SURVEY_MONGO_DB_NAME=xiaojuSurvey XIAOJU_SURVEY_MONGO_URL=mongodb://localhost:27017 XIAOJU_SURVEY_MONGO_AUTH_SOURCE=

XIAOJU_SURVEY_RESPONSE_AES_ENCRYPT_SECRET_KEY=dataAesEncryptSecretKey XIAOJU_SURVEY_HTTP_DATA_ENCRYPT_TYPE=rsa

XIAOJU_SURVEY_JWT_SECRET=xiaojuSurveyJwtSecret XIAOJU_SURVEY_JWT_EXPIRES_IN=8h

XIAOJU_SURVEY_LOGGER_FILENAME=./logs/app.log `

目测是这个问题：https://github.com/didi/xiaoju-survey/issues/70