search

DigitalPlatform / dp2

Integrated Library System / 图书馆集成系统
http://digitalplatform.github.io/dp2
Apache License 2.0
105 stars 54 forks source link

关于安装MongoDB的问题 #629

Open renyh opened 4 years ago

renyh commented 4 years ago

（此问题已解决）

2020/3/29-1711-ryh：安装上次报错的mongodb4.2.2，用新版本dp2library访问mongodb和启动正常，为读者借还，产生借阅历史数据正常。新版本dp2library已解决mongodb新版本驱动问题。

安装MongoDB最新版本mongodb-win32-x86_64-2012plus-4.2.2-signed.msi 或者之前版本mongodb-win32-x86_64-2012plus-4.2.0-signed.msi dp2library启动时会报下面错误:

2020/1/13 10:56:53 INFO: BeginWatcher
2020/1/13 10:56:53 INFO: End BeginWatcher
2020/1/13 10:56:53 LibraryService 初始化过程发生严重错误 [LoadCfg() 抛出异常: Type: MongoDB.Driver.MongoCommandException
Message: Command 'group' failed: no such command: 'group' (response: { "ok" : 0.0, "errmsg" : "no such command: 'group'", "code" : 59, "codeName" : "CommandNotFound" })
Stack:
   在 MongoDB.Driver.Operations.CommandOperation`1.Execute(MongoConnection connection)
   在 MongoDB.Driver.MongoCollection.RunCommandAs[TCommandResult](IMongoCommand command, ReadPreference readPreference, IBsonSerializer resultSerializer, IBsonSerializationOptions resultSerializationOptions)
   在 MongoDB.Driver.MongoCollection.RunCommandAs[TCommandResult](IMongoCommand command, ReadPreference readPreference)
   在 MongoDB.Driver.MongoCollection.Group(GroupArgs args)
   在 MongoDB.Driver.MongoCollection.Group(IMongoQuery query, BsonJavaScript keyFunction, BsonDocument initial, BsonJavaScript reduce, BsonJavaScript finalize)
   在 DigitalPlatform.LibraryServer.AccessLogDatabase.GetItemCount(String date)
   在 DigitalPlatform.LibraryServer.AccessLogDatabase.Open(MongoClient client, String strInstancePrefix, String& strError)
   在 DigitalPlatform.LibraryServer.LibraryApplication.InitialMongoDatabases(String& strError)
   在 DigitalPlatform.LibraryServer.LibraryApplication.LoadCfg(Boolean bReload, String strDataDir, String strHostDir, String& strError)

]，当前此服务处于残缺状态，请及时排除故障后重新启动
2020/1/13 10:56:53 *** 当前系统已经被挂起 StartingError

安装Mongodb这个版本mongodb-win32-x86_64-2008plus-ssl-4.0.2-signed.msi，dp2library启动正常。

renyh commented 4 years ago

测试mongodb版本升级1

测试目的：测试新版本mongodb是否可以使用旧版本时的数据目录和日志目录

结论：新版本的mongodb4.2.3没法直接使用旧版本3.4.0的数据目录和日志目录，不兼容。

测试步骤如下：

  1. 先装mongodb3.4.0，这个mongodb版本安装时不会自动安装为windows service，需要dp2installer辅助安装为service。
  2. 安装后为一个读者做一些借还，为mongodb里产生了借阅历史
  3. 先停止dp2library service，如果不停止后面没法卸载干净mongodb，然后再卸载mongodb，注意卸载完检查windows service是否还存在。注意不要删除原来的数据目录和日志目录。
  4. 新安装mongodb4.2.3版本，安装时指定数据目录和日志目录分别为老版本的数据目录和日志。下一步直到安装完成。
  5. 从windows service检查，发现mongodb没有启动起来，检查mongodb日志，发现新版本没法使用老版本的数据目录，不兼容。 
    2020-03-29T14:56:55.469+0800 I  CONTROL  [main] ***** SERVER RESTARTED *****
2020-03-29T14:56:56.217+0800 I  CONTROL  [main] Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'
2020-03-29T14:56:56.221+0800 I  CONTROL  [main] Trying to start Windows service 'MongoDB'
2020-03-29T14:56:56.222+0800 I  CONTROL  [initandlisten] MongoDB starting : pid=13900 port=27017 dbpath=C:\mongo_data\db 64-bit host=DESKTOP-6HBP3H4
2020-03-29T14:56:56.222+0800 I  CONTROL  [initandlisten] targetMinOS: Windows 7/Windows Server 2008 R2
2020-03-29T14:56:56.222+0800 I  CONTROL  [initandlisten] db version v4.2.3
2020-03-29T14:56:56.223+0800 I  CONTROL  [initandlisten] git version: 6874650b362138df74be53d366bbefc321ea32d4
2020-03-29T14:56:56.223+0800 I  CONTROL  [initandlisten] allocator: tcmalloc
2020-03-29T14:56:56.223+0800 I  CONTROL  [initandlisten] modules: none
2020-03-29T14:56:56.223+0800 I  CONTROL  [initandlisten] build environment:
2020-03-29T14:56:56.223+0800 I  CONTROL  [initandlisten]     distmod: 2012plus
2020-03-29T14:56:56.223+0800 I  CONTROL  [initandlisten]     distarch: x86_64
2020-03-29T14:56:56.223+0800 I  CONTROL  [initandlisten]     target_arch: x86_64
2020-03-29T14:56:56.223+0800 I  CONTROL  [initandlisten] options: { config: "C:\Program Files\MongoDB\Server\4.2\bin\mongod.cfg", net: { bindIp: "127.0.0.1", port: 27017 }, service: true, storage: { dbPath: "C:\mongo_data\db", journal: { enabled: true } }, systemLog: { destination: "file", logAppend: true, path: "C:\mongo_data\log\mongod.log" } }
2020-03-29T14:56:56.230+0800 I  STORAGE  [initandlisten] Detected data files in C:\mongo_data\db created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'.
2020-03-29T14:56:56.231+0800 I  STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=7622M,cache_overflow=(file_max=0M),session_max=33000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000,close_scan_interval=10,close_handle_minimum=250),statistics_log=(wait=0),verbose=[recovery_progress,checkpoint_progress],
2020-03-29T14:56:56.301+0800 I  STORAGE  [initandlisten] WiredTiger message [1585465016:301170][13900:140713013894736], txn-recover: Recovering log 2 through 3
2020-03-29T14:56:56.302+0800 I  STORAGE  [initandlisten] WiredTiger message [1585465016:301170][13900:140713013894736], txn-recover: Recovering log 3 through 3
2020-03-29T14:56:56.361+0800 I  STORAGE  [initandlisten] WiredTiger message [1585465016:360968][13900:140713013894736], txn-recover: Main recovery loop: starting at 2/5760 to 3/256
2020-03-29T14:56:56.413+0800 I  STORAGE  [initandlisten] WiredTiger message [1585465016:412869][13900:140713013894736], txn-recover: Recovering log 2 through 3
2020-03-29T14:56:56.416+0800 I  STORAGE  [initandlisten] WiredTiger message [1585465016:415822][13900:140713013894736], txn-recover: Recovering log 3 through 3
2020-03-29T14:56:56.478+0800 I  STORAGE  [initandlisten] WiredTiger message [1585465016:477658][13900:140713013894736], txn-recover: Set global recovery timestamp: (0, 0)
2020-03-29T14:56:56.502+0800 I  RECOVERY [initandlisten] WiredTiger recoveryTimestamp. Ts: Timestamp(0, 0)
2020-03-29T14:56:56.516+0800 I  STORAGE  [initandlisten] WiredTigerKVEngine shutting down
2020-03-29T14:56:56.516+0800 I  STORAGE  [initandlisten] Shutting down session sweeper thread
2020-03-29T14:56:56.516+0800 I  STORAGE  [initandlisten] Finished shutting down session sweeper thread
2020-03-29T14:56:56.516+0800 I  STORAGE  [initandlisten] Shutting down journal flusher thread
2020-03-29T14:56:56.603+0800 I  STORAGE  [initandlisten] Finished shutting down journal flusher thread
2020-03-29T14:56:56.603+0800 I  STORAGE  [initandlisten] Shutting down checkpoint thread
2020-03-29T14:56:56.603+0800 I  STORAGE  [initandlisten] Finished shutting down checkpoint thread
2020-03-29T14:56:56.603+0800 I  STORAGE  [initandlisten] Downgrading WiredTiger datafiles.
2020-03-29T14:56:56.700+0800 I  STORAGE  [initandlisten] WiredTiger message [1585465016:700530][13900:140713013894736], txn-recover: Recovering log 3 through 4
2020-03-29T14:56:56.752+0800 I  STORAGE  [initandlisten] WiredTiger message [1585465016:751393][13900:140713013894736], txn-recover: Recovering log 4 through 4
2020-03-29T14:56:56.804+0800 I  STORAGE  [initandlisten] WiredTiger message [1585465016:804252][13900:140713013894736], txn-recover: Main recovery loop: starting at 3/2048 to 4/256
2020-03-29T14:56:56.899+0800 I  STORAGE  [initandlisten] WiredTiger message [1585465016:898998][13900:140713013894736], txn-recover: Recovering log 3 through 4
2020-03-29T14:56:56.961+0800 I  STORAGE  [initandlisten] WiredTiger message [1585465016:960931][13900:140713013894736], txn-recover: Recovering log 4 through 4
2020-03-29T14:56:57.007+0800 I  STORAGE  [initandlisten] WiredTiger message [1585465017:6815][13900:140713013894736], txn-recover: Set global recovery timestamp: (0, 0)
**2020-03-29T14:56:57.094+0800 I  STORAGE  [initandlisten] exception in initAndListen: MustDowngrade: Collection does not have UUID in KVCatalog. Collection: admin.system.version, terminating**
2020-03-29T14:56:57.095+0800 I  NETWORK  [serviceStopWorker] shutdown: going to close listening sockets...
2020-03-29T14:56:57.095+0800 I  -        [serviceStopWorker] Stopping further Flow Control ticket acquisitions.
2020-03-29T14:56:57.095+0800 I  CONTROL  [serviceStopWorker] now exiting
  6. 卸载4.2.3版本，注意检查windows service也要卸载干净。
  7. 重装4.2.3版本，此时使用新的数据目录和日志日志，不使用老版本的数据目录，直到安装完成，从windows service检查mongodb service服务正常启动。

测试mongodb版本升级2-用冷备份和恢复的方式

1）先对旧版本停止MongoDB服务器，再备份MongoDB的数据目录。最后卸载干净旧mongodb。 2）安装mongodb新版本，然后把备份的数据目录拷到新版本mongodb的数据目录，然后启动mongodb服务器。依据报下面的错误，与安装时指定目录一样。 2020-03-29T15:49:45.693+0800 I STORAGE [initandlisten] exception in initAndListen: MustDowngrade: Collection does not have UUID in KVCatalog. Collection: admin.system.version, terminating 当初是用新版本的数据目录改了一下名，然后创建这个数据目录，再把备份的数据目录中的所有文件拷到这个新目录中，中间出现数据目录没权限的问题，如下：

2020-03-29T15:34:24.566+0800 I  STORAGE  [initandlisten] exception in initAndListen: IllegalOperation: Attempted to create a lock file on a read-only directory: C:\Program Files\MongoDB\Server\4.2\data, terminating

后来从网上查资料，通过下面这个命令解决了权限问题，重启mongodb服务，则报2)中说的UUID的错误

mongod --dbpath=$(echo ~)/data

参考地址：https://stackoverflow.com/questions/42446931/mongodb-exception-in-initandlisten-20-attempted-to-create-a-lock-file-on-a-rea/47850090#47850090

renyh commented 4 years ago

测试mongodb版本升级3

测试目的：测试新版本mongodb是否可以导入旧版本通过mongodb命令导出的数据

结论：针对mongodb旧版本3.4.0，用mongodump导出各个库的数据；再安装新版本4.2.3，用mongorestore导入备份的数据，恢复原数据表现正常。

测试步骤如下：

  1. 创建一个完全干净的环境，机器上没有mongodb安装残留文件。

  2. 先装mongodb3.4.0，这个mongodb版本安装时不会自动安装为windows service，需要dp2installer辅助安装为service。

  3. 安装后为一个读者做一些借还，为mongodb里产生了借阅历史

  4. 使用下面命令导出数据

    mongodump -d test_accessLog -o C:\0-d\wx_backup\20200329
mongodump -d test_bibliosummary -o C:\0-d\wx_backup\20200329
mongodump -d test_chargingOper -o C:\0-d\wx_backup\20200329
mongodump -d test_hitcount -o C:\0-d\wx_backup\20200329

  5. 实际mongodb还有两个collection分别为admin和local，但检查里面的数据与dp2系统无关，所以不用备份。 admin存储的是版本信息如下：

    > use admin
switched to db admin
> show collections
system.version
> db.system.version.find()
{ "_id" : "featureCompatibilityVersion", "version" : "3.4" }

    location中存储的是一些配置信息，如下：

    > use local
switched to db local
> show collections
startup_log
> db.startup_log.find()
{ "_id" : "DESKTOP-6HBP3H4-1585469055455", "hostname" : "DESKTOP-6HBP3H4", "startTime" : ISODate("2020-03-29T08:04:15Z"), "startTimeLocal" : "Sun Mar 29 16:04:15.455", "cmdLine" : { "config" : "c:\\mongo_data\\mongod.cfg", "net" : { "bindIp" : "127.0.0.1", "port" : 27017 }, "service" : true, "storage" : { "dbPath" : "c:\\mongo_data\\db" }, "systemLog" : { "destination" : "file", "path" : "c:\\mongo_data\\log\\mongod.log" } }, "pid" : NumberLong(18792), "buildinfo" : { "version" : "3.4.0", "gitVersion" : "f4240c60f005be757399042dc12f6addbc3170c1", "targetMinOS" : "Windows 7/Windows Server 2008 R2", "modules" : [ ], "allocator" : "tcmalloc", "javascriptEngine" : "mozjs", "sysInfo" : "deprecated", "versionArray" : [ 3, 4, 0, 0 ], "openssl" : { "running" : "OpenSSL 1.0.1t-fips  3 May 2016", "compiled" : "OpenSSL 1.0.1t-fips  3 May 2016" }, "buildEnvironment" : { "distmod" : "2008plus-ssl", "distarch" : "x86_64", "cc" : "cl: Microsoft (R) C/C++ Optimizing Compiler Version 19.00.23918 for x64", "ccflags" : "/nologo /EHsc /W3 /wd4355 /wd4800 /wd4267 /wd4244 /wd4290 /wd4068 /wd4351 /we4013 /we4099 /we4930 /Z7 /errorReport:none /MD /O2 /Oy- /bigobj /Gw /Gy /Zc:inline", "cxx" : "cl: Microsoft (R) C/C++ Optimizing Compiler Version 19.00.23918 for x64", "cxxflags" : "/TP", "linkflags" : "/nologo /DEBUG /INCREMENTAL:NO /LARGEADDRESSAWARE /OPT:REF", "target_arch" : "x86_64", "target_os" : "windows" }, "bits" : 64, "debug" : false, "maxBsonObjectSize" : 16777216, "storageEngines" : [ "devnull", "ephemeralForTest", "mmapv1", "wiredTiger" ] } }

  6. 卸载mongodb，注意卸载完检查windows service是否还存在。如果还存在，用管理员身份启动命令行，输入sc delete mongodb，删除这个service，刷新或者重开windows service，看到mongodb这项不存在了。 另外删除原来的数据目录和日志目录。

  7. 新安装mongodb4.2.3版本，使用安装默认的数据目录和日志目录， 例如C:\Program Files\MongoDB\Server\4.2\data\和C:\Program Files\MongoDB\Server\4.2\log\，下一步直到安装完成。安装完检查windows service中mongodb服务应正常启动。

  8. 用内务连上dp2library，查看第3步读者的借阅历史，看到没有信息。

  9. 使用下面命令给mongodb新版本导入老版本数据，成功导入老版本时导出的数据。

    mongorestore -d test_accessLog  --drop C:\0-d\wx_backup\20200329\test_accessLog
mongorestore -d test_bibliosummary --drop  C:\0-d\wx_backup\20200329\test_bibliosummary
mongorestore -d test_chargingOper --drop C:\0-d\wx_backup\20200329\test_chargingOper
mongorestore -d test_hitcount --drop C:\0-d\wx_backup\20200329\test_hitcount

执行结果如下：


C:\Program Files\MongoDB\Server\4.2\bin>mongorestore -d test_accessLog  --drop C:\0-d\wx_backup\20200329\test_accessLog
2020-03-29T16:40:56.893+0800    the --db and --collection args should only be used when restoring from a BSON file. Other uses are deprecated and will not exist in the future; use --nsInclude instead
2020-03-29T16:40:56.954+0800    building a list of collections to restore from C:\0-d\wx_backup\20200329\test_accessLog dir
2020-03-29T16:40:56.971+0800    reading metadata for test_accessLog.accessLog from C:\0-d\wx_backup\20200329\test_accessLog\accessLog.metadata.json
2020-03-29T16:40:56.984+0800    restoring test_accessLog.accessLog from C:\0-d\wx_backup\20200329\test_accessLog\accessLog.bson
2020-03-29T16:40:56.985+0800    restoring indexes for collection test_accessLog.accessLog from metadata
2020-03-29T16:40:57.006+0800    finished restoring test_accessLog.accessLog (0 documents, 0 failures)
2020-03-29T16:40:57.006+0800    0 document(s) restored successfully. 0 document(s) failed to restore.

C:\Program Files\MongoDB\Server\4.2\bin>mongorestore -d test_bibliosummary --drop  C:\0-d\wx_backup\20200329\test_bibliosummary
2020-03-29T16:40:57.641+0800    the --db and --collection args should only be used when restoring from a BSON file. Other uses are deprecated and will not exist in the future; use --nsInclude instead
2020-03-29T16:40:57.700+0800    building a list of collections to restore from C:\0-d\wx_backup\20200329\test_bibliosummary dir
2020-03-29T16:40:57.717+0800    reading metadata for test_bibliosummary.summary from C:\0-d\wx_backup\20200329\test_bibliosummary\summary.metadata.json
2020-03-29T16:40:57.730+0800    restoring test_bibliosummary.summary from C:\0-d\wx_backup\20200329\test_bibliosummary\summary.bson
2020-03-29T16:40:57.731+0800    restoring indexes for collection test_bibliosummary.summary from metadata
2020-03-29T16:40:57.776+0800    finished restoring test_bibliosummary.summary (1 document, 0 failures)
2020-03-29T16:40:57.776+0800    1 document(s) restored successfully. 0 document(s) failed to restore.

C:\Program Files\MongoDB\Server\4.2\bin>mongorestore -d test_chargingOper --drop C:\0-d\wx_backup\20200329\test_chargingOper
2020-03-29T16:40:58.414+0800    the --db and --collection args should only be used when restoring from a BSON file. Other uses are deprecated and will not exist in the future; use --nsInclude instead
2020-03-29T16:40:58.479+0800    building a list of collections to restore from C:\0-d\wx_backup\20200329\test_chargingOper dir
2020-03-29T16:40:58.510+0800    reading metadata for test_chargingOper.collection from C:\0-d\wx_backup\20200329\test_chargingOper\collection.metadata.json
2020-03-29T16:40:58.531+0800    restoring test_chargingOper.collection from C:\0-d\wx_backup\20200329\test_chargingOper\collection.bson
2020-03-29T16:40:58.536+0800    restoring indexes for collection test_chargingOper.collection from metadata
2020-03-29T16:40:58.667+0800    finished restoring test_chargingOper.collection (6 documents, 0 failures)
2020-03-29T16:40:58.667+0800    6 document(s) restored successfully. 0 document(s) failed to restore.

C:\Program Files\MongoDB\Server\4.2\bin>mongorestore -d test_hitcount --drop C:\0-d\wx_backup\20200329\test_hitcount
2020-03-29T16:41:01.494+0800    the --db and --collection args should only be used when restoring from a BSON file. Other uses are deprecated and will not exist in the future; use --nsInclude instead
2020-03-29T16:41:01.541+0800    building a list of collections to restore from C:\0-d\wx_backup\20200329\test_hitcount dir
2020-03-29T16:41:01.557+0800    reading metadata for test_hitcount.hitcount from C:\0-d\wx_backup\20200329\test_hitcount\hitcount.metadata.json
2020-03-29T16:41:01.569+0800    restoring test_hitcount.hitcount from C:\0-d\wx_backup\20200329\test_hitcount\hitcount.bson
2020-03-29T16:41:01.583+0800    restoring indexes for collection test_hitcount.hitcount from metadata
2020-03-29T16:41:01.614+0800    finished restoring test_hitcount.hitcount (0 documents, 0 failures)
2020-03-29T16:41:01.614+0800    0 document(s) restored successfully. 0 document(s) failed to restore.
  1. 再用内务登录dp2library，检查读者的借阅历史，看到有了信息。