search

crazyjohn / crazyjohn.github.io

crazyjohn's blog
9 stars 3 forks source link

mongodb manual #36

Open crazyjohn opened 8 years ago

crazyjohn commented 8 years ago

1. 基本操作

  1. show dbs 展示当前所有的数据库信息。

    > show dbs
admin       (empty)
local       0.078GB
stupidbird  0.078GB
  2. use dbName。创建/使用指定数据库。

  3. db。展示当前使用的db。

    > db
stupidbird
  4. 可以在指定的数据库中创建多个文档，然后去操作指定的文档。
  5. db.{dbName}.{docName}.find()。查看文档中的所有数据。

  6. db.{dbName}.{docName}.insert()。向文档中插入数据。

    > db.stupidbird.loginData.insert({"puid" : "bot1"})
WriteResult({ "nInserted" : 1 })

  7. 查看下插入结果。

    > db.stupidbird.loginData.find()
{ "_id" : ObjectId("567b55e4991e9066b69255b6"), "puid" : "crazyjohn", "loginTimes" : 6, "lastLogin" : ISODate("2015-12-24T02:26:45.756Z"), "rewards" : [ "first", "second" ] }
{ "_id" : ObjectId("567b5d58991e9066b69255b7"), "puid" : "bot1" }

  8. 更新指定的登陆数据：插入loginTimes属性，使用$inc修改器。此种状况下如果指定属性不存在会插入。

    > db.stupidbird.loginData.update({"puid" : "bot1"}, {"$inc" : {"loginTimes" : 1}})
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })

  9. 查看下插入效果。

    > db.stupidbird.loginData.find()
{ "_id" : ObjectId("567b55e4991e9066b69255b6"), "puid" : "crazyjohn", "loginTimes" : 6, "lastLogin" : ISODate("2015-12-24T02:26:45.756Z"), "rewards" : [ "first", "second" ] }
{ "_id" : ObjectId("567b5d58991e9066b69255b7"), "puid" : "bot1", "loginTimes" : 1 }

  10. 使用$set修改器更新。

    > db.stupidbird.loginData.update({"puid" : "bot1"}, {"$set" : {"lastLogin" : new Date()}})
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
> db.stupidbird.loginData.find()
{ "_id" : ObjectId("567b55e4991e9066b69255b6"), "puid" : "crazyjohn", "loginTimes" : 6, "lastLogin" : ISODate("2015-12-24T02:26:45.756Z"), "rewards" : [ "first", "second" ] }
{ "_id" : ObjectId("567b5d58991e9066b69255b7"), "puid" : "bot1", "loginTimes" : 1, "lastLogin" : ISODate("2015-12-24T03:05:46.535Z") }

  11. $push数组修改器。

    > db.stupidbird.loginData.update({"puid" : "bot1"}, {"$push" : {"rewards" : "first"}})
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
> db.stupidbird.loginData.find()
{ "_id" : ObjectId("567b55e4991e9066b69255b6"), "puid" : "crazyjohn", "loginTimes" : 6, "lastLogin" : ISODate("2015-12-24T02:26:45.756Z"), "rewards" : [ "first", "second" ] }
{ "_id" : ObjectId("567b5d58991e9066b69255b7"), "puid" : "bot1", "loginTimes" : 1, "lastLogin" : ISODate("2015-12-24T03:05:46.535Z"), "rewards" : [ "first" ] }
> db.stupidbird.loginData.update({"puid" : "bot1"}, {"$push" : {"rewards" : "third"}})
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
> db.stupidbird.loginData.find()
{ "_id" : ObjectId("567b55e4991e9066b69255b6"), "puid" : "crazyjohn", "loginTimes" : 6, "lastLogin" : ISODate("2015-12-24T02:26:45.756Z"), "rewards" : [ "first", "second" ] }
{ "_id" : ObjectId("567b5d58991e9066b69255b7"), "puid" : "bot1", "loginTimes" : 1, "lastLogin" : ISODate("2015-12-24T03:05:46.535Z"), "rewards" : [ "first", "third" ] }

  12. $pop删除指定的数组元素。

    > db.stupidbird.loginData.update({"puid" : "bot1"}, {"$pop" : {"rewards" : 1}})
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
> db.stupidbird.loginData.find()
{ "_id" : ObjectId("567b55e4991e9066b69255b6"), "puid" : "crazyjohn", "loginTimes" : 6, "lastLogin" : ISODate("2015-12-24T02:26:45.756Z"), "rewards" : [ "first", "second" ] }
{ "_id" : ObjectId("567b5d58991e9066b69255b7"), "puid" : "bot1", "loginTimes" : 1, "lastLogin" : ISODate("2015-12-24T03:05:46.535Z"), "rewards" : [ "first" ] }

    2. mongostat

insert  query update delete getmore command flushes mapped  vsize    res faults       locked db idx miss %     qr|qw   ar|aw  netIn netOut  conn       time
    *0     *0     *0     *0       0     0|0       0  1.03g   2.2g   523m      0 stupidbird:0.0%          0       0|0     0|0    20b     1k     2   16:34:47
    *0     *0     *0     *0       0     0|0       0  1.03g   2.2g   523m      0 stupidbird:0.0%          0       0|0     0|0    20b     1k     2   16:34:50
    *0     *0     *0     *0       0     0|0       0  1.03g   2.2g   523m      0 stupidbird:0.0%          0       0|0     0|0    20b     1k     2   16:34:53
    *0     *0     *0     *0       0     0|0       0  1.03g   2.2g   523m      0 stupidbird:0.0%          0       0|0     0|0    20b     1k     2   16:34:56
    *0     *0     *0     *0       0     0|0       0  1.03g   2.2g   523m      0 stupidbird:0.0%          0       0|0     0|0    20b     1k     2   16:34:59
    *0     *0     *0     *0       0     0|0       0  1.03g   2.2g   523m      0          .:0.0%          0       0|0     0|0    20b     1k     2   16:35:02
    *0      1     *0     *0       0     0|0       0  1.03g   2.2g   523m      0 stupidbird:0.0%          0       0|0     0|0    20b     1k     2   16:35:05
    *0     *0     *0     *0       0     0|0       0  1.03g   2.2g   523m      0 stupidbird:0.0%          0       0|0     0|0    20b     1k     2   16:35:08
    *0     *0     *0     *0       0     0|0       0  1.03g   2.2g   523m      0 stupidbird:0.0%          0       0|0     0|0    20b     1k     2   16:35:11
    *0     *0     *0     *0       0     0|0       0  1.03g   2.2g   523m      0 stupidbird:0.0%          0       0|0     0|0    20b     1k     2   16:35:14
insert  query update delete getmore command flushes mapped  vsize    res faults       locked db idx miss %     qr|qw   ar|aw  netIn netOut  conn       time
   775     *0     *0     *0       0     1|0       0  1.03g  2.21g   526m    991 stupidbird:2.0%          0       0|0     0|0   317k    44k     4   16:35:17
  6157     *0     *0     *0       0     0|0       0  1.03g  2.21g   534m   7638 stupidbird:11.6%          0       0|0     0|0     2m   345k     4   16:35:20
  6588     *0     *0     *0       0     1|0       0  1.03g  2.21g   543m   8179 stupidbird:15.1%          0       0|0     0|0     2m   370k     4   16:35:23
  7733     *0     *0     *0       0     0|0       0  1.03g  2.21g   552m   9693 stupidbird:20.8%          0       1|0     0|1     3m   434k     4   16:35:26
  7914     *0     *0     *0       0     1|0       0  1.03g  2.21g   565m   9878 stupidbird:16.8%          0       0|0     0|1     3m   444k     4   16:35:29
  7843     *0     *0     *0       0     1|0       0  1.03g  2.21g   575m   9781 stupidbird:19.7%          0       0|0     0|0     3m   440k     4   16:35:32
  7978     *0     *0     *0       0     0|0       0  1.03g  2.21g   585m   9945 stupidbird:17.6%          0       0|0     0|0     3m   447k     4   16:35:35
  8039     *0     *0     *0       0     1|0       0  1.03g  2.21g   598m   9978 stupidbird:16.0%          0       0|0     0|0     3m   451k     4   16:35:38
  7685     *0     *0     *0       0     0|0       0  1.03g  2.21g   607m   9571 stupidbird:19.0%          0       0|0     0|0     3m   431k     4   16:35:41
  7983     *0     *0     *0       0     1|0       0  1.03g  2.21g   617m   9944 stupidbird:16.4%          0       0|0     0|0     3m   448k     4   16:35:44

insert  query update delete getmore command flushes mapped  vsize    res faults         locked db idx miss %     qr|qw   ar|aw  netIn netOut  conn       time
  7682     *0     *0     *0       0     0|0       0  2.03g  4.21g  1.02g   9531  stupidbird:18.8%          0       0|0     0|0     3m   431k     4   16:37:56
  7318     *0     *0     *0       0     1|0       0  2.03g  4.21g  1.03g   9117  stupidbird:20.3%          0       0|0     0|1     2m   411k     4   16:37:59
  6380     *0     *0     *0       0     1|0       0  2.03g  4.21g  1.04g   7935  stupidbird:29.6%          0       0|0     0|1     2m   358k     4   16:38:02
  4555      1     *0     *0       0     0|0       0  2.03g  4.21g  1.04g   5725  stupidbird:41.5%          0       1|0     0|1     1m   256k     4   16:38:05
  4342     *0     *0     *0       0     1|0       0  2.03g  4.21g  1.05g   5445  stupidbird:55.7%          0       0|0     0|0     1m   244k     4   16:38:08
  5841     *0     *0     *0       0     0|0       0  2.03g  4.21g  1.06g   7252  stupidbird:34.2%          0       0|0     0|1     2m   328k     4   16:38:11
  7237     *0     *0     *0       0     1|0       0  2.03g  4.21g  1.07g   9035  stupidbird:22.1%          0       0|0     0|1     2m   406k     4   16:38:14
  8170     *0     *0     *0       0     1|0       0  2.03g  4.21g  1.07g  10209  stupidbird:14.6%          0       0|0     0|0     3m   458k     4   16:38:17
  5109     *0     *0     *0       0     0|0       0  2.03g  4.21g  1.08g   6314  stupidbird:36.7%          0       0|0     0|0     2m   287k     4   16:38:20
  7363     *0     *0     *0       0     1|0       0  2.03g  4.21g  1.09g   9193  stupidbird:20.0%          0       0|0     0|0     3m   413k     4   16:38:23

insert  query update delete getmore command flushes mapped  vsize    res faults         locked db idx miss %     qr|qw   ar|aw  netIn netOut  conn       time
  8202     *0     *0     *0       0     1|0       0  2.03g  4.21g  1.51g  10248  stupidbird:15.2%          0       0|0     0|0     3m   460k     4   16:40:27
  7676     *0     *0     *0       0     0|0       0  2.03g  4.21g  1.52g   9605  stupidbird:19.1%          0       0|0     0|0     3m   430k     4   16:40:30
  8291     *0     *0     *0       0     1|0       0  2.03g  4.21g  1.53g  10283  stupidbird:15.6%          0       0|0     0|0     3m   465k     4   16:40:33
  8287     *0     *0     *0       0     0|0       0  2.03g  4.21g  1.54g  10350  stupidbird:15.4%          0       0|0     0|0     3m   465k     4   16:40:36
  8259     *0     *0     *0       0     1|0       0  2.03g  4.21g  1.55g  10307  stupidbird:14.9%          0       0|0     0|1     3m   463k     4   16:40:39
  8045     *0     *0     *0       0     1|0       0  2.03g  4.21g  1.56g   9993  stupidbird:17.5%          0       0|0     0|0     3m   451k     4   16:40:42
  8221     *0     *0     *0       0     0|0       0  2.03g  4.21g  1.57g  10275  stupidbird:15.9%          0       0|0     0|0     3m   461k     4   16:40:45
  8092     *0     *0     *0       0     1|0       0  2.03g  4.21g  1.58g  10135  stupidbird:17.0%          0       0|0     0|0     3m   454k     4   16:40:48
  7817     *0     *0     *0       0     0|0       0  2.03g  4.21g  1.59g   9701  stupidbird:18.4%          0       0|0     0|0     3m   438k     4   16:40:51
  8302     *0     *0     *0       0     1|0       0  2.03g  4.21g   1.6g  10367  stupidbird:14.7%          0       0|0     0|0     3m   466k     4   16:40:54

insert  query update delete getmore command flushes mapped  vsize    res faults         locked db idx miss %     qr|qw   ar|aw  netIn netOut  conn       time
  7881     *0     *0     *0       0     0|0       0  4.03g   8.2g     2g   9856  stupidbird:17.5%          0       0|0     0|0     3m   442k     4   16:43:51
  7545     *0     *0     *0       0     1|0       0  4.03g   8.2g     2g   9423  stupidbird:19.0%          0       0|0     0|0     3m   423k     4   16:43:54
  7967     *0     *0     *0       0     0|0       0  4.03g   8.2g  2.02g   9902  stupidbird:16.9%          0       0|0     0|0     3m   447k     4   16:43:57
  8038     *0     *0     *0       0     1|0       0  4.03g   8.2g  2.03g  10040  stupidbird:16.4%          0       0|0     0|0     3m   451k     4   16:44:00
  7616     *0     *0     *0       0     1|0       0  4.03g   8.2g  2.04g   9513  stupidbird:19.6%          0       0|0     0|0     3m   427k     4   16:44:03
  4849     *0     *0     *0       0     0|0       0  4.03g   8.2g  2.04g   6023  stupidbird:47.8%          0       0|0     0|1     1m   272k     4   16:44:06
  7216     *0     *0     *0       0     1|0       0  4.03g   8.2g  2.05g   9003  stupidbird:26.1%          0       0|0     0|0     2m   405k     4   16:44:09
  7726     *0     *0     *0       0     0|0       0  4.03g   8.2g  2.06g   9564  stupidbird:19.0%          0       0|0     0|0     3m   433k     4   16:44:12
  7785     *0     *0     *0       0     1|0       0  4.03g   8.2g  2.07g   9703  stupidbird:18.5%          0       0|0     0|0     3m   437k     4   16:44:15
  7915      1     *0     *0       0     1|0       0  4.03g   8.2g  2.08g   9877  stupidbird:17.4%          0       0|0     0|0     3m   444k     4   16:44:18

insert  query update delete getmore command flushes mapped  vsize    res faults         locked db idx miss %     qr|qw   ar|aw  netIn netOut  conn       time
  7360     *0     *0     *0       0     0|0       0  4.03g   8.2g  2.42g   9241  stupidbird:22.0%          0       0|0     0|0     3m   413k     4   16:46:52
  6925     *0     *0     *0       0     1|0       0  4.03g   8.2g  2.41g   8681  stupidbird:26.2%          0       0|0     0|1     2m   389k     4   16:46:55
  7469     *0     *0     *0       0     1|0       0  4.03g   8.2g  2.42g   9356  stupidbird:21.3%          0       0|0     0|1     3m   419k     4   16:46:58
  1084     *0     *0     *0       0     0|0       0  4.03g   8.2g   2.4g   1350   stupidbird:4.0%          0       0|0     0|0   444k    61k     2   16:47:01
    *0     *0     *0     *0       0     0|0       0  4.03g   8.2g   2.4g      0   stupidbird:0.0%          0       0|0     0|0    20b     1k     2   16:47:04
    *0     *0     *0     *0       0     0|0       0  4.03g   8.2g   2.4g      0        local:0.0%          0       0|0     0|0    20b     1k     2   16:47:07
    *0     *0     *0     *0       0     0|0       0  4.03g   8.2g   2.4g      0   stupidbird:0.0%          0       0|0     0|0    20b     1k     2   16:47:10
    *0     *0     *0     *0       0     0|0       0  4.03g   8.2g   2.4g      0   stupidbird:0.0%          0       0|0     0|0    20b     1k     2   16:47:13
    *0      1     *0     *0       0     0|0       0  4.03g   8.2g   2.4g      2   stupidbird:0.0%          0       0|0     0|0    20b     1k     2   16:47:16
    *0     *0     *0     *0       0     0|0       0  4.03g   8.2g   2.4g      0   stupidbird:0.0%          0       0|0     0|0    20b     1k     2   16:47:19
insert  query update delete getmore command flushes mapped  vsize    res faults         locked db idx miss %     qr|qw   ar|aw  netIn netOut  conn       time
    *0     *0     *0     *0       0     0|0       0  4.03g   8.2g   2.4g      0            .:0.0%          0       0|0     0|0    20b     1k     2   16:47:22
    *0     *0     *0     *0       0     0|0       0  4.03g   8.2g   2.4g      0   stupidbird:0.0%          0       0|0     0|0    20b     1k     2   16:47:25
^C
D:\soft\mongodb\bin>

3. mongodb sharding

分片的三个组成部分：分片（分摊负载的真实服务器），monogos(请求路由器)，配置服务器（存储集群的配置信息）。

以下是操作步奏：

  1. 启动服务器
    1. 配置服务器启动。先构建好配置数据目录，比如:shard/config，然后启动配置服务器： mongod --dbpath ../shard/config --port 20000。（mongo3.2+以后要使用--configsvr启动参数，否则它将不被认为是一台config server）
    2. 启动mongos路由器。不需要构建数据目录。直接命令:mongos -- port 30000 --configdb localhost:20000。且连接到配置服务器。
    3. 添加分片1。输入命令： mongod --dbpath ../shard/shard1 --port 10000。
    4. 用mongo客户端连接到路由器。命令: mongo localhost:30000/admin.
    5. 使用addshard命令添加分片。执行： db.runCommand({addshard : "localhost:10000", allowLocal : true})
  2. 切分数据。
    1. 比如我想切分数据库stupidbird中的PlayerEntity集合。命令： db.runCommand({"enablesharding" : "stupidbird"})。
    2. 对内部集合进行分片。命令： db.runCommand({"shardcollection" : "stupidbird.PlayerEntity", "key" : {"_id" : 1}})。
  3. 管理分片。
    1. 查看shards集合中的所有的片。命令：db.shards.find()。
    2. 数据库信息。命令： db.databases.find()。
    3. 块。卡卡那数据到底是如何切分为集群的。命令: db.chunks.find()。

      4. mongo java orm - Morphia

      5. mongodb storage engines

  4. MMAPv1 存储引擎。

    它是mongo最原始的存储引擎，基于内存映射文件，它擅长高负载的插入，读以及原位更新。在mongodb3.2版本以后这个引擎就不再是默认的存储引擎了，取而代之的是WiredTiger引擎。

    Journal(日志)：为了确认所有的mongodb的更改集都已经写到了硬盘里，mongo默认会把所有记录的更改都记录到一个硬盘日志里。相对于写数据文件，mongo会更频繁的去写日志文件。日志文件可以确保Mongod在没有刷新所有更改就退出的时候，可以从数据文件中恢复过来。

    record storage characteristics: 所有的数据记录都连续的存放在硬盘上，当一条记录的大小超过分配的空间的时候，mongo需要给这条记录分配新的空间。这个新的分配行为需要mongo移动指定文档并且更新所有引用此文档的索引，这不但会消耗比原位更新更耗时，而且会导致存储碎片。

    mongdb3.0.0以后的变化：

    6. create index

给puid添加索引:

> db.PlayerEntity.ensureIndex({puid : 1})
{
        "createdCollectionAutomatically" : false,
        "numIndexesBefore" : 1,
        "numIndexesAfter" : 2,
        "ok" : 1
}

server端日志：

2015-12-29T18:14:12.006+0800 [conn3]            Index Build: 85481000/116111000 73%
2015-12-29T18:14:30.012+0800 [conn3]            Index Build: 86743900/116111000 74%
2015-12-29T18:14:42.461+0800 [conn3]            Index Build: 87246000/116111000 75%
2015-12-29T18:14:45.004+0800 [PeriodicTaskRunner] task: DBConnectionPool-cleaner took: 120ms
2015-12-29T18:14:45.160+0800 [PeriodicTaskRunner] task: WriteBackManager::cleaner took: 135ms
2015-12-29T18:14:45.347+0800 [conn3]            Index Build: 87260300/116111000 75%
2015-12-29T18:15:03.009+0800 [conn3]            Index Build: 89232700/116111000 76%
2015-12-29T18:15:31.001+0800 [conn3]            Index Build: 90475300/116111000 77%
2015-12-29T18:15:34.011+0800 [conn3]            Index Build: 90567000/116111000 78%
2015-12-29T18:16:05.013+0800 [conn3]            Index Build: 92752500/116111000 79%
2015-12-29T18:19:17.069+0800 [conn3]            Index Build: 99072300/116111000 85%
2015-12-29T18:19:20.004+0800 [conn3]            Index Build: 99098700/116111000 85%
2015-12-29T18:19:23.026+0800 [conn3]            Index Build: 99132500/116111000 85%
2015-12-29T18:19:26.001+0800 [conn3]            Index Build: 99169300/116111000 85%
2015-12-29T18:19:29.014+0800 [conn3]            Index Build: 99224800/116111000 85%
2015-12-29T18:19:32.080+0800 [conn3]            Index Build: 99319000/116111000 85%
2015-12-29T18:19:35.146+0800 [conn3]            Index Build: 99325500/116111000 85%
2015-12-29T18:20:05.005+0800 [conn3]            Index Build: 100095000/116111000        86%
2015-12-29T18:20:45.008+0800 [conn3]            Index Build: 104939400/116111000        90%
2015-12-29T18:22:12.008+0800 [conn3]            Index Build: 115799400/116111000        99%
2015-12-29T18:22:15.533+0800 [conn3]            Index Build: 115910600/116111000        99%
2015-12-29T18:22:29.029+0800 [conn3]            Index: (2/3) BTree Bottom Up Progress: 2231400/116111000        1%
2015-12-29T18:22:39.000+0800 [conn3]            Index: (2/3) BTree Bottom Up Progress: 5118500/116111000        4%
2015-12-29T18:22:45.256+0800 [PeriodicTaskRunner] task: DBConnectionPool-cleaner took: 10ms
2015-12-29T18:22:49.046+0800 [conn3]            Index: (2/3) BTree Bottom Up Progress: 8691600/116111000        7%
2015-12-29T18:22:59.000+0800 [conn3]            Index: (2/3) BTree Bottom Up Progress: 13255000/116111000       11%
2015-12-29T18:23:09.002+0800 [conn3]            Index: (2/3) BTree Bottom Up Progress: 18342800/116111000       15%
2015-12-29T18:23:19.002+0800 [conn3]            Index: (2/3) BTree Bottom Up Progress: 23089800/116111000       19%
2015-12-29T18:23:29.252+0800 [conn3]            Index: (2/3) BTree Bottom Up Progress: 26212200/116111000       22%
2015-12-29T18:23:34.807+0800 [FileAllocator] allocating new datafile ../data/db\stupidbird.31, filling with zeroes...
2015-12-29T18:24:34.873+0800 [FileAllocator] done allocating datafile ../data/db\stupidbird.31, size: 2047MB,  took 58.149 secs
2015-12-29T18:24:36.012+0800 [conn3] ExtentManager took 61 seconds to open: ../data/db\stupidbird.31
2015-12-29T18:24:36.106+0800 [conn3]            Index: (2/3) BTree Bottom Up Progress: 28854600/116111000       24%
2015-12-29T18:24:47.933+0800 [conn3]            Index: (2/3) BTree Bottom Up Progress: 28956600/116111000       24%       70%
2015-12-29T18:29:00.009+0800 [conn3]            Index: (2/3) BTree Bottom Up Progress: 83253200/116111000       71%
2015-12-29T18:29:10.195+0800 [conn3]            Index: (2/3) BTree Bottom Up Progress: 83855600/116111000       72%
2015-12-29T18:29:20.350+0800 [conn3]            Index: (2/3) BTree Bottom Up Progress: 84551200/116111000       72%
2015-12-29T18:29:30.022+0800 [conn3]            Index: (2/3) BTree Bottom Up Progress: 85159500/116111000       73%
2015-12-29T18:29:46.715+0800 [conn3]            Index: (2/3) BTree Bottom Up Progress: 86054000/116111000       74%
2015-12-29T18:29:56.080+0800 [conn3]            Index: (2/3) BTree Bottom Up Progress: 86568400/116111000       74%
2015-12-29T18:30:06.017+0800 [conn3]            Index: (2/3) BTree Bottom Up Progress: 87104200/116111000       75%
2015-12-29T18:30:16.000+0800 [conn3]            Index: (2/3) BTree Bottom Up Progress: 87649000/116111000       75%
2015-12-29T18:30:26.008+0800 [conn3]            Index: (2/3) BTree Bottom Up Progress: 90416200/116111000       77%
2015-12-29T18:30:36.101+0800 [conn3]            Index: (2/3) BTree Bottom Up Progress: 93183000/116111000       80%
2015-12-29T18:30:46.007+0800 [conn3]            Index: (2/3) BTree Bottom Up Progress: 93778600/116111000       80%
2015-12-29T18:30:56.008+0800 [conn3]            Index: (2/3) BTree Bottom Up Progress: 94635700/116111000       81%
2015-12-29T18:31:03.013+0800 [FileAllocator] allocating new datafile ../data/db\stupidbird.32, filling with zeroes...
2015-12-29T18:31:14.354+0800 [FileAllocator] done allocating datafile ../data/db\stupidbird.32, size: 2047MB,  took 11.332 secs
2015-12-29T18:31:14.603+0800 [conn3] ExtentManager took 11 seconds to open: ../data/db\stupidbird.32
2015-12-29T18:31:14.681+0800 [conn3]            Index: (2/3) BTree Bottom Up Progress: 99073400/116111000       85%
2015-12-29T18:31:24.012+0800 [conn3]            Index: (2/3) BTree Bottom Up Progress: 100721700/116111000      86%
2015-12-29T18:31:34.002+0800 [conn3]            Index: (2/3) BTree Bottom Up Progress: 104827600/116111000      90%
2015-12-29T18:31:44.002+0800 [conn3]            Index: (2/3) BTree Bottom Up Progress: 109030000/116111000      93%
2015-12-29T18:31:54.003+0800 [conn3]            Index: (2/3) BTree Bottom Up Progress: 113012300/116111000      97%
2015-12-29T18:32:01.821+0800 [conn3]     done building bottom layer, going to commit
2015-12-29T18:32:54.583+0800 [DataFileSync] flushing mmaps took 11623ms  for 38 files
2015-12-29T18:33:49.887+0800 [conn3] old journal file will be removed: ../data/db\journal\j._0
2015-12-29T18:33:49.923+0800 [conn3] old journal file will be removed: ../data/db\journal\j._1
2015-12-29T18:34:32.237+0800 [conn3] build index done.  scanned 116111000 total records. 2829.45 secs
2015-12-29T18:34:37.002+0800 [conn3] command stupidbird.$cmd command: createIndexes { createIndexes: "PlayerEntity", indexes: [ { key: { puid: 1.0 }, name: "puid_1" } ] } keyUpdates:0 numYields:0 locks(micros) r:77 w:2834141467 reslen:113 2834252ms
2015-12-29T18:34:37.323+0800 [TTLMonitor] query stupidbird.system.indexes query: { expireAfterSeconds: { $exists: true } } planSummary: COLLSCAN ntoreturn:0 ntoskip:0 nscanned:2 nscannedObjects:2 keyUpdates:0 numYields:0 locks(micros) r:282994 nreturned:0 reslen:20 276ms
2015-12-29T18:34:37.440+0800 [conn4] assertion 11601 operation was interrupted ns:stupidbird.PlayerEntity query:{ query: { _id: ObjectId('567ba72e4db30b7022860b7b') }, $explain: true }
2015-12-29T18:34:37.440+0800 [conn4] query stupidbird.PlayerEntity query: { query: { _id: ObjectId('567ba72e4db30b7022860b7b') }, $explain: true } ntoreturn:0 keyUpdates:0 exception: operation was interrupted code:11601 numYields:0 locks(micros) r:362299 reslen:71 366ms
2015-12-29T18:34:37.477+0800 [conn4] SocketException handling request, closing client connection: 9001 socket exception [SEND_ERROR] server [127.0.0.1:2921]

查看查询计划：

> db.PlayerEntity.find({puid : "bot100"}).explain()
{
        "cursor" : "BtreeCursor puid_1",
        "isMultiKey" : false,
        "n" : 1,
        "nscannedObjects" : 1,
        "nscanned" : 1,
        "nscannedObjectsAllPlans" : 1,
        "nscannedAllPlans" : 1,
        "scanAndOrder" : false,
        "indexOnly" : false,
        "nYields" : 1,
        "nChunkSkips" : 0,
        "millis" : 49,
        "indexBounds" : {
                "puid" : [
                        [
                                "bot100",
                                "bot100"
                        ]
                ]
        },
        "server" : "X6X8-20141209SF:27017",
        "filterSet" : false
}

MBD我执行了一个无索引的查询（基于id，数据量级在100000000）, 内存完全被mongod吃光了尼玛：

> db.PlayerEntity.find({puid : "bot100"}).explain()
{
        "cursor" : "BtreeCursor puid_1",
        "isMultiKey" : false,
        "n" : 1,
        "nscannedObjects" : 1,
        "nscanned" : 1,
        "nscannedObjectsAllPlans" : 1,
        "nscannedAllPlans" : 1,
        "scanAndOrder" : false,
        "indexOnly" : false,
        "nYields" : 1,
        "nChunkSkips" : 0,
        "millis" : 49,
        "indexBounds" : {
                "puid" : [
                        [
                                "bot100",
                                "bot100"
                        ]
                ]
        },
        "server" : "X6X8-20141209SF:27017",
        "filterSet" : false
}

上头这货就导致服务器卡死了。然后看下当前的操作列表：

> db.currentOp()
{
        "inprog" : [
                {
                        "opid" : 190,
                        "active" : true,
                        "secs_running" : 167,
                        "microsecs_running" : NumberLong(167405629),
                        "op" : "query",
                        "ns" : "stupidbird.PlayerEntity",
                        "query" : {
                                "query" : {
                                        "id" : 100
                                },
                                "$explain" : true
                        },
                        "planSummary" : "COLLSCAN",
                        "client" : "127.0.0.1:2760",
                        "desc" : "conn3",
                        "connectionId" : 3,
                        "locks" : {
                                "^" : "r",
                                "^stupidbird" : "R"
                        },
                        "waitingForLock" : false,
                        "numYields" : 1197283,
                        "lockStats" : {
                                "timeLockedMicros" : {
                                        "r" : NumberLong(223789364),
                                        "w" : NumberLong(0)
                                },
                                "timeAcquiringMicros" : {
                                        "r" : NumberLong(335161),
                                        "w" : NumberLong(0)
                                }
                        }
                }
        ]
}

杀掉指定操作id:

> db.killOp(190)
{ "info" : "attempting to kill op" }
> db.currentOp()
{ "inprog" : [ ] }

7. shard key

在使用分片的时候如何选择合适的分片键值呢：

首先mongodb在分片平衡(balance)的时候使用的是在片集之间的块移动，而块(chunk)又可以预设大小，比如200M，在块超过这个大小的时候就会根据shard key进行分裂。

举例我们以一个自增的{playerId : 1}作为分片键，假设playerId到了100000的时候chunk的块到了上限200M，那么分片的块就会由原来的（-max, max），分裂为2个块，片集分别为：（-max, 50000）,[50000, max)。

8. jurnal

插入时候服务器端的日志输出：

conn16] insert stupidbird.PlayerEntity query: { _id: ObjectId('568372114db37416080af89d'), className: "com.stupidbird.game.entity.PlayerEntity", id: 19295445, puid: "bot19295445" } ninserted:1 keyUpdates:0 numYields:0 locks(micros) w:239770 239ms
conn16] command stupidbird.$cmd command: insert { insert: "PlayerEntity", ordered: true, documents: [ { _id: ObjectId('568372114db37416080af89d'), className: "com.stupidbird.game.entity.PlayerEntity", id: 19295445, puid: "bot19295445" } ] } keyUpdates:0 numYields:0 locks(micros) w:14498 reslen:40 254ms
journal] old journal file will be removed: ../data/db\journal\j._3
conn16] insert stupidbird.PlayerEntity query: { _id: ObjectId('568372364db37416080face2'), className: "com.stupidbird.game.entity.PlayerEntity", id: 19603738, puid: "bot19603738" } ninserted:1 keyUpdates:0 numYields:0 locks(micros) w:117430 117ms
conn16] command stupidbird.$cmd command: insert { insert: "PlayerEntity", ordered: true, documents: [ { _id: ObjectId('568372364db37416080face2'), className: "com.stupidbird.game.entity.PlayerEntity", id: 19603738, puid: "bot19603738" } ] } keyUpdates:0 numYields:0 locks(micros) w:698 reslen:40 118ms
initandlisten] connection accepted from 127.0.0.1:11149 #17 (7 connections now open)
conn17] command admin.$cmd command: serverStatus { serverStatus: 1 } ntoreturn:1 keyUpdates:0 numYields:0 locks(micros) r:15 reslen:3402 115ms
conn16] insert stupidbird.PlayerEntity query: { _id: ObjectId('568372394db37416081014ae'), className: "com.stupidbird.game.entity.PlayerEntity", id: 19630310, puid: "bot19630310" } ninserted:1 keyUpdates:0 numYields:0 locks(micros) w:247573 247ms
conn16] command stupidbird.$cmd command: insert { insert: "PlayerEntity", ordered: true, documents: [ { _id: ObjectId('568372394db37416081014ae'), className: "com.stupidbird.game.entity.PlayerEntity", id: 19630310, puid: "bot19630310" } ] } keyUpdates:0 numYields:0 locks(micros) w:676 reslen:40 248ms
conn16] insert stupidbird.PlayerEntity query: { _id: ObjectId('5683723a4db37416081014af'), className: "com.stupidbird.game.entity.PlayerEntity", id: 19630311, puid: "bot19630311" } ninserted:1 keyUpdates:0 numYields:0 locks(micros) w:114423 114ms
conn16] command stupidbird.$cmd command: insert { insert: "PlayerEntity", ordered: true, documents: [ { _id: ObjectId('5683723a4db37416081014af'), className: "com.stupidbird.game.entity.PlayerEntity", id: 19630311, puid: "bot19630311" } ] } keyUpdates:0 numYields:0 locks(micros) w:731 reslen:40 115ms
conn16] insert stupidbird.PlayerEntity query: { _id: ObjectId('5683723d4db3741608107e0f'), className: "com.stupidbird.game.entity.PlayerEntity", id: 19657287, puid: "bot19657287" } ninserted:1 keyUpdates:0 numYields:0 locks(micros) w:109101 110ms
conn16] command stupidbird.$cmd command: insert { insert: "PlayerEntity", ordered: true, documents: [ { _id: ObjectId('5683723d4db3741608107e0f'), className: "com.stupidbird.game.entity.PlayerEntity", id: 19657287, puid: "bot19657287" } ] } keyUpdates:0 numYields:0 locks(micros) w:645 reslen:40 110ms
conn16] insert stupidbird.PlayerEntity query: { _id: ObjectId('5683723d4db3741608107f6a'), className: "com.stupidbird.game.entity.PlayerEntity", id: 19657634, puid: "bot19657634" } ninserted:1 keyUpdates:0 numYields:0 locks(micros) w:293885 293ms
conn16] command stupidbird.$cmd command: insert { insert: "PlayerEntity", ordered: true, documents: [ { _id: ObjectId('5683723d4db3741608107f6a'), className: "com.stupidbird.game.entity.PlayerEntity", id: 19657634, puid: "bot19657634" } ] } keyUpdates:0 numYields:0 locks(micros) w:728 reslen:40 294ms
conn16] insert stupidbird.PlayerEntity query: { _id: ObjectId('5683723e4db3741608108b46'), className: "com.stupidbird.game.entity.PlayerEntity", id: 19660670, puid: "bot19660670" } ninserted:1 keyUpdates:0 numYields:0 locks(micros) w:189701 189ms
conn16] command stupidbird.$cmd command: insert { insert: "PlayerEntity", ordered: true, documents: [ { _id: ObjectId('5683723e4db3741608108b46'), className: "com.stupidbird.game.entity.PlayerEntity", id: 19660670, puid: "bot19660670" } ] } keyUpdates:0 numYields:0 locks(micros) w:664 reslen:40 190ms
conn17] command admin.$cmd command: serverStatus { serverStatus: 1 } ntoreturn:1 keyUpdates:0 numYields:0 locks(micros) r:34 reslen:3402 141ms
conn16] insert stupidbird.PlayerEntity query: { _id: ObjectId('5683723e4db3741608108bc9'), className: "com.stupidbird.game.entity.PlayerEntity", id: 19660801, puid: "bot19660801" } ninserted:1 keyUpdates:0 numYields:0 locks(micros) w:146417 146ms
conn16] command stupidbird.$cmd command: insert { insert: "PlayerEntity", ordered: true, documents: [ { _id: ObjectId('5683723e4db3741608108bc9'), className: "com.stupidbird.game.entity.PlayerEntity", id: 19660801, puid: "bot19660801" } ] } keyUpdates:0 numYields:0 locks(micros) w:371 reslen:40 146ms
clientcursormon] mem (MB) res:207 virt:119248
clientcursormon]  mapped (incl journal view):118792
clientcursormon]  connections:7
conn16] insert stupidbird.PlayerEntity query: { _id: ObjectId('5683725b4db37416081447bd'), className: "com.stupidbird.game.entity.PlayerEntity", id: 19905525, puid: "bot19905525" } ninserted:1 keyUpdates:0 numYields:0 locks(micros) w:117525 117ms
conn16] command stupidbird.$cmd command: insert { insert: "PlayerEntity", ordered: true, documents: [ { _id: ObjectId('5683725b4db37416081447bd'), className: "com.stupidbird.game.entity.PlayerEntity", id: 19905525, puid: "bot19905525" } ] } keyUpdates:0 numYields:0 locks(micros) w:674 reslen:40 118ms
conn16] insert stupidbird.PlayerEntity query: { _id: ObjectId('568372794db3741608184b66'), className: "com.stupidbird.game.entity.PlayerEntity", id: 20168606, puid: "bot20168606" } ninserted:1 keyUpdates:0 numYields:0 locks(micros) w:105012 105ms
conn16] command stupidbird.$cmd command: insert { insert: "PlayerEntity", ordered: true, documents: [ { _id: ObjectId('568372794db3741608184b66'), className: "com.stupidbird.game.entity.PlayerEntity", id: 20168606, puid: "bot20168606" } ] } keyUpdates:0 numYields:0 locks(micros) w:350 reslen:40 105ms
conn16] insert stupidbird.PlayerEntity query: { _id: ObjectId('568372794db3741608184bbe'), className: "com.stupidbird.game.entity.PlayerEntity", id: 20168694, puid: "bot20168694" } ninserted:1 keyUpdates:0 numYields:0 locks(micros) w:218367 218ms
conn16] command stupidbird.$cmd command: insert { insert: "PlayerEntity", ordered: true, documents: [ { _id: ObjectId('568372794db3741608184bbe'), className: "com.stupidbird.game.entity.PlayerEntity", id: 20168694, puid: "bot20168694" } ] } keyUpdates:0 numYields:0 locks(micros) w:777 reslen:40 219ms
conn16] insert stupidbird.PlayerEntity query: { _id: ObjectId('5683727a4db3741608184fb2'), className: "com.stupidbird.game.entity.PlayerEntity", id: 20169706, puid: "bot20169706" } ninserted:1 keyUpdates:0 numYields:0 locks(micros) w:451014 451ms
conn16] command stupidbird.$cmd command: insert { insert: "PlayerEntity", ordered: true, documents: [ { _id: ObjectId('5683727a4db3741608184fb2'), className: "com.stupidbird.game.entity.PlayerEntity", id: 20169706, puid: "bot20169706" } ] } keyUpdates:0 numYields:0 locks(micros) w:739 reslen:40 451ms
conn16] insert stupidbird.PlayerEntity query: { _id: ObjectId('5683727b4db37416081853a2'), className: "com.stupidbird.game.entity.PlayerEntity", id: 20170714, puid: "bot20170714" } ninserted:1 keyUpdates:0 numYields:0 locks(micros) w:149624 149ms
conn16] command stupidbird.$cmd command: insert { insert: "PlayerEntity", ordered: true, documents: [ { _id: ObjectId('5683727b4db37416081853a2'), className: "com.stupidbird.game.entity.PlayerEntity", id: 20170714, puid: "bot20170714" } ] } keyUpdates:0 numYields:0 locks(micros) w:842 reslen:40 150ms
conn16] insert stupidbird.PlayerEntity query: { _id: ObjectId('5683727b4db37416081856fb'), className: "com.stupidbird.game.entity.PlayerEntity", id: 20171571, puid: "bot20171571" } ninserted:1 keyUpdates:0 numYields:0 locks(micros) w:156950 156ms
conn16] command stupidbird.$cmd command: insert { insert: "PlayerEntity", ordered: true, documents: [ { _id: ObjectId('5683727b4db37416081856fb'), className: "com.stupidbird.game.entity.PlayerEntity", id: 20171571, puid: "bot20171571" } ] } keyUpdates:0 numYields:0 locks(micros) w:730 reslen:40 157ms
conn16] insert stupidbird.PlayerEntity query: { _id: ObjectId('5683727b4db374160818579a'), className: "com.stupidbird.game.entity.PlayerEntity", id: 20171730, puid: "bot20171730" } ninserted:1 keyUpdates:0 numYields:0 locks(micros) w:112769 112ms
conn16] command stupidbird.$cmd command: insert { insert: "PlayerEntity", ordered: true, documents: [ { _id: ObjectId('5683727b4db374160818579a'), className: "com.stupidbird.game.entity.PlayerEntity", id: 20171730, puid: "bot20171730" } ] } keyUpdates:0 numYields:0 locks(micros) w:750 reslen:40 113ms
clientcursormon] mem (MB) res:314 virt:119248

888. others

  1. 查看数据库基础信息。
    1. db.serverStatus()查看整体服务器信息。
    2. db.stats()查看当前db的性能状况。
    3. db.collection.stats()查看指定集合的性能状况。
  2. 备份还原。
    1. 备份。mongodump -h 127.0.0.1:27017 -d(指定数据库) -o(输出到指定目录) -u(用户名) -p(密码)
    2. 还原。mongorestore -h 127.0.0.1 -d(数据库) --dir(输入源，3.0版本之前叫做--directoryperdb)