pymongo 不能insert 带“_id”的json, Hexadecimal string with odd number of characters

[mervynzhang]

导出json

from bson.json_util import dumps
from pymongo import MongoClient
import json

client = MongoClient()
db = client.authordb
collection = db.authors
cursor = collection.find({})
with open('authors.json', 'w') as file:
    json.dump(json.loads(dumps(cursor)), file)

导入json文件 报错 pymongo.errors.OperationFailure: Hexadecimal string with odd number of characters: "($oid:65530eb140d1cd61d595d70d)" [90003-0], full error: {'minWireVersion': 0, 'maxWireVersion': 17, 'ok': 0, 'n': 1, 'code': 90003, 'errmsg': 'Hexadecimal string with odd number of characters: "($oid:65530eb140d1cd61d595d70d)" [90003-0]'}

client = MongoClient()
db = client.authordb
collection = db.authors_restore
with open('authors.json', 'r') as f:
    file_data = json.load(f)

print(len(file_data))
collection.insert_many(file_data)     

去掉_id 可以insert

[mervynzhang]

导入脚本有点问题， mongodb 也报错， 不过报的不一样。我改改脚本先

pymongo.errors.BulkWriteError: batch op errors occurred, full error: {'writeErrors': [{'index': 0, 'code': 52, 'errmsg': "_id fields may not contain '$'-prefixed fields: $oid is not valid for storage.", 'op': {'_id': {'$oid': '65531b07c04d64e658949d08'}, 'author': 'Mike', 'text': 'My first blog post!', 'tags': ['mongodb', 'python', 'pymongo'], 'date': {'$date': '2023-11-14T07:00:23.233Z'}}}], 'writeConcernErrors': [], 'nInserted': 0, 'nUpserted': 0, 'nMatched': 0, 'nModified': 0, 'nRemoved': 0, 'upserted': []}

[mervynzhang]

mongoimport 可以导入mongodb

mongoimport --db authordb --collection authors_import --file authors2.json --jsonArray    
2023-11-14T16:13:25.524+0800    connected to: mongodb://localhost/
2023-11-14T16:13:25.532+0800    3 document(s) imported successfully. 0 document(s) failed to import.

mongoimport 到lealone的时候报错

Failed: an inserted document is too large

[codefollower]

mongoimport 可以导入mongodb

mongoimport --db authordb --collection authors_import --file authors2.json --jsonArray    
2023-11-14T16:13:25.524+0800    connected to: mongodb://localhost/
2023-11-14T16:13:25.532+0800    3 document(s) imported successfully. 0 document(s) failed to import.

mongoimport 到lealone的时候报错

Failed: an inserted document is too large

lealone 没有定义这个错误，如果 lealone 出错了你可以看看 lealone 那边的错误是什么。

[codefollower]

我也试了一下 mongoimport，很小的文档也出现 Failed: an inserted document is too large （lealone 没有这个错误信息) lealone 这边并没有收到 json 文档的数据，还不清楚为什么，导入 mongodb 是正常的。

可能 mongoimport 需要服务器端给它返回一个什么参数，然后 lealone 没有这个参数它就判断最大文档是0，然后它自己 import 文档前就提前报错了。

[codefollower]

给客户端返回 maxBsonObjectSize、maxMessageSizeBytes、maxWriteBatchSize 参数后 mongoimport 能正常导入数据了。

[mervynzhang]

给客户端返回 maxBsonObjectSize、maxMessageSizeBytes、maxWriteBatchSize 参数后 mongoimport 能正常导入数据了。

数据量大些的话， mongodump/mongorestore更方便些