to_json dict value with key which is not in queryset is cleaned as {}

[CanoeFZH]

https://github.com/MongoEngine/mongoengine/blob/835d3c3d186144f1b7d4d929d40712c8ef75dd21/mongoengine/queryset/base.py/#L1739

[wojcikstefan]

@CanoeFZH would you please provide an example of a document schema, its expected to_json output, and the actual one?

[CanoeFZH]

@wojcikstefan right schema: In [65]: ArticleProfile.objects(article_seq_id=x).first().title_name_entities_tfidf {u'\u0938\u093e\u0932': -0.02852155634541126, u'\u092c\u0947\u091f\u0940': 0.027081640431142032, u'\u092a\u093f\u0924\u093e': 0.01680922909611616}

wrong data: ArticleProfile.objects(article_seq_id=x).to_json() ... "title_name_entities_tfidf": {}}]'

while you use first() will get right data: ArticleProfile.objects(article_seq_id=x).first().to_json() ... title_name_entities_tfidf": {"\\u0938\\u093e\\u0932": -0.02852155634541126, "\\u092c\\u0947\\u091f\\u0940": 0.027081640431142032, "\\u092a\\u093f\\u0924\\u093e": 0.01680922909611616},

to check in the code, you can see that when the data type is dict and key field is not the in the queryset schema, it will return {} as result.

[wojcikstefan]

I see that in the "wrong data" example, you use content_name_entities_tfidf instead of title_name_entities_tfidf used in the other two samples. What is the link between the two?

Also, I'm not sure what you mean by the "queryset schema". Do you mean the document schema? If so is this the correct description of the issue:

If you have a dict field which exists in a raw document in MongoDB but is not in the MongoEngine document schema, calling to_json on the MongoEngine document directly includes that field, but calling to_json on a queryset does not include it.

Let me know if that's correct or no. If it is, would you be able to submit a pull request with at least a simple failing unit test?

[CanoeFZH]

@wojcikstefan I updated the data example. Actually content_name_entities_tfidf and title_name_entities_tfidf are similar with each other. But to make it more clear, I update the data example. As for the query set schema, I mean document schema as you mentioned. faked test case: {id: 1, dict: {123: 1}} with document schema: id, dict

[CanoeFZH]

also update the url to direct to the code at line 1739 directly. @wojcikstefan