When requesting a specific field from the database on a small number of observations, if the field is not in any document returned by MongoDB then no column will be created for it in the dataset.
This is potentially surprising for a developper as you can be requesting 30 fields and only get 25 in the resulting dataset.
Solution
Since this is the expected/normal behavior from a document-oriented DB like Mongo, the solution should be implemented on our side.
I suggest having the self.fetch_data method compare self.fields with self.data.columns and create columns of NAs
Problem
When requesting a specific field from the database on a small number of observations, if the field is not in any document returned by MongoDB then no column will be created for it in the dataset.
This is potentially surprising for a developper as you can be requesting 30 fields and only get 25 in the resulting dataset.
Solution
Since this is the expected/normal behavior from a document-oriented DB like Mongo, the solution should be implemented on our side.
I suggest having the
self.fetch_datamethod compareself.fieldswithself.data.columnsand create columns of NAs