Connection with mongoengine

[rocky-holms]

This is a really awesome lib. I wanted to see if I am able to start a db, and use it with mongoengine. I'm not sure if it requires an actual connection.

[scottrogowski]

Hey Rocky,

Not sure if I completely understand your question. To start a client you can either do:

client = mongita.MongitaClientMemory() or client = mongita.MongitaClientDisk()

The second option allows you to optionally specify a directory where files should be stored (default is ~/.mongita). Once you have a client, you can run a subset of the pymongo interface without needing MongoDB installed. Hope that helps, let me know if I didn't answer it.

[mikeckennedy]

Hi guys. I spend 30 minutes with this today and we have, drum roll please...

class Post(Document):
    title = StringField(max_length=120, required=True)
    author = ReferenceField(User)
    tags = ListField(StringField(max_length=30))

post1 = Post(title='Fun with MongoEngine')
post1.content = 'Took a look at MongoEngine today, looks pretty cool.'
post1.tags = ['mongodb', 'mongoengine']
post1.save()

post2 = Post(title='MongoEngine Documentation')
post2.link_url = 'http://docs.mongoengine.com/'
post2.tags = ['mongoengine']
post2.save()

print(Post.objects().all())

Outputs:

[
    POST: Fun with MongoEngine by None tagged ['mongodb', 'mongoengine'], 
    POST: MongoEngine Documentation by None tagged ['mongoengine']
]

There is a tiny bit of work that needs to be done in Mongita to make this work. Scott you open for PRs?

[mikeckennedy]

The setup / connect / patch code is:

import pymongo
import mongita

pymongo.MongoClient = mongita.MongitaClientMemory

# noinspection PyUnresolvedReferences
import mongoengine

from mongoengine import Document, StringField, ReferenceField, ListField

mongoengine.connect('mongita_test_db')

But Mongita can't handle a few basic operations, hence needs updating. In particular MongitaClientMemory needs to accept more kwargs, from:

def __init__(self, strict=False)

to

def __init__(self, strict=False, host="memory", port='NAN', read_preference=None)

And needs a few "implemented" fields:

self.is_primary = True

And Collection needs some fields too:

self.write_concern =  WriteConcern("w", 1, True, False)
self.with_options = lambda write_concern: self

But that should be an easy PR.

[scottrogowski]

Hi Michael,

As I understand it, you're suggesting adding some dummy fields / parameters (which will always be the same value) to allow Mongita to be integrated into MongoEngine, correct?

My primary design goal is for Mongita to be easily swappable in and out with PyMongo so given that, absolutely, I'm in.

I suspect that, for the parameters, it probably makes sense to also include asserts

On Fri, Apr 23, 2021 at 12:24 PM Michael Kennedy @.***> wrote:

The setup / connect / patch code is:

import pymongo import mongita

pymongo.MongoClient = mongita.MongitaClientMemory

noinspection PyUnresolvedReferences

import mongoengine

from mongoengine import Document, StringField, ReferenceField, ListField

mongoengine.connect('mongita_test_db')

But Mongita can't handle a few basic operations, hence needs updating. In particular MongitaClientMemory needs to accept more kwargs, from:

def init(self, strict=False)

to

def init(self, strict=False, host="memory", port='NAN', read_preference=None)

And needs a few "implemented" fields:

self.is_primary = True

And Collection needs some fields too:

self.write_concern = WriteConcern("w", 1, True, False) self.with_options = lambda write_concern: self

But that should be an easy PR.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/scottrogowski/mongita/issues/4#issuecomment-825839719, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADR6O3QW2UOYVHE6RGC6OLTKG3NFANCNFSM43MYNRXQ .

[mikeckennedy]

Hi Scott. Yes, exactly. The problem was it was crashing when the keyword argument host, etc was being passed. We know it's not used, but jsut for allowing it to fill in where that method is being called, it should offer no-op params to make the Python code still run.

Same for write_concern on Collection.

The one thing that actually seemed to be needed to make it work is self.with_options is called during the queries.

I think having asserts of whats possible might make sense. But you could also make them warnings if there is any chance it would still work.

[scottrogowski]

Got it! And everything is clear now with @rocky-holms's question which I apparently read too fast before 😂.

[mikeckennedy]

Great. Here's the file I got running in its entirety. It should show most aspects of MongoEngine. It has a query and a couple of inserts.

There is a small bug about querying nested lists but that's only relevant once ODMs are actually supported so one thing at a time. I suspect the fix is easy.

program.py.zip

Requirements to program.py: mongoengine mongita

[scottrogowski]

Cool. Thank you for putting this together! I'll be able to make the changes on Monday or you're also plenty welcome to open a PR if you have everything in place already.

[scottrogowski]

When you have the chance, I'm also curious about the bug you found.

[mikeckennedy]

The bug is in querying an element of an embedded list.

So queries like this work fine:

Post.objects().filter(link_url='http://docs.mongoengine.com/').all()

but this one fails:

Post.objects().filter(tags='mongoengine').all()

We have tags = ['mongoengine', 'mongodb'] and I'm just guessing this is comparing [] == 'mongoengine' rather than 'mongoengine' in [] effectively.

I think this is equivalent to: db.posts.find( { 'tags': 'mongoengine'}) in the native query syntax.

[rocky-holms]

This is awesome @scottrogowski and @mikeckennedy. This is exactly what I was looking for. Very excited for this to be added.

[scottrogowski]

Today I learned that the default MongoDB behavior, when a document element is a list, is to perform a 'contains' operation. Thanks for that @mikeckennedy. The bug is fixed and released as 0.1.3.

I have the MongoEngine example you sent over working in a pull request https://github.com/scottrogowski/mongita/pull/6. I'm not very familiar with MongoEngine so was curious whether either of you had any tips for testing before I merge it in. Specifically, what things could go wrong with MongoEngine that are not on the happy path?

Thanks for everything so far!

[scottrogowski]

Closing this issue as basic support has been merged. Please do open another if you run into any specific issues