Question about CRUD example

[silvanaalbert]

I am struggling to find an example with a basic CRUD using this promising library. I was able to create a document. However, it is not clear to me how I can retrieve it from mongo, using pymongo, then changing just one field and saving it back. I wouldn't want to default to pymongo basics since there has to be a umongo friendly way that I haven't found yet. Also, I could help writing a basic crud example that could serve to future noobs such as myself. Here is what I have so far:

Project.py file

from datetime import datetime
from pymongo import MongoClient
import umongo
from umongo import Instance, Document, fields, validate

db = MongoClient().test
instance = Instance(db)

@instance.register
class Project(Document):
    projectId = fields.IntegerField(required=True)
    name = fields.StrField(required=True)
    ownerName = fields.StrField(required=True)
    ownerORCID = fields.StrField(required=True)
    description = fields.StrField()
    createdDate = fields.DateTimeField(default=datetime.now)
    updatedDate = fields.DateTimeField(default=datetime.now)
    samples = fields.ListField(fields.ReferenceField("Sample"))
    isLocked = fields.BooleanField(default="false")

    class Meta:
        collection_name = "project"

And Project_DAO.py file

from project import Project

def createProject(projectJson):
    new_project = Project(**projectJson)
    new_project.commit()
    new_project.dump()

    print("Created: " + new_project.name)
    return Project.find_one({"projectId": 5})

def updateProject(id):
    project = fromProject.find_one({"projectId": id})

    if(project):
        project.dump()
        project.isLocked = "true"
# here is where I need help since it does not have all my fields, just the one I set above: isLocked
        project.commit()
        updated_project = Project.find_one({"projectId": id})
        updated_project.dump()
        print (updated_project.isLocked)
    else:
        print ("Project not found")

def updateProject(id):
    # no idea how yet

if __name__ == '__main__':
    new_project = {
        "projectId": "5",
        "name": "My Project",
        "ownerName": "Silvana Albert",
        "ownerORCID": "0000-0001-6719-9139",
        "description": "Sample Project",
        "isLocked": "false"
    }
    createProject(new_project)
    updateProject(5)

Thank you in advance and keep up the good work!

[lafrech]

Hi Silvana. Welcome on board.

I don't understand your question. When you fetch your document with find, the return value should contain all the fields from the doc in database.

[silvanaalbert]

Thank you for your answer. You are right..I had a problem with different objects and different ids and missing marshmallow values. Now I have create, read and update working. I don't know how to delete since there is no remove method..I was thinking something like this:

def deleteProject(id):
    deleted_project = Project.remove({"projectId": id})

where

from datetime import datetime
from pymongo import MongoClient
import umongo
from umongo import Instance, Document, fields, validate

db = MongoClient().test
instance = Instance(db)

@instance.register
class Project(Document):
    projectId = fields.IntegerField(required=True)
    name = fields.StrField(required=True)
    ownerName = fields.StrField(required=True)
    ownerORCID = fields.StrField(required=True)
    description = fields.StrField()
    createdDate = fields.DateTimeField(default=datetime.now)
    updatedDate = fields.DateTimeField(default=datetime.now)
    samples = fields.ListField(fields.ReferenceField("Sample"))
    isLocked = fields.BooleanField(default="false")

    class Meta:
        collection_name = "project"

Could it work? If not, I have the following plain pymongo option:

    project = mongo.db.Project
    q = project.find_one({'id': id})
    x = project.delete_one(q)

I would like to have a Project_DAO with all the methods consistent. Does that make sense?

[lafrech]

umongo allows you to

• commit or delete an object (object method)
• find objects using filters (class method)

You can delete an instantiated object with the delete object method

obj = Project.find_one({'id': id})
obj.delete()

with the cost of an extra instantiation.

We could add a class delete method such as

Project.delete({'id': id})

but let alone the name collision with the delete object method, it would basically just proxy pymongo so it wouldn't be much better than

Project.collection.delete({'id': id})

You can use the latter if you need.

I don't know the whole architecture of your app but you might not even need this DAO layer. I think the ODM is meant to replace the DAO.

Deal with objects. Instantiate them. Modify them. Commit to record them. Delete them.

The delete without instantiation case is an optimization. In my app, I don't even do it. Ideally, I should skip the instantiation for performance but it's no big deal. To do so, I would use the method described above using pymongo through collection.

I hope this helps.

[silvanaalbert]

It does. This is exactly what I need, and I agree, with the exsting methods there is no need for another one. I would suggest however updating this document since it talks only about pre_delete and post_delete. Thank you for taking the time and helping. As promised, I will attach my first working draft, in case it helps anybody. Please let me know if it is against the principles of use and I will remove it. I would not want to influence in a negative way (I have a strong background with Java, JDBC/Hibernate, Spring...so this new way of thinking is new to me)

from datetime import datetime
from pymongo import MongoClient
import umongo
from umongo import Instance, Document, fields, validate

db = MongoClient().test
instance = Instance(db)

@instance.register
class Project(Document):
    projectId = fields.IntegerField(required=True)
    name = fields.StrField(required=True)
    ownerName = fields.StrField(required=True)
    ownerORCID = fields.StrField(required=True)
    description = fields.StrField()
    createdDate = fields.DateTimeField(default=datetime.now)
    updatedDate = fields.DateTimeField(default=datetime.now)
    samples = fields.ListField(fields.ReferenceField("Sample"))
    isLocked = fields.BooleanField(default="false")

    class Meta:
        collection_name = "project"

and

from project import Project

def createProject(projectJson):
    new_project = Project(**projectJson)
    new_project.commit()

    created_project = Project.find_one({"projectId": 15})

    return created_project

def getProjectById(id):
    return Project.find_one({"projectId": id})

def updateProjectStatus(id, status):
    project = Project.find_one({"projectId": id})

    if(project):
        project.isLocked = status
        project.commit()

        updated_project = Project.find_one({"projectId": id})
        print (updated_project.isLocked)
    else:
        print ("Project not found")

def deleteProject(id):
    project_to_delete = Project.find_one({"projectId": id})

    if(project_to_delete):
        deleted_count = project_to_delete.delete().deleted_count
        if(deleted_count > 0):
            return 'Deleted ' + str(deleted_count) + ' project(s)' 
        else:
            return 'Could not delete'
    else:
        return 'There is no project with the id: ' + str(id)

if __name__ == '__main__':
    project_id = 15

    new_project = {
        "projectId": project_id,
        "name": "JD Project",
        "ownerName": "Joana Doe",
        "ownerORCID": "0000-0000-0000-0000",
        "description": "Sample Project",
        "isLocked": "false"
    }

    createProject(new_project)

    print(getProjectById(project_id))

    updateProjectStatus(project_id, 'true')

    print(deleteProject(project_id))

Thank you again!

[lafrech]

Yep, your Java background inspires your variable naming. Welcome to the light side of the Force.

In createProject, you don't need to find after committing. Just return the object.

Again, I suspect you don't need this layer in the first place. Or perhaps you don't need umongo.

The point of umongo is to provide objects that know how to serialize in DB. In other words, you build your objects and you have DAOs for free (cheap would be more accurate).

If you instantiate a umongo object in a DOA just to serialize it in DB, then perhaps you should have committed your dict in DB right away, validation aside.

What an ODM such as umongo provide is the ability to develop a OO program while abstracting the DAO layer.

So, no, I don't think this code sample is an example to follow, but please don't erase it, as it is relevant to this conversation.

[silvanaalbert]

Interesting...my first draft was just the api with Flask which directly committed in the DB, but I was worried about "separation of concerns", making gradual changes to an entity, having in place a kind of schema easy to follow. I started implementing a basic validator and I did not like it so I started searching for an ORM. Using something like Umongo makes it closer to the OOP world for me, so I think I will keep it, at least until my next refactor. I remember in a lot of frontend tutorials it was sometimes a lot easier to start as a complete beginner, instead of having prior knowledge on something else because you might end up "over-engineering" something simple. Maybe it is the same case here. Thanks and keep up the good work!

[lafrech]

You're on the right track.

If you want to build a CRUD-ish API, I can give you a few tips.

You need a validation layer to sanitize your inputs before you do anything, to protect yourself from invalid or malicious inputs. That's what marshmallow is made for.

Then an ODM is nice because as soon as your inputs are validated, you can instantiate your objects and implement your business logic in OOP with a DB abstraction.

umongo does provide validation but the API you expose is not always exactly the database document structure. You may want to hide fields, or expose embedded documents as sub-resources. So you need that separate validation layer.

The good thing about umongo is that it is marshmallow-based and it can generate schemas you can tweak and use for the API. This avoids duplication as you get API schemas for free with only a few modifications to make if needed.

Here's my stack:

• Flask
• umongo for MongoDB management
• marshmallow for the API serialization/deserialization
• webargs to inject marshmallow-deserialized request into view function
• apispec to generate a nice Swagger documentation
• flask-smorest to wrap marshmallow/webargs/apispec together

Here's a simple example with SQLAlchemy in place of umongo: https://github.com/lafrech/flask-smorest-sqlalchemy-example. The concepts are similar.

I don't know if you speak french. If you do, you may be interested in this PyConFR presentation:

https://lafrech.github.io/marshmallow-pyconfr2019/ https://pyvideo.org/pycon-fr-2019/marshmallow-de-la-serialisation-a-la-construction-dune-api-rest.html

Good luck!

[luckystar1992]

 updatedDate = fields.DateTimeField(default=datetime.now)
    samples = fields.ListField(fields.ReferenceField("Sample"))

I am struggling to find an example with a basic CRUD using this promising library. I was able to create a document. However, it is not clear to me how I can retrieve it from mongo, using pymongo, then changing just one field and saving it back. I wouldn't want to default to pymongo basics since there has to be a umongo friendly way that I haven't found yet. Also, I could help writing a basic crud example that could serve to future noobs such as myself. Here is what I have so far:

Project.py file

from datetime import datetime
from pymongo import MongoClient
import umongo
from umongo import Instance, Document, fields, validate

db = MongoClient().test
instance = Instance(db)

@instance.register
class Project(Document):
    projectId = fields.IntegerField(required=True)
    name = fields.StrField(required=True)
    ownerName = fields.StrField(required=True)
    ownerORCID = fields.StrField(required=True)
    description = fields.StrField()
    createdDate = fields.DateTimeField(default=datetime.now)
    updatedDate = fields.DateTimeField(default=datetime.now)
    samples = fields.ListField(fields.ReferenceField("Sample"))
    isLocked = fields.BooleanField(default="false")

    class Meta:
        collection_name = "project"

And Project_DAO.py file

from project import Project

def createProject(projectJson):
    new_project = Project(**projectJson)
    new_project.commit()
    new_project.dump()

    print("Created: " + new_project.name)
    return Project.find_one({"projectId": 5})

def updateProject(id):
    project = fromProject.find_one({"projectId": id})

    if(project):
        project.dump()
        project.isLocked = "true"
# here is where I need help since it does not have all my fields, just the one I set above: isLocked
        project.commit()
        updated_project = Project.find_one({"projectId": id})
        updated_project.dump()
        print (updated_project.isLocked)
    else:
        print ("Project not found")

def updateProject(id):
    # no idea how yet

if __name__ == '__main__':
    new_project = {
        "projectId": "5",
        "name": "My Project",
        "ownerName": "Silvana Albert",
        "ownerORCID": "0000-0001-6719-9139",
        "description": "Sample Project",
        "isLocked": "false"
    }
    createProject(new_project)
    updateProject(5)

Thank you in advance and keep up the good work!

Hi, I was interested in the samples = fields.ListField(fields.ReferenceField("Sample")) line in your code, I have a similar work with you, when I get samples with lazy query like:

@property def samples: return self.samples.fetch()

and in my query code: @async_run def get_project(self): project = await Project.find_one({...}) # async get samples = await project.samples # lazy fetch

but in the lazy fetch line, it raise an error: marshmallow.Missiing. Can you meet this?