search

limpyd / redis-limpyd

Provide an easy way to store python objects in Redis, without losing the power and the control of the Redis API
https://redis-limpyd.readthedocs.org/
Do What The F*ck You Want To Public License
72 stars 11 forks source link

Having some trouble getting things working #140

Open mikeywaites opened 4 years ago

mikeywaites commented 4 years ago

Hey!

I stumbled across your project this weekend I was immediately excited. I have been thinking about the need for this recently so it was great to find something that it looked like it solved all the use cases I have.

I was keen to get a simple POC together and have a play around but things are quite working. I had some questions that I hoped you wouldn't mind answering.

TLDR;

1 - Are you still maintaining the project? Should it be used for a production use case?

2 - Is there a way to pass a dict containing nested dicts and have those automatically create related models?

3 - is there a way to fetch an object from the cache and have it "inflate" all the related models into nested dicts?

Longer Example

Below is a test script I've put together to try and understand if how I interpreted limpyd to work is correct. What i was hoping would be possible was that you could simply instantiate the ProjectCache obj with a dict that contained keys for the related models. This would mean i'd simply store a project object and for any related keys it would "get or create" those models on the fly.

With a project object cached and each of it's related keys cached we would then be able to pull the project and all the related keys out in one go.

class BaseCache(related.RelatedModel):
    database = main_database
    abstract = True
    id = fields.PKField()
    created_at = fields.StringField()
    updated_at = fields.StringField()

class CompanyRelationshipCache(BaseCache):
    company = related.FKInstanceHashField("Company", related_name="customer")
    other_party = related.FKInstanceHashField("Company", related_name="supplier")

class CompanyCache(BaseCache):
    name = fields.InstanceHashField()

class UserCache(BaseCache):
    email = fields.InstanceHashField()
    name = fields.InstanceHashField()
    title = fields.InstanceHashField()
    profile_picture = fields.InstanceHashField()
    timezone = fields.InstanceHashField()
    state = fields.InstanceHashField()
    company = related.FKInstanceHashField("CompanyCache")

class ActivityCache(BaseCache):
    abstract = True
    name = fields.InstanceHashField()
    user = related.FKInstanceHashField("UserCache", related_name="activity_user")
    owner = related.FKInstanceHashField("UserCache", related_name="activity_owner")
    company_relationship = related.FKInstanceHashField("CompanyRelationshipCache")

class ProjectCache(ActivityCache):

    status = fields.InstanceHashField()

if __name__ == "__main__":
    try:
        ProjectCache.get("project-id-1").delete()
    except Exception:
        pass
    try:
        CompanyCache.get("company-1").delete()
    except Exception:
        pass
    try:
        UserCache.get("user-1").delete()
    except Exception:
        pass

    company_data = {"id": "company-1", "name": "Acme"}
    user_data = {
        "id": "user-1",
        "name": "Mike Waites",
        "email": "foo@acme.co",
        "company": company_data,
        "profile_picture": "https://img.acme.co/xxx.jpg",
        "timezone": "Europe/London",
        "title": "Job Title",
        "state": "ACTIVE",
    }
    rel = {
        "id": "rel-1",
        "company": company_data,
        "other_party": company_data,
    }
    project = {
        "id": "project-id-1",
        "user": user_data,
        "owner": user_data,
        "company_relationship": rel,
    }
    obj = ProjectCache(
        id="project-id-1",
        user=user_data,
        owner=user_data,
        company_relationship=rel,
        name="Test Project",
    )

    assert obj.hgetall() == project

This doesn't work and fails with the error 

File "/usr/local/lib/python3.6/dist-packages/limpyd/contrib/related.py", line 388, in func
    args[0] = self.from_python(args[0])
IndexError: list index out of range

This time I create the related models up front and pass them to the ProjectCache

class BaseCache(related.RelatedModel):
    database = main_database
    abstract = True
    id = fields.PKField()
    created_at = fields.StringField()
    updated_at = fields.StringField()

class CompanyRelationshipCache(BaseCache):
    company = related.FKInstanceHashField("Company", related_name="customer")
    other_party = related.FKInstanceHashField("Company", related_name="supplier")

class CompanyCache(BaseCache):
    name = fields.InstanceHashField()

class UserCache(BaseCache):
    email = fields.InstanceHashField()
    name = fields.InstanceHashField()
    title = fields.InstanceHashField()
    profile_picture = fields.InstanceHashField()
    timezone = fields.InstanceHashField()
    state = fields.InstanceHashField()
    company = related.FKInstanceHashField("CompanyCache")

class ActivityCache(BaseCache):
    abstract = True
    name = fields.InstanceHashField()
    user = related.FKInstanceHashField("UserCache", related_name="activity_user")
    owner = related.FKInstanceHashField("UserCache", related_name="activity_owner")
    company_relationship = related.FKInstanceHashField("CompanyRelationshipCache")

class ProjectCache(ActivityCache):

    status = fields.InstanceHashField()

if __name__ == "__main__":
    try:
        ProjectCache.get("project-id-1").delete()
    except Exception:
        pass
    try:
        CompanyCache.get("company-1").delete()
    except Exception:
        pass
    try:
        UserCache.get("user-1").delete()
    except Exception:
        pass
    try:
        CompanyRelationshipCache.get("rel-1").delete()
    except Exception:
        pass

    company_data = {"id": "company-1", "name": "Acme"}
    user_data = {
        "id": "user-1",
        "name": "Mike Waites",
        "email": "foo@acme.co",
        "profile_picture": "https://img.acme.co/xxx.jpg",
        "timezone": "Europe/London",
        "title": "Job Title",
        "state": "ACTIVE",
    }
    rel = {
        "id": "rel-1",
        "company": company_data,
        "other_party": company_data,
    }
    project = {
        "id": "project-id-1",
        "user": user_data,
        "owner": user_data,
        "company_relationship": rel,
    }
    company_cache = CompanyCache(**company_data)
    user_cache = UserCache(company=company_cache, **user_data)
    rel_cache = CompanyRelationshipCache(
        id=rel["id"], company=company_cache, other_party=company_cache
    )
    obj = ProjectCache(
        id="project-id-1",
        user=user_cache,
        owner=user_cache,
        company_relationship=rel_cache,
        name="Test Project",
    )

    assert obj.hgetall() == project

This time the ProjectCache object is created using refs to other models which doesn't error. The issue here however is that there isn't a way to "inflate" the object back into its full nested state.

Sorry for the massive dump of notes. I think the project is really great and I'm really hoping to use it but want to make sure I'm going in the right direction or what I'm trying to achieve would be possible before I go too far.

Thanks in advance.

twidi commented 4 years ago

I'll read in detail your issue in a few day, no time for complex thinking today :)

But I read the first lines and so I can answer your 1/ : yes this project is still maintained (only by me) and yes you can use it in production, I do, heavily, and I have a client (I'm a freelancer) that use is too.

mikeywaites commented 4 years ago

Hey @twidi

Thanks for the quick response. It's great to hear the project is still active. I'll look forward to hearing back from you about the best way to approach my use case.