search

Refty / mongo-thingy

:leaves: Powerful schema-less ODM for MongoDB and Python (sync + async)
https://mongo-thingy.readthedocs.io
MIT License
68 stars 12 forks source link

How to refresh/ reload data from database? #33

Closed Dewsworld closed 3 years ago

Dewsworld commented 3 years ago

Hello,

As the title suggests, is there any method for reload?

ramnes commented 3 years ago

There isn't, for now! What's your use case?

Dewsworld commented 3 years ago

There isn't, for now! What's your use case?

I want to update the data from the database every 60 secs

ramnes commented 3 years ago

For a single object or a cursor?

Why not using find or find_one again, every 60 seconds?

amcgregor commented 3 years ago

"Refreshing" an object does not involve the construction of a new one, and may potentially be limited to a subset (projection) of fields to be refreshed. Example. (The "TODO" there relates to handling nested/"complex" objects better and the "top level" current limitation.)

Notably, a "refreshed" object will share an identity. (Calling .reload() on one handle reloads the object in-place.)

some_record is some_record.reload()

Whereas the same is not true of multiply-queried documents. Querying again will potentially leave a hanging reference to the old version of the object. (Edit to add: a possible memory leak.)

# With a single record present...
SomeDocument.find_one() is not SomeDocument.find_one()

This seems to be the case, here, as well, unless Thingy is doing some fairly deep magic behind-the-scenes to deduplicate and "cache" multiple instances of the underlying data. Something I strongly dislike other DAO engines doing, as it can lead to problems, which can, ironically, involve a greater need for a reload method.

I want to update the data from the database every 60 secs

@Dewsworld This is referred to as "polling", and is the worst¹ solution to periodic or intermittent data processing. Have you examined Change Streams to pivot to realtime / event-based processing? (I.e. have your application do something when new data comes in, versus asking periodically if there's new data?)

¹ Though sometimes only viable solution. Regardless, it should always remain the solution of last resort.

ramnes commented 3 years ago

I get the identity argument, but I still don't see a use case for a reload method. On the opposite, I see a lot of value in supporting change streams, yes!

amcgregor commented 3 years ago

I apologize for distracting you from a 15 line fix with a potentially multi-hundred-line one. Practicality beats purity, but I will gladly accept a differentiating feature between your solution and mine. Change streams and reloading are distinct, and independently useful. The suggestion was for the developer asking the question initially as an avenue of investigation superior to polling, not that change streams would be a solution to reloading.

ramnes commented 3 years ago

Alright, let's close this issue for now, we're all loosing our time here.