Non-Document Storage Transfer

[MattNewberry]

In the wiki, you mention for larger datasets needing to wait for future updates... can you elaborate?

Also, given the framework is highly geared around document-based storage, how difficult would it be to incorporate the changeset format into a RESTful interface?

Thanks for your work, it's quite awesome.

[drewmccormack]

Hi Matt,

At the moment, there are times when basically the whole store needs to be loaded into memory. For large stores, that could be a problem.

I am going to be optimizing that in the coming months.

Building a rest API would probably not be very easy at the moment, because there is no one file containing the latest data. The latest data is determined by playing back the changes on the client.

Also, the change sets are stored as binary core data stores. It probably wouldn't be too difficult to move to something like JSON, but you would still have the issue that the data is stored as transactions, rather than as a copy of the latest values.

Hope that is clear. I am starting to document some of these things in the book on ensembles.

Kind regards, Drew

[MattNewberry]

Thank you Drew for the thorough explanation... greatly appreciated. You've done a great job with this framework and it's one of the more appealing I've seen for my purpose.

In short, my goal is to have multiple devices be in sync whether online or offline. I'm planning on primarily focusing on sockets with long polling as a backup, and the Multipeer Connectivity Framework when network connectivity is unavailable.

Looking forward to seeing your progress, Matt

[drewmccormack]

I am hearing from a lot of people re: multipeer. I think it is an exciting direction that the framework can go, and I expect to add a backend for it in the not too distant future.

Drew

[MattNewberry]

That's great to hear! The last thing I want to do is say "why doesn't your code do X?". Rather, I'm starting to understand your philosophy and approach to see where an entry point may be.

In an ideal scenario, a simple entry point that handled new data and gave a protocol for raw data changes is what I'm looking for. You've got a very strong foundation, the missing link seems to merely come on transfer and storage protocols.

If ensembles proves to be a fit, would you be opposed to a pull request for a protocol for providing an encoder (JSON, binary, etc...)?

Again, very impressive work.

[drewmccormack]

I'd certainly be open to that, especially if I could make it configurable to switch between them.

The point to make this type of change would be in the migrator class.

Drew

On 28 Mar 2014, at 21:38, Matthew Newberry notifications@github.com wrote:

That's great to hear! The last thing I want to do is say "why doesn't your code do X?". Rather, I'm starting to understand your philosophy and approach to see where an entry point may be.

In an ideal scenario, a simple entry point that handled new data and gave a protocol for raw data changes is what I'm looking for. You've got a very strong foundation, the missing link seems to merely come on transfer and storage protocols.

If ensembles proves to be a fit, would you be opposed to a pull request for a protocol for providing an encoder (JSON, binary, etc...)?

Again, very impressive work.

— Reply to this email directly or view it on GitHub.