[RFC] Add alternative frontend API without proxy

[zhang123cnn]

• This is only a RFC PR for comments. Please do not merge. *

Hey folks,

My name is Xiaomeng and I am a software from Facebook. My colleague @camiboj and I are working on an experimental project to leverage automerge to decentralise state management in our infrastructure.

Automerge uses JS Proxy extensively for its front end API. However, in FB, we need to support multiple JS runtime which does not support Proxy.

I can see a similar question has been raised here before.

https://github.com/automerge/automerge/issues/257

@ept advised us we could implement an alternative API which does not use proxies. So we went ahead and implemented a minimal version for ourselves.

It is very similar to current API. Because we could not use proxy, the new API requires lib users to be explicit on built-in functions for list, map. For example, You cannot do doc['key1'] = 'value1'. Instead you need to do doc.set('key1', 'value1'). Check our unit test file test/proxies_polyfill_test.js for more examples in code.

Our fork worked for us, but ideally we would prefer not to keep a permanent fork. So we have been wondering if it is possible to merge this alternative API back to automerge repo. It would help us not to maintain a fork and maybe help others potential automerge users who also lives with similar constraints.

What do you think?

CC @jtannady

[ept]

Hi @zhang123cnn! Great to see you've gone ahead with this. I understand the need for a proxy-free API, and I'd be happy to have such an API be part of what we maintain in the Automerge project and its official releases.

We'd probably want to iterate a bit on the details of the API before committing to it. From a code structure point of view, it might make sense to separate the proxy-based frontend and the proxy-free frontend into two separate directories, and allow users to import the one they want, but I'd want to get some more opinions from people with more experience in JS packaging.

Another thing I'm thinking about is that @orionz is currently working on a redesign of the frontend-backend API with a view towards improving performance and making it easier to implement frontends. Assuming that works out, a proxy-free API will probably become a fairly small amount of code, since most of the current frontend complexity will move to the backend.

I'd also love to hear more about what you're doing with Automerge at Facebook!

[zhang123cnn]

@ept Thanks for your reply!

I'd be happy to have such an API be part of what we maintain in the Automerge project and its official releases.

This is great news!

it might make sense to separate the proxy-based frontend and the proxy-free frontend into two separate directories

This makes sense to me. The majority of the code is already in a standalone js file frontend/proxy_polyfill.js. We could certainly put them into a different directory and separate the entry point from the original proxy.js file.

We currently have a global function setFacebookSyntax to switch proxy-based API to non-proxy-based one. I guess we would like to rename it to something more general.

Assuming that works out, a proxy-free API will probably become a fairly small amount of code

This is also great! Do you know what would be the rough timeline on this? We do want to ship things with automerge in a few weeks so we would prefer to adapt our code to new design slightly later.

I'd also love to hear more about what you're doing with Automerge at Facebook!

In facebook, we are working on a new state management API for AR effects running in video calls. We use automerge because we want all state sync logic happening on client side to better protect user's privacy. We have some interesting use case. Once our announcement is out, we are probably come back to automerge community with more questions and thoughts.

About next steps, I guess we could just create a formal PR to start the iteration. CC @camiboj We do hope to release our API soonish. So please let us know about anything we need to amend to proceed.

Thanks in advance.

[ept]

Great. Please do share your announcement when it's out!

As a first step, before we get into the code, can I suggest that you write up a document explaining the API you're proposing? I know you have the tests, but I think a document would help facilitate a community discussion about the API design. It would be good to include a discussion of the edge cases that arise, such as: how do you deal with nested objects? conflicts? how do you handle the various datatypes that Automerge supports, such as lists, text, and counters?

I'm not sure when the frontend-backend API changes will happen — it's a fairly complex project, since it affects most of the codebase, and requires coordination between the JS and Rust implementations of Automerge. My guess is that it's a couple of weeks away.

[zhang123cnn]

closing this PR since the formal PR has been created here. https://github.com/automerge/automerge/pull/439

[douira]

Assuming that works out, a proxy-free API will probably become a fairly small amount of code, since most of the current frontend complexity will move to the backend.

This is interesting. Do you think it would be possible to also implement other frontends, such as one that produces a mutated state instead of an immutable one? (for a more elegant integration with mutation-based frameworks like Vue.js for example)

[ept]

Do you think it would be possible to also implement other frontends, such as one that produces a mutated state instead of an immutable one?

Yes, that should definitely be possible! If you want to think about what your ideal API would look like for integrating with Vue, I'd love it if you could write up a brief proposal for an API.