Research viability of direct hooks into Overleaf

[wrongu]

Current git-based design

Pros

• able to get it up and running instantly
• don't need to know anything about overleaf document models, javascript, and all that jazz. Just need to be able to read/write files using python.
• easy enough to "trigger" AI suggestions with % @ai: do a thing for me syntax
• all requests are initiated on our end. no need to worry about websockets or accepting incoming updates from overleaf

Cons

• git merge can be tricky (#2) and the best way we have to handle it now involves lots of extra calls to OpenAI (#3)
• unclear if git fetch/commit/merge loop is fast enough / scalable
• little visual indication to the user that something is happening
• no access to "current cursor position"
• we end up wanting to implement our own database (#9)
• mild slowdown due to needing to poll for git updates. but not a huge deal because LLM calls are still the bottleneck

Hypothetical direct integration with overleaf

Speculative pros

• overleaf is open source
• maybe able to get the assistant to appear as a collaborator in the doc / give other visual indications
• maybe able to use "suggestions" and "comments" rather than just edits
• maybe get access to cursor position
• maybe get access to the document model and history directly
• maybe able to debounce user edits more directly / hook into however overleaf already does this (e.g. with their spell-checker)

Speculative cons and things to look into

• despite being open source, they don't appear to have docs about their own api. we'd have to invest a bit of time in reverse-engineering things / familiarizing ourselves with it
• can we still do it in our own python server, or will there be tons of CORS / auth issues? For comparison, writefull is all javascript. But unclear if this is because they're opting for a browser plugin model or if this is necessary.

[j6k4m8]

I'm totally supportive and also overwhelmingly unmotivated to write it myself

---

• we end up wanting to implement our own database

I'd consider this more an analytics and prompt-engineering feature opportunity than a con

• unclear if git fetch/commit/merge loop is fast enough / scalable

LLM roundtrips are the dominant rate-limiter here, not sure that document IO speed really matters in a meaningful way beyond the one or two seconds to put in a "pending" indicator, like you mentioned before

[j6k4m8]

Also there's rate limiting which kinda makes git endpoints a showstopper.

[wrongu]

I spent a bit of time in the overleaf source today and wrote some preliminary notes on how it works. My notes document then somehow got erased... so, here's what I can try to piece together again quickly:

• there's some initial round trip of magic auth stuff (oauth2, I think) used to check the user account and verify that they have access to the given document. Auth is outside my comfort zone, so hopefully someone else can figure out what this looks like from python. Possible overleaf source starting point is here
• once auth is done, all subsequent client <-> server synchronization is done using a websocket that is kept alive for a long time
• the "real time" editor is supported by operational transforms or OT, which is how things like google docs handle multiple simultaneous edits by different users. OT updates are sent over the socket here. Note in the lines above this there's some stuff having to do with track-changes, which is overleaf's equivalent of "suggestion mode" in google docs
• another kind of data sent over the socket is info about is user A being notified of where user B's cursor is. If we want to show users where the AI "cursor" is, we would send a 'clientTracking.clientUpdated' packet over the socket, as seen here

Aaand that's about as far as I've gotten so far.