search

beakerbrowser / beaker

An experimental peer-to-peer Web browser
https://beakerbrowser.com/
MIT License
6.75k stars 544 forks source link

Live reloading chokes when there are a lot of quick updates #603

Closed rjcorwin closed 7 years ago

rjcorwin commented 7 years ago

Tonight @dwblair and I created a google docs like experience by creating an HTML doc with a contenteditable div with some Javascript that commits when changes are detected. A Google Doc like experience as in a Google Doc with permission of "Anyone with URL can read" granted this HTML doc is being published by a Beaker browser and someone else is on it's dat URL with live reloading turned on. See this gist for an example https://gist.github.com/rjsteinert/966525c209402320931dfd8b04dfd997. I would have published to hasbase or published the dat repo on an "always on raspberry pi" but I haven't gotten that far yet!

We some some interesting behavior. It worked ok if we cranked up debouncing but turn it down and we get some funkiness. The strangest behavior by far was not seeing an update on my friend @dwblair's browser until I hit the space bar on my end. As in... Type a bunch of things, wait for the change to propogate to friend's browser, nothing happens, hit space bar, instantly reloads for friend!

We thought it might be our code having the issue but we see the file changing correctly on the file system on the host machine. I realize now we should have captured the whole experiment on a screen recording but we didn't think of it at the time. Another day we will! Cheers.

rjcorwin commented 7 years ago

Hashbase'd!

pfrazee commented 7 years ago

Yeah I just reproduced the issue with live reloading. I've added it to my todos!

rjcorwin commented 7 years ago

Thank for taking a look @pfrazee! For what it's worth, @dwblair and I wonder if it could be an issue on the dat level but have not had the time to test that.

pfrazee commented 7 years ago

No this is probably an issue with some Beaker code. The changes are propagating fine, but the live-reload trigger is failing.

On Fri, Jul 14, 2017 at 12:11 PM, R.J. Steinert notifications@github.com wrote:

Thank for taking a look @pfrazee https://github.com/pfrazee! For what it's worth, @dwblair https://github.com/dwblair and I wonder if it could be an issue on the dat level but have not had the time to test that.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/beakerbrowser/beaker/issues/603#issuecomment-315414064, or mute the thread https://github.com/notifications/unsubscribe-auth/ABNhU25kO0X1GV87TbAt8n8HY6w0HkU4ks5sN6FIgaJpZM4OX39b .

dwblair commented 7 years ago

@pfrazee -- thank you so much for this, and for all the incredible work on Beaker. I'd been talking up Beaker and the decentralized web a lot -- but after @rjsteinert lead me last night through a 'decentralized live publishing' process', I'm 'mind blown' at the possibilities.

Do you have a sense for what the 'speed limit' on this sort of 'live updating' might be? Can dat handle fast data rates / streaming? Wondering now about 'live sensor data' on the p2p web ...

pfrazee commented 7 years ago

@dwblair thanks!

That's an interesting question to unpack. You have 3 main components that will affect performance:

I dont have any hard numbers atm, nor have I experimented much with very large datasets, but anecdotally I can say it has handed live streaming of video (recording a screen) just fine. The crypto operations add a fixed overhead but that's probably not that different than SSL. The other most significant overhead is peer-discovery, since that requires a lookup.

dwblair commented 7 years ago

@pfrazee --Thanks for the thoughtful reply, and parsing out all of the contributions to performance. The ability to live stream video indicates that the kind of sensor data exchange we're considering should be well-supported -- great to hear. Still learning how peer-discovery works, but I'm really amazed at how fast it is already!

@rjsteinert and I are now on a (p2p, Web RTC) call right now talking about where to head with this 'decentralized google docs for dat' idea ... and wondering about what's possible inside dat archives regarding distributing write permissions / keys -- please join us if you like! https://talky.io/beakerbrowser

pfrazee commented 7 years ago

@rjsteinert @dwblair the more I think about it, the more I think your WYSIWYG Wiki demo is a pretty perfect demo for beaker. (WYSIWYWiki?) It shows the ability to write pages with really minimal effort, and those formatting controls you added were so nice

dwblair commented 7 years ago

@pfrazee @rjsteinert -- Saw this yesterday quickly, then found myself on a hike testing 'WYSIWYWiki' out-loud to myself. It's fun to say.

It does seem like using Beaker for wiki / collaborative document editing could be a nice, accessible way for newcomers to develop an intuition for how decentralized file systems work / can work. (It's already been performing that function for us!) And with dat's new mulit-writer protocol coming online, it could be a nice way to demonstrate various approaches to granting editing permissions, versioning, and resolving file conflicts.

I like the point that you've emphasized elsewhere, @pfrazee, that (paraphrasing) a blockchain isn't always necessary to handle decentralized collaborative file editing. One could almost imagine an ecosystem of Beaker wiki apps that take a range of approaches to handling decentralized editing -- from the simple, old-school "I will edit the doc on Monday. Then you edit it on Tuesday", all the way up to "blockchain".

Thanks again for all the awesome ideation, and accessible explanation!

rjcorwin commented 7 years ago

Hi @dwblair @pfrazee - Just coming back to this thread now because I have an update but I haven't read your last two comments, I'm in the middle of building a PoC for Tangerine and Beaker with Polymer Elements + CKeditor for advanced survey building. I updated the beaker wiki concept to use CKEditor because Aloha is actively encouraging itself to not be used. Turns out CKEditor has nice inline functionality these days.

See the repo with links to live examples: https://github.com/rjsteinert/beaker-wiki

pfrazee commented 7 years ago

Nice! That's looking good

On Mon, Jul 17, 2017 at 1:29 AM, R.J. Steinert notifications@github.com wrote:

Hi @dwblair https://github.com/dwblair @pfrazee https://github.com/pfrazee - Just coming back to this thread now because I have an update but I haven't read your last two comments, I'm in the middle of building a PoC for Tangerine and Beaker with Polymer Elements

  • CKeditor for advanced survey building. I updated the beaker wiki concept to use CKEditor because Aloha is actively encouraging itself to not be used. Turns out CKEditor has nice inline functionality these days.

See the repo with links to live examples: https://github.com/rjsteinert/ beaker-wiki

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/beakerbrowser/beaker/issues/603#issuecomment-315676496, or mute the thread https://github.com/notifications/unsubscribe-auth/ABNhUyNWitly5hqXhrG5NXnC3hpyb2nKks5sOv84gaJpZM4OX39b .

rjcorwin commented 7 years ago

@pfrazee We did some thinking tonight how one might approach publishing a form survey and then have folks fill it out in their own fork which causes the form data to write to disk, and then the person who created the form collect that data. Right now, in order for the person who created the form to collect data, folks who fill out the form in their own fork would have to send the dat url to the person who created the URL by way of email or some other method. Perhaps there could be a way for dat archives to know who forked them so the app in the root archive could open all children archives and collect the data. This reminds me of how you can see forks of a repo on Github, yet in many cases it would only be desirable for security reasons for the owner of the root archive to be able to know the children so people who fork and fill out the form app can't collect the data of others. We have lots of ideas on how to accomplish this but lack a foundational knowledge of dat to really know what we are talking/thinking about :P.

pfrazee commented 7 years ago

That's a use-case that we're really bad at right now: push messages. The design you're suggesting is what I call the "outbox method." That's where you create a dat that puts outgoing messages in a folder (essentially an outbox) and then the recipient has to pull the message from the folder.

The outbox method works ok but, like you said, the recipient has to know it needs to pull the data. This may be a usecase that simply relies on traditional services, for now.

On Tue, Jul 18, 2017 at 12:25 AM, R.J. Steinert notifications@github.com wrote:

@pfrazee https://github.com/pfrazee We did some thinking tonight how one might approach publishing a form survey and then have folks fill it out in their own fork which causes the form data to write to disk, and then the person who created the form collect that data. Right now, in order for the person who created the form to collect data, folks who fill out the form in their own fork would have to send the dat url to the person who created the URL by way of email or some other method. Perhaps there could be a way for dat archives to know who forked them so the app in the root archive could open all children archives and collect the data. This reminds me of how you can see forks of a repo on Github, yet in many cases it would only be desirable for security reasons for the owner of the root archive to be able to know the children so people who fork and fill out the form app can't collect the data of others. We have lots of ideas on how to accomplish this but lack a foundational knowledge of dat to really know what we are talking/thinking about :P.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/beakerbrowser/beaker/issues/603#issuecomment-315961401, or mute the thread https://github.com/notifications/unsubscribe-auth/ABNhUwKBx_CGhVAgPFJnBvqiHwpzfmzPks5sPEHAgaJpZM4OX39b .

rjcorwin commented 7 years ago

"outbox method" is a great way to describe that model. I'm also reminded of "pull request" where the owner of the fork requests to the parent they pull something from them (in there outbox). If dat had some permissive messaging protocol between Dats then you could load up a Dat in Beaker, see the requests, approve them, then the application deals with what is sent over wire. That might be saving that data directly into the current Dat or perhaps saving that data into another Dat as to keep the messages from other request senders. The application dealing with the messages is nice because it can decide how to deal with conflicts (as you have mentioned to me).

Use case I'm very much involved in: Decentralized Forms where you want the Form to be public but the responses private to the owner of the Form.

Perhaps there is an existing decentralized messaging protocol that could help mediate these messages? WebRTC perhaps? As you know that requires a signaling server but perhaps Dat can provide those signals. That's easier to do when things are live, storing a request seems like a nonstarter due to DDoS potential of filling up hard drives (somewhere).

rjcorwin commented 7 years ago

Coming back around to the root of this issue, if dat could handle signaling for connecting two BeakerBrowsers over WebRTC, then we may have this performance issue of quick updates nailed down.

pfrazee commented 7 years ago

If dat had some permissive messaging protocol between Dats then you could load up a Dat in Beaker, see the requests, approve them, then the application deals with what is sent over wire

The 'permissive' part is difficult because you have to handle spam.

Coming back around to the root of this issue, if dat could handle signaling for connecting two BeakerBrowsers over WebRTC, then we may have this performance issue of quick updates nailed down.

It's possible that we can solve WebRTC signaling with a DHT, but it's more likely that we'll use a decentralized service model, because you also need hole-punching, fallback proxies, and key-distribution. We'll probably introduce a set of specs and implement in hashbase.

All that said, the downside of using WebRTC to send survey responses is, it requires the device that created the survey to act like a server: it has to be online, and it has to accept the incoming connections. The 'being online' part is a non-starter IMO if we want this tech to "just work" for everybody.

So I think we have to look for some other solutions.

pfrazee commented 7 years ago

Ok I managed to solve the issue. You should see all changes cause reloads now

dwblair commented 7 years ago

Cooooool!

On Thu, Jul 20, 2017 at 4:06 PM, Paul Frazee notifications@github.com wrote:

Ok I managed to solve the issue. You should see all changes cause reloads now

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/beakerbrowser/beaker/issues/603#issuecomment-316815005, or mute the thread https://github.com/notifications/unsubscribe-auth/AAnDENKFKFDffqy_ElV0c6B659BUdJjlks5sP7MpgaJpZM4OX39b .

rjcorwin commented 7 years ago

@pfrazee Don and I have built from master and are about to test out the live editing now. Join us if you like https://talky.io/beaker