scripting / river5

A river-of-news RSS aggregator in JS running in Node.
MIT License
428 stars 122 forks source link

Tom Critchlow wants to run River5 on Glitch #22

Open scripting opened 6 years ago

scripting commented 6 years ago

Tom Critchlow wants to run River5 on Glitch. Has anyone had any success with this?

https://twitter.com/tomcritchlow/status/988452117823348736

Dave

SparkyT commented 6 years ago

Yes to some extent :-) - http://newriver5.glitch.me/

Steps:

tomcritchlow commented 6 years ago

@SparkyT amazing! Thank you. I'm gonna check this out. Thanks

scripting commented 6 years ago

Questions for @SparkyT --

  1. Thanks for helping us out. That's the first thing I want to say. Now the questions.

  2. Someone told me that background tasks don't work on Glitch. Not sure what this means. River5 couldn't work without being able to set up some calls using setInterval.

  3. It looks like your server reset in the last hour or so, started reading feeds from scratch? Not sure if you restarted it, or something went wrong. Have you been using this River5 installation or is it something you set up as an experiment?

Dave

scripting commented 6 years ago

Looking at the dashboard I can answer one question --

http://newriver5.glitch.me/dashboard

The server has been up for 2.1 hours. :-)

Dave

SparkyT commented 6 years ago

Isn't it always the way, you have something running nicely, share it and it all breaks :-( ?

I'm not sure how Glitch deals with the attempts to run continuously but it does go to sleep after a while and then completely rebuild/recreate the feeds when I go back to it.

It appears to have filled up the assigned space on the newriver5 instance (maybe due to it being up longer than my tests previously made happen). Getting it back up and working seems to require some wrangling which I will attempt now.

scripting commented 6 years ago

Thanks — let’s figure out what’s needed to keep it running smoothly. Maybe we could have the client call back periodically to keep it running.

scripting commented 6 years ago

I added a one-line script here that loads a quick page on your server every five minutes.

http://newriver5.glitch.me/now

SparkyT commented 6 years ago

Timezones meant I had to leave the conversation but the ride to work this morning triggered a memory or two

scripting commented 6 years ago

Yes, I came to the same conclusion.

http://scripting.com/2018/04/24/124932.html

I like the way you used the feedBase top 100. Very inventive. ;-)

Dave

cori commented 6 years ago

I don't know anything about the configuration available for River5, but if the app's data could be stored in .data instead of in data, glitch maintains the content of that folder when the container is restarted, and it's also not copied on remix (so a remix wouldn't start out with someone else's River5 data folder contents).

scripting commented 6 years ago

Cori, that makes perfect sense, and River5 is totally configurable this way.

Here are the docs for config.json --

https://github.com/scripting/river5/blob/master/docs/CONFIG.md

If @SparkyT is listening --

All you should have to do is change the value of dataFolder in config.json to have the value ".data/".

All the data folders that River5 creates or looks for are in that folder.

Perfect. I like the way the designers at Glitch think.

Dave

cori commented 6 years ago

Great! I thought it might be but didn't see it in newriver5.glitch.com's config.json at a quick glance.

For reference there have been a number of efforts to port River5 to Glitch previously: https://glitch.com/search?q=river5

scripting commented 6 years ago

Cori, that means he's accepting the default. You only have to put things in config.json if you want to override the defult.

Adding this one line to his config.json should fix this problem --

"dataFolder": ".data/",

That should cause it to create all its data in a sub-folder of the Radio5 folder called .data

Dave

SparkyT commented 6 years ago

Thanks all, dataFolder change made and stuff is being written there.

scripting commented 6 years ago

The app seems to be gone now.

I still wonder if what it's doing in the background is kosher.

I would search for setInterval in the Glitch docs and see if it has any problems with running code from those threads, or what happens to them when they shut off an app.

It could be they shut River5 down when it's in the middle of writing a file. That could definitely screw things up.

scripting commented 6 years ago

@cori -- I've been emailing with Anil Dash, and he says you work at Glitch now. That's really interesting. Maybe there's more we could do here. I told him that the .data approach is very good design, and it dovetails perfectly with the design of my apps. I guess we all have enough experience to have a sense for how data and code mix or don't mix. ;-)

@SparkyT -- thanks for sticking with this! I just checked the demo app, and it seems to still be running. That's really interesting. We now have a free place to deploy River5, and if there are any performance limits, it's really hard to see them.

Before closing this issue, I'd like to if possible work with you a bit on getting a really good clonable project there, a good demo for people to build from. I think all this can be done through work on config.json, and the lists folder, and writing a bit of docs.

The first question I have is how did you get the feedBase list in there? Did you take a snapshot? Or did you link to it from within a one-item list (that's the way to do it). I can help if you need it.

Also in config.json you can give titles to the tabs. The one for feedBase is awkwardly long. There are docs on this, I think. I'll go have a look now.

Anyway -- great teamwork! I love the braintrust. You guys are great. :hearts:

cori commented 6 years ago

@scripting it's great that you're in contact with Anil. I do work at Fog Creek, although not on Glitch at the moment - my apologies if I wasn't diligent enough with a disclaimer; I didn't intend to muddy the waters. We've definitely got some really smart people working on Glitch!

Also for posterity if anyone runs across this issue in the future; you can remix the newriver5 project from https://glitch.com/edit/#!/newriver5 (since it can be a little hard to get from the public view to the editor without a little help).

SparkyT commented 6 years ago

@scripting, yes - happy to tweak the setup so it works as a good template for people, just let me know what you want me to change or alternatively I can send you a link to collaborate on Glitch if you want to take it on?

The feedBase river is done with a single type="include" line in an opml file - I was going to ask whether you had implemented this, checked the docs one more time and there it was starting at me.

I wondered whether the river files should be put into .data/rivers as well so that they aren't copied over when people remix the project? Keeping the lists in the default folders seems a good compromise as then people who remix will see what lists are being used & how they are implemented.

tomcritchlow commented 6 years ago

I just want to chime in as the n00b who originally sparked this idea and say thank you! Looks like you're getting close to something that gives a beginner like me a way to get up and running with river5 in a few clicks which I think is a great thing. Really appreciate it.

scripting commented 6 years ago

@cori -- I didn't need a disclaimer. I recognize your name as a long-term Scripting News community person, I just wanted to close the loop. Totally appreciate the help. Please continue. ;-)

@sparkyt -- I'm glad you're using the include type. That's the right kind of example to propogate. I want all feed readers to have that feature! I know it's a tall order, but it's a good kind of network to build, really creates choice for users. You can send the same set of subscriptions to a variety of apps. As you did.

Anyway -- I think the only thing that needs to be done is edit the titles of the tabs on the home page. That's done through config.json, and it is documented here --

https://github.com/scripting/river5/blob/master/docs/CONFIG.md#configuring-the-home-page

I would prefer if you do the editing. ;-)

Let's shorten the first tab to "feedBase 100".

Name the second tab "World News" because that's what it seems that it is.

I guess the third tab is a sample list? Not sure what to call it. Maybe just eliminate it from the homePage array? Not sure. Two panels is probably enough for a demo.

You're right about the rivers folder, but 1. that's a change to the codebase, not the configuration and 2. it corrects itself almost immediately, overwriting the rivers with the ones it builds. So it's a "nice to have" thing, but I don't think it's an immediate thing to do (if ever).

@tomcritchlow -- you provided a reason to explore this, I wanted to know what Glitch was, and this is the most meaningful way for me to find out, make one of my apps work there. And you get an app to play with. I'm also a big believer in working together, so look at all the interesting people this has brought together, so that's the really big win. ;-)

scripting commented 6 years ago

@SparkyT -- here's a better idea

This is what the virgin River5 tab setup looks like

image

Make yours look like that. You could have your version of the Main tab point to the feedBase 100 OPML file. I really love that idea and want to preserve it.

Otherwise the NYT and Hacker News provide a pretty good set of extremes for a demo.

Dave

SparkyT commented 6 years ago

OK, I'm adding the NYT and HN tabs back in so the app may disappear temporarily if they produce to much information for the Glitch limited space. If it does, I'll truncate the lists slightly to deal with it.

I saw the response to #23 which is great news as getting the https version working will be great given that's the default when one clicks "view" in Glitch. There is one other issue which I'll log separately once I can describe it sensibly relating to secure web sockets (wss://).

Tim

scripting commented 6 years ago

@sparkyT -- I made the changes to eliminate the errors from the document.write includes.

I found a simple way to address it that I hadn't thought of earlier, since all the content is stored in S3 I can use their version of the address, which does support https. I didn't really need to use a CDN to get that feature. Wish I had thought of it earlier, could have saved a bunch of time.

There's another issue we should fix for the Glitch version, turning off the websockets interface. It's not working, and throwing a lot of errors.

The way to turn it off is to add this to config.json:

"flWebSocketEnabled": false,

Let me know when you do it, and I'll have a look at the remaining errors.

Dave

yudel commented 6 years ago

It's no doubt premature, but I cloned the River5 installation using Glitch's clone feature. I changed some of the feeds. https://jazzy-pike.glitch.me/ reads the feeds, according to the dashboard, but the page itself shows blank.

Once this is all working , I'm eager to write a River5 for Poets guide for setting up a customized river of news on Glitch.

scripting commented 6 years ago

It looks like you cloned it before the last fixes. Try cloning again.

On Wed, Apr 25, 2018 at 3:40 PM yudel notifications@github.com wrote:

It's no doubt premature, but I cloned the River5 installation using Glitch's clone feature. I changed some of the feeds. https://jazzy-pike.glitch.me/ reads the feeds, according to the dashboard, but the page itself shows blank.

Once this is all working , I'm eager to write a River5 for Poets guide for setting up a customized river of news on Glitch.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/scripting/river5/issues/22#issuecomment-384409564, or mute the thread https://github.com/notifications/unsubscribe-auth/ABm9Oyi4Agcv7vVagRh2FbF1hnnIx5k-ks5tsNEigaJpZM4TgVlz .

yudel commented 6 years ago

And we're live at https://dew-quarter.glitch.me/ One sad thing I discovered is that many sites I'd like to include no longer have feeds.

SparkyT commented 6 years ago

Thanks, "flWebsocketEnabled": false is set but strangely doesn't appear to be stopping the websocket connection attempts in the console when I view it.

scripting commented 6 years ago

@SparkyT -- I fixed the problem where it was trying to connect to the websocket server if it's not enabled. It was literally a one-line fix. ;-)

To get the fix, just reload the page.

SparkyT commented 6 years ago

@scripting - thanks, no more websocket errors :-)

I think this is OK enough now, let me know if you want anything else changing. The river5 name on Glitch isn't one of the ones I own I'm afraid so I can't switch to it. I'm glad we could get this working :-)

yudel commented 6 years ago

So it's working, but the river that I added (by cloning myJsonFeeds.json and myJsonFeeds.js and adding myJsonFeeds to config.json) was read in once but isn't being updated. Any suggestions?

SparkyT commented 6 years ago

@yudel , could you post the link to your remix and I'll have a quick look?

It may be something to do with the age of the newriver5 instance which might be affecting how somethings work when remixed - I'll create a fresh one this & send you that to try as well.

Tim

scripting commented 6 years ago

BTW -- @sparkyt -- it seems to still be working. When I log on I check the dashboard and see that it's been started a few seconds before. Meaning it's relaunching on every hit, potentially (I think they say they put it to sleep after five minutes of inactivity).

Then you have to reload the page a few times to get the really new stuff. It almost always finds new stuff after a period of inactivity.

This is a totally acceptable, and realistic use-case.

However it also says that having the websockets interface working would be a good thing here.

It might be worth finding out at some time why it wasn't working.

yudel commented 6 years ago

I should note that the the other rivers work fine. The site is https://dew-quarter.glitch.me/

You can edit it at https://glitch.com/edit/#!/join/c4461cfd-e9a1-4a35-9aeb-be17dcf4147b

yudel commented 6 years ago

Found the bug: An extra comma at the end of the list of feeds in myJewsNews.json

tomcritchlow commented 2 years ago

I know it's been a few years but I'm still interested in this idea! Since this thread kicked off:

Most of the Glitch links in this thread unfortunately won't load for me. Would love a reference level Replit/Glitch instance that can be easily remixed and built on top of

Update: I tried to install river5 directly from Github into a Replit and it appears to work out of the box! No idea what happens over time.... https://river5.tomcritchlow.repl.co/