Web Browser News - Githubissues

GreenLunar commented 8 years ago

News section under Announcements section, similarly to the one that was on USO.

New news items to be featured would be Qupzilla, Tampermonkey, Violentmonkey, and WebExtensions.

I think that updates in concern to WebExtensions and the deprecation of XUL API would certainly fit.

Martii commented 8 years ago

Still hunting for a suitable RSS/Atom feed "puller" as well as "pusher" package... until this is assigned you are welcome to go visit https://npmjs.com , do some light reading, and give some recommendations. :)

GreenLunar commented 8 years ago

I find it appropriate to ask, with roughly 600 results for "rss" and over 1000 results for "atom", what do you want me to look up for?

RSS/Atom feed "puller" as well as "pusher" package.

If I understand correctly, you want a package that would download (pull) RSS/Atom feeds and create (push) a feed that would join all the downloaded feeds, is that correct?

Would you accept more than a package to do this task?

Martii commented 8 years ago

Hehe now you see my dilemma in past searches... lots of results and I have other coding tasks to take care of that are of higher priority... basically we need a fully configurable package that preferably has it's own cache, but not mandatory... if we have to make a model for it we will, ... configurable feed urls, limits, etc. This is generally how the node community works... each project handles smaller "things" well and the community at large pulls it all together... assuming you didn't know this and I do constantly teach.

This is probably a later label but perhaps something you've seen somewhere else might be good... so IDK yet.

GreenLunar commented 8 years ago

I think, I have found a nominee. https://github.com/nikezono/node-combine-rss https://www.npmjs.com/package/combine-rss

GreenLunar commented 8 years ago

assuming you didn't know this and I do constantly teach.

Please do assume so. Your elaborations and explanations are useful to me.

Martii commented 8 years ago

https://www.npmjs.com/package/combine-rss

Some observations:

~3 years since last update... yellow flag
No Licensing detected in initial sweep... red flag... must be FOSS and GPL cooperative (most OSI is if it's a module/package)
Coffee script based (see below ref) ... red flag
Not much activity for updates... means one of two things... either it's really good or really bad/abandoned ... white flag

Misc:

We don't mind if it's not "popular" with downloading stats as long as it's stable ... white flag
Try to avoid jQuery only based packages... it's starting to negatively impact newer browsers in a bad way... yellow flag if only jQuery js

Generally with packages we like to start with middleware first, then asynchronous, and dead last synchronous packages... all JavaScript (ECMAScript based). "On the fly" is recommended because we would like to be able to use a JSON to configure it dynamically.

The "flags" are my own internal system and is usually what determines if a package if viable or not. We used to have a basic html sanitizer but it was quite limited before I came on board... now we have the top rated, most supported, nice professional maintainers, good documentation, good activity, stability, etc.

Refs:

https://github.com/OpenUserJs/OpenUserJS.org/blob/master/STYLEGUIDE.md#restrictions (see coffee script bases)

There are a lot of ideas that we share from USO that I would like to implement here but every spare moment of free time is taken with the labeled priorities and other duties... so any help is much appreciated. The other members have their own lives too but contribute when they can. The TOS pretty much explains it all... progress can be slow at times and faster at other times.

GreenLunar commented 8 years ago

rss-braider

Braid/aggregate one or more RSS feeds (file or url) into a single feed (RSS or JSON output). Process resulting feed through specified plugins. Automatic deduplication

License: MIT

https://github.com/KQED/rss-braider https://www.npmjs.com/package/rss-braider https://www.npmjs.com/search?q=combine%20rss

feedmix

Merge RSS 2.0 feed(s) into a feed.

License: MIT

https://github.com/hail2u/node-feedmix https://www.npmjs.com/package/feedmix https://www.npmjs.com/search?q=rss&page=5

Congregator RSS-feed Reader

This is an attempt to make a rss-feed parser that can parse the html content of an rss-entry as well as the entries themselves. It will give you an output of formatted JSON-articles which can be processed at will. It also has the ability to visit the article links and fetch a processed version of the content (done by node-read). Now go and build your own feedly! :)

License: MIT

https://github.com/eiriklv/congregator-rssreader https://www.npmjs.com/package/rssreader https://www.npmjs.com/search?q=rss&page=5

~~json2rss~~

~~Converts a json file into an RSS feed~~ ~~Quick and dirty RSS generator from JSON files~~ ~~Built for use with Harp server and static site generation~~

~~License: BSD~~

~~https://github.com/trodrigues/json2rss~~ ~~https://www.npmjs.com/package/json2rss~~ ~~https://www.npmjs.com/search?q=rss&page=5~~

zzparser

feed to json

License: Unknown

https://github.com/vflash/zzparser https://www.npmjs.com/package/zzparser https://www.npmjs.com/search?q=rss&page=4

Extra resources http://vflash.ru/ http://zzreader.com/

Update: April 29th, 2016 Continue from https://www.npmjs.com/search?q=rss&page=8 at absolution by boutell

Stopped https://www.npmjs.com/search?q=rss&page=11 at jstoxml by davidcalhoun ( 638 results for ‘rss’ ) Stopped https://www.npmjs.com/search?q=planet&page=11 at 25th-style by jauer ( 212 results for ‘planet’ )

opml-generator and opml-generator2

Generate opml from object.

License: MIT

https://github.com/azu/opml-generator https://www.npmjs.com/package/opml-generator2 https://www.npmjs.com/search?q=rss&page=8

https://github.com/azu/opml-generator https://www.npmjs.com/package/opml-generator https://www.npmjs.com/search?q=rss&page=9

reactifier

A blog post aggregator bringing together React developers from across the globe.

License: MIT

https://github.com/bkonkle/reactifier https://www.npmjs.com/package/reactifier https://www.npmjs.com/search?q=rss&page=9

feedtitles

Extract titles from rss feeds (npm) Extract titles from an rss feed (git)

License: MIT

https://github.com/TJkrusinski/feedtitles https://www.npmjs.com/package/feedtitles https://www.npmjs.com/search?q=rss&page=9

manger

The manger Node package caches RSS and Atom formatted XML feeds using LevelUP. It provides an interface to query entries by feed and time.

License: MIT

https://github.com/michaelnisi/manger https://www.npmjs.com/package/manger https://www.npmjs.com/search?q=rss&page=10

molecule-js

RSS / Atom Blog Feed Aggregator

License: MIT

https://github.com/jahed/molecule-js https://www.npmjs.com/package/molecule-js https://www.npmjs.com/search?q=planet&page=2

cupid

An alternative of venus the feed aggregator

License: MIT

https://github.com/dotnil/cupid https://www.npmjs.com/package/cupid https://www.npmjs.com/search?q=planet&page=3

name

description

License: license

https://github.com/ https://www.npmjs.com/package/ https://www.npmjs.com/search?q=rss&page=

GreenLunar commented 8 years ago

No Licensing... red flag... must be FOSS and GPL cooperative (most OSI is if it's a module/package)

For record, combine-rss is licensed under BSD.

GreenLunar commented 8 years ago

~~It feels, to me, that Congregator RSS-feed Reader would be a good choice.~~ Perhaps, not; we mostly need to display titles, so I think.

GreenLunar commented 8 years ago

Would you accept a command-line feed reader package, and generate a feed with another package?

Readers https://github.com/yeexel/node-reader https://github.com/lorefnon/amelia

Generators https://github.com/JamieREvans/node-rss https://github.com/dylang/node-rss https://github.com/exeto/fullrss

Note to myself, I have stopped at page https://www.npmjs.com/search?q=rss&page=8 at absolution by boutell

Martii commented 8 years ago

Would you accept a command-line feed reader package

The general terminology across the board for this type is CLI (Command Line Interface)... as long as it has a matching API as well, that we can programmatically use in node, it's fine if it has a CLI tool too... however I won't be adding a cron task in the VPS for CLI updating nor the process manager... the task needs to be done in our code via node.

... I have stopped at page...

Well you've given quite a few... very much appreciated and thank you... I'll spend some time checking them out... one by one... in between other issues. :)

Martii commented 8 years ago

Hmmm I am seeing a pattern here with the current candidates... all seem to be missing Atom support. We need both RSS 2.0 and Atom aggregation... and would prefer it in one package if possible.

A newer RSS feeds aggregator at https://github.com/filipedeschamps/rss-feed-emitter ... tested okay with Atom even though it's not mentioned anywhere but the keywords. hmmm. No limits though... that could get a little noisy (traffic wise).
Combo at https://github.com/fent/node-feedsub ... single feed pull; would require aggregation with multiple feeds... has "smart" limits... untested yet

GreenLunar commented 8 years ago

Please do let me know what is missing and I will resume seeking. Eventually, I will lookup for a package that does it all.

What are has "smart" limits?

Martii commented 8 years ago

What are ... "smart" limits?

https://github.com/fent/node-feedsub#new-feedsubfeed-options ... see maxHistory, skipHours, skipDays, etc... e.g. general limits on when/what not to pull. Also the current description at https://github.com/fent/node-feedsub#feedsub which says:

It works by checking the feed every once in a while, comparing the date of the document via a conditional GET if supported. Otherwise it looks for a date tag in the feed. If it's the same as the last date, it stops downloading it and parsing the xml/json. If it's an updated document, then it looks through it top to bottom taking note of all the new items. Once it finds something it has already read, it stops downloading and parsing the document.

... stopping parsing of the document download is a very big plus!

One problem I see currently with all of them so far is when I restart the server it has to download everything again in order to store the "history" in memory... this is where we can model in something for "last date" and some other API method/property from a package/project. e.g. don't continue to pull earlier than this date/time which seems to be missing for a manual specification... although skipDays might be transformable from a "last date" that we store.

GreenLunar commented 8 years ago

I think it might be better to have a feed, either RSS or Atom for #914 (how is the search going over there btw) ... however...

I thought you find an appropriate module already.

I think I should gh-wiki. This thread is too big.

Martii commented 8 years ago

I think I should gh-wiki. This thread is too big.

No, the issue discussion stays here... the wiki isn't meant for that.

I thought you find an appropriate module already.

I'll ignore that ... you know exactly what I've been up to and my patience is thin today.

GreenLunar commented 8 years ago

I apologies. I will continue to seek for it, weekly. Page https://github.com/OpenUserJs/OpenUserJS.org/wiki/RSS was deleted.

GreenLunar commented 8 years ago

In concern to the following resources:

https://github.com/filipedeschamps/rss-feed-emitter https://github.com/fent/node-feedsub

Would it help to open issues in concern of lack of features that require for our (this) issue?

GreenLunar commented 8 years ago

There are two packages that seems to be relevant, but links to repositories are not valid.

@krl https://www.npmjs.com/package/rssb https://github.com/krl/rssb

@nfour https://www.npmjs.com/package/feedman https://github.com/nfour/feedman

Gentlemen, kindly provide up to date links, if you may.

GreenLunar commented 8 years ago

I find it appropriate to ask, with roughly 600 results for "rss" and over 1000 results for "atom", what do you want me to look up for?

There are only 21 results for "opml".

https://www.npmjs.com/package/planet-feeds https://github.com/planet-templates/planet-feeds

feeds - planet template pack - rss 2.0, atom, opml, and friends

(written in .erb)

https://www.npmjs.com/package/reading https://github.com/mikekreuzer/Reading

A minimal RSS/Atom feed reader

(written in Javascript)

The latter seems to be relevant.

GreenLunar commented 8 years ago

Is it possible to combine keywords in search?

https://www.npmjs.com/search?q=planet%20feed (1 result) https://www.npmjs.com/search?q=planet%20rss (no results) https://www.npmjs.com/browse/keyword/rss

Added more potential packages

Update: April 29th, 2016 Continue from https://www.npmjs.com/search?q=rss&page=8 at absolution by boutell

Stopped https://www.npmjs.com/search?q=rss&page=11 at jstoxml by davidcalhoun ( 638 results for ‘rss’ ) Stopped https://www.npmjs.com/search?q=planet&page=11 at 25th-style by jauer ( 212 results for ‘planet’ )

Martii commented 8 years ago

Would it help to open issues in concern of lack of features that require for our (this) issue?

Always provided they are receptive to that... licensing is a key as I previously mentioned too... no CLA's too as those are unenforceable and something we don't want for maintaining.

Is it possible to combine keywords in search?

Again you see the issue that I was having... npmjs.com has search related issues. I appreciate your traversing through these and doing a pre-analysis so we can weed out the bad ones and concentrate on a good supported one when it's found. :)

Martii commented 8 years ago

https://github.com/planet-templates/planet-feeds

Mailing list is dead e.g. gone :-1: Not to mention ".erb". (whatever that is)

https://github.com/mikekreuzer/Reading

Uses a banned sub-dep; a little overkill too since we really just need the urls and some "smart" limits to maintain traffic. :-1: (Core is feedparser)

GreenLunar commented 8 years ago

Stopped https://www.npmjs.com/search?q=rss&page=11 at jstoxml by davidcalhoun ( 638 results for ‘rss’ ) Stopped https://www.npmjs.com/search?q=planet&page=11 at 25th-style by jauer ( 212 results for ‘planet’ )

I should have done a list of modules that I have seen. Results are 646 (was 638) and 216 (was 212). I hope I have not skipped any results. Do you have a way to aggregate a list of results from npm?

Martii commented 8 years ago

Do you have a way to aggregate a list of results from npm?

The whole of JavaScript under node and all of our current dependencies. If we need storage we can allocate a DB model (BSON) or a file system JSON ... but noise level via traffic will always be a concern. Probably an egg timer or a user initiated sweep on a period could be used to update. The storage mediums are outlined over here... currently in red.

GreenLunar commented 8 years ago

@Martii, I want to try to add this feature. Do you have a recommended resource to learn node.js?

I know basic Javascript. Should I learn jQuery as well?

Martii commented 8 years ago

Do you have a recommended resource to learn node.js?

Not really. However it depends on your learning curve. Some are reference learners (that's me) and others are tutorial style (usually not me).

Should I learn jQuery as well?

jQuery is only on the front end DOM however at some point when I have time it will be converted to plain JavaScript. Some dependencies use jQuery which is unfortunate. There is also "jQuery slim" which I ran by a couple of weeks ago and haven't had time to see what it is. So for OUJS concentrate on real JavaScript but still keep jQuery in mind.

GreenLunar commented 8 years ago

I am a reference learner, but I am currently referring to tutorials.

Martii commented 8 years ago

I am currently referring to tutorials.

Try this... haven't visited those yet but may be something for you:

https://nodejs.org/en/docs/guides/

Search engines too. I'll help when I can.

sjehuda commented 9 months ago

Because we only need title and link, and an (x)HTML parser should be built in, I think this task can be done with no additional libraries.

I would like to work on this task too.

Martii commented 9 months ago

@sjehuda

I would like to work on this task too.

You are assigned.

Just as a, possible reminder, note since GHacks.net has increased their news count we will need a limiter and probably some sort of filtering as we really don't want advertising to bleed through. Same goes for any site that provides a RSS/Atom/other feed.

sjehuda commented 9 months ago

On Sun, 03 Dec 2023 09:35:04 -0800 Marti Martz @.***> wrote:

@sjehuda

I would like to work on this task too.

You are assigned.

Thank you.

Just as a, possible reminder, note since GHacks.net has increased their news count we will need a limiter and probably some sort of filtering as we really don't want advertising to bleed through. Same goes for any site that provides a RSS/Atom/other feed.

Then we can add an approval mechanism.

The news items will be aggregated and the OUJS administrator will get a notification for each item, in both forms, email and as a notification in the UI of OUJS.

I like the idea of filtering, which I myself need to improve in my own RSS bot (Slixfeed).

I think filtering would be an extra feature to colour/mark suspicious items to ease on the administrator.

Martii commented 9 months ago

OUJS administrator will get ...

I'm thinking of relegating this to Moderator and up... I literally have too much on my plate... forwarding up a questionable item may be Admin and up. Since I also maintain the back end and migration I run out of time constantly.

email ...

The VPS in its current state will not be configured this way for some time. i.e. no emailing at this time.

OpenUserJS / OpenUserJS.org

Web Browser News #914