scripting / feedBase

A project to get feeds into a base.
MIT License
33 stars 3 forks source link

Sunday work #9

Open scripting opened 6 years ago

scripting commented 6 years ago

It's going to be a lite day today, lots of sports on TV -- Knicks at noon (they might actually win vs Atlanta) and of course the SuperBowl in the evening.

So here's what I'm working on right now...

Keeping the feeds table updated

I want to read feeds every so often to make sure the data in the feeds table is up to date.

The first question is which feed has been least recently checked.

SELECT feedurl, whenupdated FROM feeds ORDER BY whenupdated ASC LIMIT 1;

That's the feed I will read, and will set its whenupdated value so that is the most recently updated.

There will also be a ping function, so if you update your feed and you want feedBase to read it immediately you can tell us to do that.

scripting commented 6 years ago

Okay I've got my work done for the day -- v0.4.18 is up.

Big change in this release -- we're now regularly updating the info we have about the feeds, one update every five seconds. That's of course configurable. We may need to decrease the number as there are more feeds. For now this seems like enough.

Added two new columns to the feeds table: code and ctsecs, which are the HTTP code for the most recent request, and the number of seconds, as a floating point number, that the request took. I want to add more info, the number of reads, number of consecutive errors. It's a beginning of helping spot feeds that are broken or no longer updating. Certainly something feedBase should track.

Also started a struct for server-wide stats, to be written as stats.json.

scripting commented 6 years ago

Knicks lost again, this time to the worst team in the NBA. I guess that makes them the worst now?

Anyway I added more stats, so I can better watch what's going on on the server.

http://feedbase.io/stats

Dave