DocNow / diffengine

track changes to the news, where news is anything with an RSS feed
MIT License
177 stars 30 forks source link

Document how to work with multiple feeds #10

Closed ruebot closed 7 years ago

ruebot commented 7 years ago

I have accounts setup, and a couple of have many urls for RSS feeds. Not sure if I have everything setup right. So, we should probably document the best way to setup multiple accounts, and an account that has multiple RSS feeds in it.

Happy to do this work.

I have each account setup in it's own home directory:

Toronto Sun has multiple RSS feeds, and I have config.yaml setup like so:

- name: Top Home stories
  twitter:
    access_token: SOMETHING
    access_token_secret: SOMETHING
  url: http://www.torontosun.com/photos/rss.xml
- name: Top Home stories
  twitter:
    access_token: SOMETHING
    access_token_secret: SOMETHING
  url: http://www.torontosun.com/videos/rss.xml
- name: Top Home stories
  twitter:
    access_token: SOMETHING
    access_token_secret: SOMETHING
  url: http://www.torontosun.com/sunshine-girl/rss.xml
...
phantomjs: phantomjs
twitter:
  consumer_key: SOMETHING
  consumer_secret: SOMETHING
edsu commented 7 years ago

Yes, that is correctly set up! You are right, it's a guessing game for anyone trying to figure it out.

I'm wondering if it might be good if diffengine evolved some commands?

% diffengine add-feed
% diffengine run
etc ...
ruebot commented 7 years ago

Cool. I can add some documentation to the README if you'd like.

I do like the idea about addition commands!

edsu commented 7 years ago

Also, there is a tradeoff. The nice thing about keeping the configurations and databases separate as you had them is that they can run independently. As you accumulate more and more entries it can take a while to run sometimes, which causes it to fall behind (see #9). Also #4 becomes more important, depending on what your cron schedule looks like.

ruebot commented 7 years ago

Resolved with: https://github.com/DocNow/diffengine/commit/4ee2e7605bf7dbb13da76b8bdacca91ee5f27803