ahernp / django-feedreader

RSS Aggregator
BSD 3-Clause "New" or "Revised" License
91 stars 29 forks source link

can you add options change for time crawl and do crawl now? #14

Closed yasabaz closed 6 years ago

ahernp commented 7 years ago

I presume you mean the poll_feeds Django command which checks all the feeds for updates (run using python manage.py poll_feeds).

On my website I just have a cron entry which runs that command once an hour. There is no scheduling built in to django-feedreader.

Clicking on the down arrow in the interface next to a Group or Feed name should trigger an immediate poll of the Feeds in that Group or of the individual Feed. See the documentation here, references 5 and 6.

I thought it safer not to have an option to poll all the feeds at once on demand because that is quite an expensive process.

FYI, the Feed model also polls brand new feeds on save.

yasabaz commented 7 years ago

i want change period update feeds. any options access ?

ahernp commented 7 years ago

There is no scheduling in this application. You have to use a scheduler (such as cron) to run the poll_feeds command at whatever periodicity you wish. That will keep the feeds up to date.

ssmusoke commented 6 years ago

When I run ./manage.py poll_feeds in my virtual environment I get the following error:

Traceback (most recent call last): File "./manage.py", line 29, in <module> execute_from_command_line(sys.argv) File "/Users/user/Virtualenvs/popsicle/lib/python2.7/site-packages/django/core/management/__init__.py", line 363, in execute_from_command_line utility.execute() File "/Users/user/Virtualenvs/popsicle/lib/python2.7/site-packages/django/core/management/__init__.py", line 355, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/Users/user/Virtualenvs/popsicle/lib/python2.7/site-packages/django/core/management/__init__.py", line 205, in fetch_command klass = load_command_class(app_name, subcommand) File "/Users/user/Virtualenvs/popsicle/lib/python2.7/site-packages/django/core/management/__init__.py", line 40, in load_command_class module = import_module('%s.management.commands.%s' % (app_name, name)) File "/usr/local/Cellar/python/2.7.13_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/importlib/__init__.py", line 37, in import_module __import__(name) File "/Users/user/Virtualenvs/popsicle/lib/python2.7/site-packages/feedreader/management/commands/poll_feeds.py", line 18, in <module> class Command(BaseCommand): File "/Users/user/Virtualenvs/popsicle/lib/python2.7/site-packages/feedreader/management/commands/poll_feeds.py", line 21, in Command option_list = BaseCommand.option_list + ( AttributeError: type object 'BaseCommand' has no attribute 'option_list'

ahernp commented 6 years ago

Looks like BaseCommand.option_list was deprecated in Django version 1.10 and removed in 1.11. If you downgrade to Django 1.10 then this should work. I shall try to update django-feedreader over the weekend to make it work with the latest version of Django.

ahernp commented 6 years ago

Made a fix (1093566361c8c8d70347879d7f11afa9a633f6ff) and pushed a new version 1.3.8 of django-feedreader to PyPI.

ssmusoke commented 6 years ago

@ahernp That was fast and efficient thank you. However now the error is

Traceback (most recent call last): File "./manage.py", line 29, in <module> execute_from_command_line(sys.argv) File "/Users/user/Virtualenvs/popsicle/lib/python2.7/site-packages/django/core/management/__init__.py", line 363, in execute_from_command_line utility.execute() File "/Users/user/Virtualenvs/popsicle/lib/python2.7/site-packages/django/core/management/__init__.py", line 355, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/Users/user/Virtualenvs/popsicle/lib/python2.7/site-packages/django/core/management/base.py", line 283, in run_from_argv self.execute(*args, **cmd_options) File "/Users/user/Virtualenvs/popsicle/lib/python2.7/site-packages/django/core/management/base.py", line 330, in execute output = self.handle(*args, **options) File "/Users/user/Virtualenvs/popsicle/lib/python2.7/site-packages/feedreader/management/commands/poll_feeds.py", line 48, in handle entries = Entry.objects.filter(feed=feed)[settings.MAX_ENTRIES_SAVED:] File "/Users/user/Virtualenvs/popsicle/lib/python2.7/site-packages/django/conf/__init__.py", line 57, in __getattr__ val = getattr(self._wrapped, name) AttributeError: 'Settings' object has no attribute 'MAX_ENTRIES_SAVED'

ahernp commented 6 years ago

Just add the following to your Django settings.py file:

# Feedreader
MAX_ENTRIES_SAVED = 100
MAX_ENTRIES_SHOWN = 100
MAX_DAYS_SHOWN = 1
ssmusoke commented 6 years ago

@ahernp Errors fixed.

I am working with data feeds which do not have title, title_detail and link attributes. Is there anyway I can configure the required attribute check?

ahernp commented 6 years ago

If you are using bespoke feeds then django-feedreader is probably not going to be useful. It is an RSS feed aggregator. title and link are required attributes according to the RSS 2 specification. Even if django-feedreader didn't require them, I expect that the feedparser library it uses would reject such entries as malformed. Sorry.

ssmusoke commented 6 years ago

@ahernp Thanks for all your assistance and now I get it ...