Handle when more than 1000 new entries are found for a feed

cannot get more than 1000 keys in a single call
Traceback (most recent call last):
  File
"/base/python_lib/versions/1/google/appengine/ext/webapp/__init__.py", line
509, in __call__
    handler.post(*groups)
  File
"/base/data/home/apps/pubsubhubbub/injector.335461899033304081/main.py",
line 252, in decorated
    return func(myself, *args, **kwargs)
  File
"/base/data/home/apps/pubsubhubbub/injector.335461899033304081/main.py",
line 1918, in post
    if parse_feed(feed_record, headers, content):
  File
"/base/data/home/apps/pubsubhubbub/injector.335461899033304081/main.py",
line 1818, in parse_feed
    feed_record.topic, format, content)
  File
"/base/data/home/apps/pubsubhubbub/injector.335461899033304081/main.py",
line 1728, in find_feed_updates
    topic, entries_map.keys())
  File
"/base/data/home/apps/pubsubhubbub/injector.335461899033304081/main.py",
line 950, in get_entries_for_topic
    for entry_id in entry_id_list])
  File "/base/python_lib/versions/1/google/appengine/ext/db/__init__.py",
line 868, in get
    results = get(keys)
  File "/base/python_lib/versions/1/google/appengine/ext/db/__init__.py",
line 1095, in get
    entities = datastore.Get(keys)
  File "/base/python_lib/versions/1/google/appengine/api/datastore.py",
line 222, in Get
    raise _ToDatastoreError(err)
  File "/base/python_lib/versions/1/google/appengine/api/datastore.py",
line 2100, in _ToDatastoreError
    raise errors[err.application_error](err.error_detail)
BadRequestError: cannot get more than 1000 keys in a single call

Original issue reported on code.google.com by bslatkin on 18 Aug 2009 at 9:12

Tallefer / pubsubhubbub

Handle when more than 1000 new entries are found for a feed #51