search

bitsoffreedom / newspeak

Newspeak van de Nederlandse overheid.
https://rejo.zenger.nl/inzicht/newspeak-van-de-nederlandse-overheid
BSD 3-Clause "New" or "Revised" License
9 stars 4 forks source link

No full update / disable XPath extraction when adding feeds #25

Open dokterbob opened 11 years ago

dokterbob commented 11 years ago

Currently, a full update is run when adding feeds through the admin. When XPath extraction is used, this can cause significant blocking/freezing of the admin because for all imported entries the links have to be followed.

Possible solutions:

  1. Disable xpath fields in the admin create form.
  2. Factor out feed metadata (title etc.) extraction code from the crawler and only apply this part rather than the full blown extraction.

I suppose 2 is the more elegant here.