1d2d0b76811b1e41ae4cea2b306ebee6ccdc611d so that failing to insert snapshot will rollback updates on the article.
changes up to 82c6f554eab8eb26826e0db21d8e6a3eb0da4712 so that pipeline does not depend on spider.name any more, making it feasible to add more spiders so long as they allow the protocol.
Stop taking snapshots if hit 404 by setting next_snapshot_at to 0 as in other cases. Also made spiders and db pipeline better separated.