[crawler] Skip static/media paths as it doesn't makes sense to crawl them

[tino]

This could also be implemented with an option for the management command if people do depend on / like crawling the static files.