Replace the heap with a priority queue

nietaki / crawlie

A simple Elixir library for writing decently-performing crawlers with minimum effort.

MIT License

89 stars 11 forks source link

Replace the heap with a priority queue #9

Closed nietaki closed 7 years ago

nietaki commented 7 years ago

For example this one: https://hex.pm/packages/pqueue

The heap compares the urls as well as the crawling depth. It's doing unnecessary string comparison and it looks weird when the pages go roughly reverse-alphabetically.