spatie / crawler

An easy to use, powerful crawler implemented in PHP. Can execute Javascript.
https://freek.dev/308-building-a-crawler-in-php
MIT License
2.53k stars 358 forks source link

Added laravel application as exemple of using laravel model as queue #368

Closed insign closed 3 years ago

insign commented 3 years ago

The main reason is the others queues packages store all items in one single array, which can be a RAM problem for big sites. Furthermore, you can preserve and use crawled links as you want.

freekmurze commented 3 years ago

Thanks!