issues
search
spatie
/
crawler
An easy to use, powerful crawler implemented in PHP. Can execute Javascript.
https://freek.dev/308-building-a-crawler-in-php
MIT License
2.51k
stars
357
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Laravel and Vapor
#377
Andrew-Hike
closed
2 years ago
1
Best way to limit similar urls
#376
aaronbauman
closed
2 years ago
2
Test against php 8.1
#375
sergiy-petrov
closed
2 years ago
1
Can't install with Laravel 8.57
#374
morawcik
closed
2 years ago
6
Update README.md
#373
mansoorkhan96
closed
3 years ago
0
change response type hint
#371
scrlkx
closed
3 years ago
1
Crawler doesn't push subdomain URLs to queue if CrawlProfile doesn't extend CrawlSubdomains
#370
kejkej31
closed
3 years ago
0
Binary files are parsed, ignoring setParseableMimeTypes
#369
jespejoh
closed
2 years ago
2
Added laravel application as exemple of using laravel model as queue
#368
insign
closed
3 years ago
1
Error install with laravel 6.x
#366
caocanhlinh
closed
3 years ago
0
V7
#365
freekmurze
closed
3 years ago
0
Add PHP 8-only Support (v7)
#364
patinthehat
closed
3 years ago
1
Redirect URLs including info similar to cURL
#361
RobertEcker
closed
3 years ago
0
How do we read the html response in the crawled method of my Observer.
#359
stockarea
closed
3 years ago
0
Bugfix: infinite loops when a CrawlProfile prevents crawling
#358
mattiasgeniar
closed
3 years ago
1
Crawl stops when 1st page is a redirection
#357
flmommens
closed
3 years ago
1
How can I get the response body from crawled?
#355
vFire
closed
3 years ago
0
How Can I get the status code of each response
#354
vFire
closed
3 years ago
1
"The promise was rejected" Exception
#353
flmommens
closed
3 years ago
10
setParseableMimeTypes() not working
#352
flmommens
closed
3 years ago
5
cURL error 28: Operation timed
#351
flmommens
closed
3 years ago
1
[requestException] cURL error 60: SSL certificate problem: unable to get local issuer certificate
#347
GabMic
closed
3 years ago
0
Improve PHPunit fixture
#346
peter279k
closed
3 years ago
0
Fix support for PHP 8.0 in v5
#345
Bapawe
closed
3 years ago
1
Update run-tests.yml
#344
Bapawe
closed
3 years ago
3
Fix support for PHP 8.0 in v4
#343
Bapawe
closed
3 years ago
1
Allow HTML response transformations prior to DOM parser execution
#342
pajoda
closed
3 years ago
1
Undefined method setCurrentCrawlLimit
#341
QuantBits
closed
3 years ago
0
Crawler processes out of scope page after redirect
#340
spekulatius
closed
3 years ago
2
Not install package in framwork yii3
#339
sanchezzzhak
closed
3 years ago
1
test
#338
emreakay
closed
3 years ago
0
v6
#337
freekmurze
closed
3 years ago
0
how to know when a link is found for the second/third... time?
#336
bobemoe
closed
3 years ago
3
Modify html before DomCrawler
#335
posipa
closed
3 years ago
3
Support for PHP 8.0
#334
pascalbaljet
closed
3 years ago
7
Support for symfony v4?
#333
Defcon0
closed
3 years ago
6
Links with query parameters (GET parameters) are only crawled when present in the URL passed to startCrawling()
#332
Defcon0
closed
3 years ago
1
Allow for chunked crawls by not running the queue empty (fixes #325)
#331
spekulatius
closed
3 years ago
32
Fixing namespace in README and reference to CollectionCrawlQueue
#330
spekulatius
closed
3 years ago
2
Readme: Fix wrong namespace for CrawlObserver
#329
bjoernhasse
closed
3 years ago
1
Crawl with php
#328
aditya3008
closed
3 years ago
0
Tweak variable naming in ArrayCrawlQueue
#326
spekulatius
closed
3 years ago
2
Crawling larger sites
#325
spekulatius
closed
3 years ago
7
Browsershot exceptions unhandled
#324
kmcluckie
closed
3 years ago
8
Updating list of Queue options
#323
spekulatius
closed
3 years ago
2
SSL Problem
#322
Tomut0
closed
3 years ago
0
Fix incorrect CrawlProfile namespace in readme
#321
kmcluckie
closed
3 years ago
1
Apply fixes from StyleCI
#320
freekmurze
closed
3 years ago
0
Apply fixes from StyleCI
#319
freekmurze
closed
3 years ago
0
v5
#318
freekmurze
closed
3 years ago
0
Previous
Next