search

spatie / crawler

An easy to use, powerful crawler implemented in PHP. Can execute Javascript.
https://freek.dev/308-building-a-crawler-in-php
MIT License
2.51k stars 357 forks source link

Check original URL against depth tree when visited link is a redirect #467

Closed superpenguin612 closed 1 month ago

superpenguin612 commented 2 months ago

Fixes #466.

Could, in theory, pass only the original link to addToDepthTree instead, if a technically breaking change is undesirable.

freekmurze commented 2 months ago

Is it possible to add a test for this to make sure the problem is fixed by this code?

superpenguin612 commented 1 month ago

The latest commit adds a test that fails on main, but succeeds on this branch.

freekmurze commented 1 month ago

The tests are failing. Could you take a look?

superpenguin612 commented 1 month ago

Sorry about that!

freekmurze commented 1 month ago

Thanks!