search

hedii / php-crawler

A php crawler that finds emails on the internets
MIT License
134 stars 65 forks source link

Empty list of emails #31

Closed sn4f closed 7 years ago

sn4f commented 7 years ago

Hi! I have problem with crawling. When I put url and start crawler I always get 0 emails and 0 urls. Dont see any errors on laravel logs.

sn4f commented 7 years ago

Ok this problem is only on https pages

hedii commented 7 years ago

What is the exact url ?

sn4f commented 7 years ago

for example https://katalog-firmy.net/

hedii commented 7 years ago

@sn4f I just clean-installed the repo on my mac. I followed the instructions on the readme.

I started a new crawl for the url https://katalog-firmy.net/

Everything is ok as shown in the screenshot.

I am a mac user, i don't have access to a windows or linux machine right now. I don't know on which type of OS you are running this repo. Please find the issue on your OS, and if it is not a configuration issue (i really think your issue is a conf issue), please submit a pull request that will not break the crawler on other operating systems.

I am closing the issue for now, because I cannot reproduce any problem on my side.

capture d ecran 2017-11-26 a 13 20 18
sn4f commented 7 years ago

I flush database and start crawler again - now its ok. Thanks for check

sn4f commented 7 years ago

I use mariaDB mby this is a problem? again get this same problem :<

hedii commented 7 years ago

I am also using mariadb

sn4f commented 7 years ago

Hmm... when I use php artisan crawler:crawl command on command line then work.. mby I need to set cron?

hedii commented 7 years ago

No need to setup a cron task. What is your operating system?