search

ilyasozkurt / mobilephone-brands-and-models

A database includes mobilephone manufacturers and their models.
GNU General Public License v3.0
78 stars 41 forks source link

problem #11

Closed Aikanairo closed 1 year ago

Aikanairo commented 1 year ago

the procedure is blocked at 4800 records, how can I read the errors? or the reason for the block? can i restart it from the breakpoint?

Aikanairo commented 1 year ago

i think it was the gsmarena server that was blocking the requests. imports stopped at 4800 items. is there a solution? I read on the internet that there are services like scraperapi.com that bypass the limitations.

forgive my ignorance, I approached scraping out of pure curiosity, I ran your script and was happy to see it working, too bad it crashed

ilyasozkurt commented 1 year ago

It's a protection that they're running. You can continue from the point that you are stuck. You just need to restart from another day, or you can slow down the requests with sleep(1); etc.

Aikanairo commented 1 year ago

To restart from the point of interruption, just send the command? The settings like sleep etc where can I change them Thanks again for your patience

ilyasozkurt commented 1 year ago

It's enough to run the same command from the point you stay. You're welcome