MySQL servers timing out before script completes

[GoogleCodeExporter]

In order to alleviate the issue I presented in Issue 29 whereby php scripts 
were timing out on commercial hosts, I set up my own web server.  This 
presented and entirely new problem in that remotely hosted mySQL databases also 
have a timeout on their connections.

After 10 minutes of running listupdater.php, it presented me with a few 
thousand of these:

COULD NOT SAVE goog-malware-shavar SUB INDEXS MySQL server has gone away...

Original issue reported on code.google.com by uprom...@gmail.com on 2 Mar 2012 at 5:16

[GoogleCodeExporter]

Hmm, I'll run some local benchmarks on average runtime. The script itself 
shouldn't run for that long (perhaps 30 seconds). In any case the fact that it 
is on a minute cron doesn't mean that it actually runs every minute, Google 
gives a timeout of around 15 minutes between updates. It would be easy to fix 
the mySQL server gone away issue using the mysql_ping function but I'd like to 
investigate the root cause first.

Original comment by cleaver....@gmail.com on 2 Mar 2012 at 11:07

• Changed state: Investigating

[GoogleCodeExporter]

Issue 29 has been merged into this issue.

Original comment by cleaver....@gmail.com on 2 Mar 2012 at 11:07

[GoogleCodeExporter]

Part of the problem might be that I'm only running the listupdater every day or 
two.  

Original comment by uprom...@gmail.com on 3 Mar 2012 at 5:48

[GoogleCodeExporter]

Added the listupdater to a crontab on my Linux box and it seems to be working 
just fine now. Guess it helps if you read the instructions.  This however makes 
it almost necessary to have your own server running. I know many hosts don't 
allow crontab usage. I know GoDaddy has that feature on it's Linux hosts and I 
assume something similar on the Windows hosts but no idea if it behaves like 
every other GoDaddy feature and eventually shuts down.

Otherwise, GREAT STUFF! Works as advertised. Building a Shareware listing 
website and use it to check url's before displaying a listing to a user. You 
just saved me a lot of nasty red Google pages.

Original comment by uprom...@gmail.com on 3 Mar 2012 at 8:16

[GoogleCodeExporter]

Ahh yea it does need to be run every minute, 99% of web hosts do allow for 
crons (even Godaddy!), some call it scheduled tasks but its all the same thing. 
When Google designed it I think they had scripts running as daemons in mind; 
obviously PHP doesn't run like that so we just have to trigger it every minute 
to get the same effect.

I'll keep this ticket open as I do want to investigate performance to ensure 
we're not hanging unneccesarily at any points. Glad you've found the script 
useful and just create a new ticket if you have any feature requests/bug 
reports etc.

Original comment by cleaver....@gmail.com on 5 Mar 2012 at 12:43