Add exceptions scraperwiki.CPUTimeExceededError and scraperwiki.Error.

[petterreinholdtsen]

This bring the library closer to the classic API, which provided both these exceptions.

[drj11]

thanks!

[pwaller]

When are these exceptions ever generated @drj11?

[petterreinholdtsen]

[Peter Waller]

When are these exceptions ever generated @drj11?

It was done in the old execute code, see for example <URL: https://github.com/rossjones/ScraperWikiX/blob/master/services/scriptmgr/scripts/exec.py >.

At the moment it is not done by the library, but I have added this code in my init.py to do so:

CPU limits.

import resource resource.setrlimit(resource.RLIMITCPU, (300, 302,)) def sigXCPU(signum, frame_): raise CPUTimeExceededError("ScraperWiki CPU time exceeded") import signal signal.signal(signal.SIGXCPU, sigXCPU)

I'm not quite sure if that is a good place to put it, and have not yet submitted a patch for this part.

The reason I submitted the patch is that I had scrapers trying to catch these exceptions, and the library lacked the classes/exceptions, causing the code to fail when other exceptions came in.

Happy hacking Petter Reinholdtsen

[pwaller]

Ah. Sadly, this isn't safe:

http://741mhz.com/signal-handler/

You can't really do much in a signal handler. Let alone run python code :(