search

internetarchive / brozzler

brozzler - distributed browser-based web crawler
Apache License 2.0
669 stars 97 forks source link

Skip loading behavior when behavior_timeout=0 #205

Closed vbanos closed 4 years ago

vbanos commented 4 years ago

The user may set behavior_timeout=0. This means that they don't want to run the behavior. As it is now, Brozzler will invoke brozzler.behavior_script to load the script and self.run_behavior to execute it. We will run the behavior using Runtime.evaluate but then it will be terminated immediately because of timeout=0.

It is better to skip behavior loading and running when behavior_timeout=0.

galgeek commented 4 years ago

behavior_timeout is an existing parameter for Browser.browse_page, which is used directly by other code.