gaenserich
commented
8 years ago This isn't such a big deal for me anymore. The jump to the 1.0 branch greatly reduced the impact of serialization through the extensive use of grep (which is truly amazing performance-wise, especially compared to sed).
Likewise, I'm leery of introducing parallel as a dependency due to its prior, gigantic dependency on perl. If we were to introduce similar functionality, I would prefer to have it implimented via bash/sh.
That said, there are just a few areas that would benefit a little bit from asynchronous execution, namely the entries in the following list with an asterisk:
All other processes do need to executed serially, at least as our existing paradigm exits. Right now, hostsblock follows this order: -Check for available utilities, source config files for parameters. -Create temporary work directories -_Check and, if changed, download blocklists (one after the other serially) (could be done in parallel, but since our use of curl only downloads files as needed, the benefit would only be marginal) -If no changes have been detected in the blocklists, stop, otherwise copy/extract all blocklists to the work directory (done in parallel) -_backup old blocklist (this could actually be trivially changed to happen in parallel with previous step) -copy over hosts.head to target hosts.block file, if so optioned. -extract, sort, annotate, and compile block entries from blockfiles into hosts.block target file (done through pipes) -extract, sort, annotate, and compile redirection entries from blockfiles into hosts.block target file, if so optioned (also via pipes) -*sort and, if asked for, compress the annotation file. (could be done in parallel with next step) -if optioned, count the number of blocked and redirected entries -execute post-processing options (e.g. for restarting a dnscacher) -clean up
pickfire
I am now researching for
xargsandparallel.