search

coriane / jwpl

Automatically exported from code.google.com/p/jwpl
0 stars 0 forks source link

[RevisionMachine] Add feature to DiffTool-ConfigGUI to allow creation of multiple config files for distributed processing #24

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago 
When working with multiple source archives, the DiffTools normally processes 
them all sequentially and builds a single output files.

In order to speed up processing, multiple configuration files can be written to 
allow for calculating revisions using several instances of the DiffTool. e.g. 
if we have 15 source archives, we might create 5 configuration files - each 
definining 3 archives. We can then process the data with 5 instances of the 
tool in parallel.

Output folders have to be adapted in case of multiple instances.

This is, of course, no replacement for REAL distributed processing. The 
limiting factor here probably is the RAM.

Original issue reported on code.google.com by oliver.ferschke on 6 Jun 2011 at 4:24