search

ufal / treex

Treex NLP framework
33 stars 6 forks source link

A tiny fix to allow local parallel execution #31

Closed tuetschek closed 8 years ago

tuetschek commented 8 years ago

Parallel execution on a single local machine has been here all along, since the time Martin Majlis implemented it (the parameter --local is even documented in Treex::Core::Run)... I only never knew.

The only problem was that killing the Head process did not kill all the processing Nodes. This tiny change fixes the problem, so we may now run Treex in parallel on a single machine (using e.g. treex -p -j 4 --local).

Note that no memory checking is done, so the user must make sure that the whole thing will fit into memory (e.g. parsers/translation models need to load n times).

martinpopel commented 8 years ago

Thank you. (BTW: --local was implemented by Zdeněk Žabokrtský already before Martin Majliš came to UFAL.)