zerospeech / benchmarks

A command line tool that helps use the "Zero Ressource Challenge" benchmarks
https://zerospeech.com/toolbox/
GNU General Public License v3.0
8 stars 2 forks source link

tde17 benchmark is killed on the Grouping computation #29

Closed nhamilakis closed 11 months ago

nhamilakis commented 11 months ago

When calculating Grouping in the tde17 benchmark we use a SIGALRM to put a timeout, if the computation takes more than grouping_max_time to compute we skip it.

This was inherited from the previous benchmark implementation, we should check if the reasoning behind skiping that computation is based on some reasoning

The SIGALRM sometimes causes the process to be killed (maybe if the computation ends too quickly ?).

nhamilakis commented 11 months ago

An interesting alternative is to use a timer from threading, as shown on this stack overflow thread.

nhamilakis commented 11 months ago

Implementing 6f1223c showed that the process kill does not come from the SIGALRM, but from the Grouping computing itself.

HINT: the exit code 137 indicated the process received a SIGKILL, this could be from the OS as the computing used too many resources (see attached screenshot) ?

HINT2: should check on a correct way to debug the issue (obtain a proper exit message)

zrc-error

nhamilakis commented 11 months ago

The cause of the kill was the OOM-Killer that runs on Linux and kills any process that uses too much memory, the solution was to use a stronger PC (16GB of RAM seems to be enough).

Though observation, the most used was 10 GB of RAM.