search

keeleinstituut / tv-tolkevarav

Tõlkevärav (Translation Hub)
1 stars 0 forks source link

Splitting the project gives error (BE) #784

Open MariusJulius opened 3 weeks ago

MariusJulius commented 3 weeks ago

After project (xliff) is created (can be openened in matecat) then it is not possible to split as analyses is still in progress. It seems that since analyses takes too long time, it is also not possible to split.

Examples LOL-2024-06-T-487 LOL-2024-06-TT-486 LOL-2024-06-T-485

kadmit commented 3 weeks ago

@MariusJulius

I did investigations and found that our test server has not enough RAM to perform TM analysis. Currently, we allocate 1GB of RAM for the container.

I checked the pod state during analyzing of a file and found that it exited with error code 137, which indicates that the container was terminated due to an out-of-memory (OOM) condition. Based on my understanding it happening because we have 15 workers that work in parallel and perform volume analysis and 1GB is not sufficient for them.

I see two possible ways of how we can handle the issue:

  1. Decrease the number of workers (at least for the test env) it will end up with a slower processing speed but will fit into memory restriction.
  2. Increase the memory limit for the CAT container.

@thenouan please advise what would be a better solution for it.

MariusJulius commented 3 weeks ago

okay, so probably not happening in production. Tagging @plakitkelly as well to be informed of the situation

kadmit commented 1 week ago

Blocked by: https://github.com/keeleinstituut/tv-tolkevarav/issues/793