Error: Mining phase failure

Hello everyone, I'm having problems running the mining phase. I'm using a computer with 60Gb of RAM and 16 CPU cores. When running the mining phase I get the error below.

Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/raphael_assis4347/raphael/cc_net/cc_net/__main__.py", line 18, in <module>
    main()
  File "/home/raphael_assis4347/raphael/cc_net/cc_net/__main__.py", line 14, in main
    func_argparse.parse_and_call(cc_net.mine.get_main_parser())
  File "/home/raphael_assis4347/.local/lib/python3.8/site-packages/func_argparse/__init__.py", line 72, in parse_and_call
    return command(**parsed_args)
  File "/home/raphael_assis4347/raphael/cc_net/cc_net/mine.py", line 631, in main
    all_files = mine(conf)
  File "/home/raphael_assis4347/raphael/cc_net/cc_net/mine.py", line 341, in mine
    ex(_mine_shard, repeat(conf), hashes_files, *_transpose(missing_outputs))
  File "/home/raphael_assis4347/raphael/cc_net/cc_net/execution.py", line 200, in custom_map_array
    raise Exception(message)
Exception: 9 / 9 jobs failed while running _mine_shard
Will run cc_net.mine.main with the following config: Config(config_name='base', dump='2019-09', output_dir=PosixPath('data'), mined_dir='mined_data', execution='auto', num_shards=9, num_segments_per_shard=750, metadata=None, min_len=300, hash_in_mem=9, lang_whitelist=['pt'], lang_blacklist=[], lang_threshold=0.5, keep_bucket=['head', 'middle'], lm_dir=PosixPath('data/lm_sp'), cutoff=PosixPath('/home/raphael_assis4347/raphael/cc_net/cc_net/data/cutoff.csv'), lm_languages=['pt'], mine_num_processes=9, target_size='4G', cleanup_after_regroup=True, task_parallelism=-1, pipeline=['dedup', 'lid', 'keep_lang', 'sp', 'lm', 'pp_bucket', 'drop', 'split_by_lang'], experiments=[], cache_dir=PosixPath('data/wet_cache'))
Submitting 9 jobs for _mine_shard, with task_parallelism=16
Waiting on 9 running jobs. Job ids: 69378,69397,69400,69419...
Failed job 69516 (1 / 9): Job 69516 (task: 0) with path /home/raphael_assis4347/raphael/cc_net/data/logs/69516_0_result.pkl
has not produced any output (state: FINISHED)
Error stream produced:
----------------------------------------
2022-09-14 16:43 INFO 69535:cc_net.jsonql - preparing [<cc_net.dedup.DuplicatesRemover object at 0x7fd7313c8d00>, Classifier(bin/lid.bin), <cc_net.jsonql.where object at 0x7fd7313c8df0>, <cc_net.perplexity.MultiSentencePiece object at 0x7fd7313c8d30>, <cc_net.perplexity.DocLM object at 0x7fd7313c8e50>, <cc_net.perplexity.PerplexityBucket object at 0x7fd7313c8d60>, <cc_net.perplexity.DropKeys object at 0x7fd7313c8fa0>]

Waiting on 8 running jobs. Job ids: 69378,69397,69400,69419...
Failed job 69400 (2 / 9): Job 69400 (task: 0) with path /home/raphael_assis4347/raphael/cc_net/data/logs/69400_0_result.pkl
has not produced any output (state: FINISHED)
Error stream produced:
----------------------------------------
2022-09-14 16:43 INFO 69418:cc_net.jsonql - preparing [<cc_net.dedup.DuplicatesRemover object at 0x7fcee923a970>, Classifier(bin/lid.bin), <cc_net.jsonql.where object at 0x7fcee923aa60>, <cc_net.perplexity.MultiSentencePiece object at 0x7fcee923a9a0>, <cc_net.perplexity.DocLM object at 0x7fcee923aac0>, <cc_net.perplexity.PerplexityBucket object at 0x7fcee923a9d0>, <cc_net.perplexity.DropKeys object at 0x7fcee923ac10>]

I couldn't identify the problem by looking at the logs. In the process .log.err file it only contains the pipeline objects vector. Does anyone have any idea what it could be?

This is my configuration file:

{
    "dump": "2019-09",
    "hash_in_mem": 9,
    "num_shards": 9,
    "mine_num_processes": 9,
    "num_segments_per_shard": 750,
    "lang_whitelist": ["pt"],
    "lm_languages": ["pt"],
    "pipeline": [
        "dedup",
        "lid",
        "keep_lang",
        "sp",
        "lm",
        "pp_bucket",
        "drop",
        "split_by_lang"
    ],
    "execution": "auto",
    "output_dir": "data",
    "mined_dir": "mined_data",
    "target_size": "4G",
    "keep_bucket": ["head", "middle"],
    "cache_dir": "data/wet_cache"
}

facebookresearch / cc_net

Error: Mining phase failure #36