Error in python rsfb/benchmark.py evaluate experiments/bsbm/config_small.yaml

[momo54]

Error during:

• python rsfb/benchmark.py evaluate experiments/bsbm/config_small.yaml

Everything run fine for while, but error appears after 67 jobs :

Building DAG of jobs... Using shell: /usr/bin/bash Provided cores: 1 (use --cores to define parallelism) Rules claiming more threads will be scaled down. Select jobs to execute... python rsfb/engines/fedx.py transform-provenance experiments/bsbm/benchmark/evaluation/fedx/q05/instance_0/batch_0/attempt_0/source_selection.txt experiments/bsbm/benchmark/evaluation/fedx/q05/instance_0/batch_0/attempt_0/provenance.csv experiments/bsbm/benchmark/generation/q05/instance_0/prefix_cache.json Traceback (most recent call last): File "rsfb/engines/fedx.py", line 244, in cli() File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1130, in call return self.main(args, kwargs) File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1055, in main rv = self.invoke(ctx) File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1657, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1404, in invoke return ctx.invoke(self.callback, ctx.params) File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 760, in invoke return __callback(args, *kwargs) File "rsfb/engines/fedx.py", line 172, in transform_provenance in_df = pd.read_csv(infile) File "/usr/local/lib/python3.8/dist-packages/pandas/util/_decorators.py", line 311, in wrapper return func(args, kwargs) File "/usr/local/lib/python3.8/dist-packages/pandas/io/parsers/readers.py", line 678, in read_csv return _read(filepath_or_buffer, kwds) File "/usr/local/lib/python3.8/dist-packages/pandas/io/parsers/readers.py", line 575, in _read parser = TextFileReader(filepath_or_buffer, kwds) File "/usr/local/lib/python3.8/dist-packages/pandas/io/parsers/readers.py", line 932, in init self._engine = self._make_engine(f, self.engine) File "/usr/local/lib/python3.8/dist-packages/pandas/io/parsers/readers.py", line 1234, in _make_engine return mapping[engine](f, self.options) File "/usr/local/lib/python3.8/dist-packages/pandas/io/parsers/c_parser_wrapper.py", line 75, in init self._reader = parsers.TextReader(src, kwds) File "pandas/_libs/parsers.pyx", line 551, in pandas._libs.parsers.TextReader.cinit pandas.errors.EmptyDataError: No columns to parse from file [Sun May 14 09:58:56 2023] Error in rule transform_provenance: jobid: 0 output: experiments/bsbm/benchmark/evaluation/fedx/q05/instance_0/batch_0/attempt_0/provenance.csv

RuleException: CalledProcessError in line 146 of /FedShop/experiments/bsbm/evaluate.smk: Command 'set -euo pipefail; python rsfb/engines/fedx.py transform-provenance experiments/bsbm/benchmark/evaluation/fedx/q05/instance_0/batch_0/attempt_0/source_selection.txt experiments/bsbm/benchmark/evaluation/fedx/q05/instance_0/batch_0/attempt_0/provenance.csv experiments/bsbm/benchmark/generation/q05/instance_0/prefix_cache.json' returned non-zero exit status 1. File "/FedShop/experiments/bsbm/evaluate.smk", line 146, in __rule_transform_provenance File "/usr/lib/python3.8/concurrent/futures/thread.py", line 57, in run Shutting down, this might take some time. Exiting because a job execution failed. Look above for error message Shutting down, this might take some time. Exiting because a job execution failed. Look above for error message Complete log: .snakemake/log/2023-05-14T095854.930701.snakemake.log

[Yotlan]

This error is due to an empty file. To fix it, at the beginning of transform_provenance method, we need to add the following code:

if os.stat(infile).st_size == 0:
        Path(outfile).touch()
        return

[mhoangvslev]

I would rather find out the reason why it produced no results...