phnmnl / container-galaxy-k8s-runtime

PhenoMeNal runtime for Galaxy running inside a container orchestrator
Apache License 2.0
5 stars 18 forks source link

galaxy.jobs.runners: Failed to write job script, could not verify job script integrity. #204

Open sneumann opened 6 years ago

sneumann commented 6 years ago

Hi, I have seen cases where jobs (here: upload from FTP) don't seem to run, and the history item remains grey. I now found a corresponding log entry, see below. It seems to be an issue with the filesystem setup. Here, it was in Giessen with a single gluster node, and several gluster performance settings disabled.

galaxy.jobs.runners ERROR 2018-06-19 08:09:46,945 (19) Unhandled exception calling queue_job
Traceback (most recent call last):
  File "lib/galaxy/jobs/runners/__init__.py", line 104, in run_next
    method(arg)
  File "lib/galaxy/jobs/runners/local.py", line 87, in queue_job
    command_line, exit_code_path = self.__command_line(job_wrapper)
  File "lib/galaxy/jobs/runners/local.py", line 76, in __command_line
    self.write_executable_script(job_file, job_file_contents)
  File "lib/galaxy/jobs/runners/__init__.py", line 327, in write_executable_script
    write_script(path, contents, self.app.config, mode=mode)
  File "lib/galaxy/jobs/runners/util/job_script/__init__.py", line 112, in write_script
    _handle_script_integrity(path, config)
  File "lib/galaxy/jobs/runners/util/job_script/__init__.py", line 147, in _handle_script_integrity
    raise Exception("Failed to write job script, could not verify job script integrity.")
Exception: Failed to write job script, could not verify job script integrity.

Gluster performance disabled:

performance.open-behind off
performance.write-behind off
read-ahead off

I have seen such grey jobs in Bielefeld as well, but didn't catch the logs.

ilveroluca commented 6 years ago

any news with this issue?