search

galaxyproject / galaxy

Data intensive science for everyone.
https://galaxyproject.org
Other
1.38k stars 998 forks source link

Trying to log bad output containg NUL characters fails #8432

Open pvanheus opened 5 years ago

pvanheus commented 5 years ago

Background: feeding gzipped FASTQ files to shovill triggers an error due to a weakness in the wrapper, as described here: https://github.com/galaxyproject/tools-iuc/issues/1804

This results in an error when Galaxy tries to process the output of this failed job, as seen on Galaxy 19.05:

galaxy.jobs.runners ERROR 2019-08-08 15:16:26,500 [p:27106,w:0,m:1] [SlurmRunner.work_thread-1] (10290/14387) Job wrapper finish method failed
Traceback (most recent call last):
  File "/projects/galaxy/main_galaxy1/galaxy/lib/galaxy/jobs/runners/__init__.py", line 497, in _finish_or_resubmit_job
    self.sa_session.flush()
  File "/projects/galaxy/main_galaxy1/galaxy/.venv/local/lib/python2.7/site-packages/sqlalchemy/orm/scoping.py", line 162, in do
    return getattr(self.registry(), name)(*args, **kwargs)
  File "/projects/galaxy/main_galaxy1/galaxy/.venv/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 2446, in flush
    self._flush(objects)
  File "/projects/galaxy/main_galaxy1/galaxy/.venv/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 2584, in _flush
    transaction.rollback(_capture_exception=True)
  File "/projects/galaxy/main_galaxy1/galaxy/.venv/local/lib/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 67, in __exit__
    compat.reraise(exc_type, exc_value, exc_tb)
  File "/projects/galaxy/main_galaxy1/galaxy/.venv/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 2544, in _flush
    flush_context.execute()
  File "/projects/galaxy/main_galaxy1/galaxy/.venv/local/lib/python2.7/site-packages/sqlalchemy/orm/unitofwork.py", line 416, in execute
    rec.execute(self)
  File "/projects/galaxy/main_galaxy1/galaxy/.venv/local/lib/python2.7/site-packages/sqlalchemy/orm/unitofwork.py", line 583, in execute
    uow,
  File "/projects/galaxy/main_galaxy1/galaxy/.venv/local/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 236, in save_obj
    update,
  File "/projects/galaxy/main_galaxy1/galaxy/.venv/local/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 976, in _emit_update_statements
    statement, multiparams
  File "/projects/galaxy/main_galaxy1/galaxy/.venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 980, in execute
    return meth(self, multiparams, params)
  File "/projects/galaxy/main_galaxy1/galaxy/.venv/local/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 273, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/projects/galaxy/main_galaxy1/galaxy/.venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1099, in _execute_clauseelement
    distilled_params,
  File "/projects/galaxy/main_galaxy1/galaxy/.venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1240, in _execute_context
    e, statement, parameters, cursor, context
  File "/projects/galaxy/main_galaxy1/galaxy/.venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1460, in _handle_dbapi_exception
    util.reraise(*exc_info)
  File "/projects/galaxy/main_galaxy1/galaxy/.venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1236, in _execute_context
    cursor, statement, parameters, context
  File "/projects/galaxy/main_galaxy1/galaxy/.venv/local/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 536, in do_execute
    cursor.execute(statement, parameters)
ValueError: A string literal cannot contain NUL (0x00) characters.

While this is a challenging situation for Galaxy, it is means the user (and Galaxy admin) is left with a difficult situation to diagnose. The job output should ideally be turned into a valid string - or treated as bytes.

nsoranzo commented 5 years ago

@pvanheus Which version of Galaxy is that? I think this should have been fixed by https://github.com/galaxyproject/galaxy/pull/7889 (in dev) / https://github.com/galaxyproject/galaxy/pull/8156 (19.05).