sqlite3.OperationalError when using docker for data on HDD

Flufferooo commented 2 years ago

What happened?

Hi, trying to run docker with my data being stored on an HDD. Added that volume specifically to docker under preferences/file sharing. It has writing permissions. But keep getting the same error over and over again. On the bottom it says to report here - can someone help please? thx!

What command did you use?

fmriprep-docker /Volumes/HDD/test  /Volumes/HDD/test/derivatives participant --participant-label 1006 --fs-license-file /Volumes/HDD/license.txt --fs-no-reconall --verbose --output-spaces MNI152NLin2009cAsym:res-2  --stop-on-first-crash --w /Volumes/HDD/temp

What version of fMRIPrep are you running?

21.0.0

How are you running fMRIPrep?

Docker

Is your data BIDS valid?

Yes

Are you reusing any previously computed results?

No

Please copy and paste any relevant log output.

Traceback (most recent call last):
  File "/opt/conda/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
    self.dialect.do_execute(
  File "/opt/conda/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
    cursor.execute(statement, parameters)
sqlite3.OperationalError: attempt to write a readonly database

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/conda/bin/fmriprep", line 8, in <module>
    sys.exit(main())
  File "/opt/conda/lib/python3.8/site-packages/fmriprep/cli/run.py", line 38, in main
    parse_args()
  File "/opt/conda/lib/python3.8/site-packages/fmriprep/cli/parser.py", line 668, in parse_args
    config.from_dict(vars(opts))
  File "/opt/conda/lib/python3.8/site-packages/fmriprep/config.py", line 664, in from_dict
    execution.load(settings)
  File "/opt/conda/lib/python3.8/site-packages/fmriprep/config.py", line 238, in load
    cls.init()
  File "/opt/conda/lib/python3.8/site-packages/fmriprep/config.py", line 481, in init
    cls._layout = BIDSLayout(
  File "/opt/conda/lib/python3.8/site-packages/bids/layout/layout.py", line 140, in __init__
    self.connection_manager = ConnectionManager(
  File "/opt/conda/lib/python3.8/site-packages/bids/layout/db.py", line 45, in __init__
    self.reset_database(init_args, config)
  File "/opt/conda/lib/python3.8/site-packages/bids/layout/db.py", line 104, in reset_database
    Base.metadata.create_all(self.engine)
  File "/opt/conda/lib/python3.8/site-packages/sqlalchemy/sql/schema.py", line 4664, in create_all
    bind._run_visitor(
  File "/opt/conda/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2095, in _run_visitor
    conn._run_visitor(visitorcallable, element, **kwargs)
  File "/opt/conda/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1656, in _run_visitor
    visitorcallable(self.dialect, self, **kwargs).traverse_single(element)
  File "/opt/conda/lib/python3.8/site-packages/sqlalchemy/sql/visitors.py", line 145, in traverse_single
    return meth(obj, **kw)
  File "/opt/conda/lib/python3.8/site-packages/sqlalchemy/sql/ddl.py", line 783, in visit_metadata
    self.traverse_single(
  File "/opt/conda/lib/python3.8/site-packages/sqlalchemy/sql/visitors.py", line 145, in traverse_single
    return meth(obj, **kw)
  File "/opt/conda/lib/python3.8/site-packages/sqlalchemy/sql/ddl.py", line 827, in visit_table
    self.connection.execute(
  File "/opt/conda/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
    return meth(self, multiparams, params)
  File "/opt/conda/lib/python3.8/site-packages/sqlalchemy/sql/ddl.py", line 72, in _execute_on_connection
    return connection._execute_ddl(self, multiparams, params)
  File "/opt/conda/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1068, in _execute_ddl
    ret = self._execute_context(
  File "/opt/conda/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1316, in _execute_context
    self._handle_dbapi_exception(
  File "/opt/conda/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1510, in _handle_dbapi_exception
    util.raise_(
  File "/opt/conda/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
    raise exception
  File "/opt/conda/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
    self.dialect.do_execute(
  File "/opt/conda/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) attempt to write a readonly database
[SQL: 
CREATE TABLE configs (
    name VARCHAR NOT NULL, 
    _default_path_patterns VARCHAR, 
    PRIMARY KEY (name)
)

]
(Background on this error at: http://sqlalche.me/e/13/e3q8)
fMRIPrep: Please report errors to https://github.com/nipreps/fmriprep/issues

Additional information / screenshots

No response

effigies commented 2 years ago

--w /Volumes/HDD/temp

I'm assuming this was actually -w, as --w should raise an error.

I would suggest adding the --shell option to your fmriprep-docker command, which will perform all the usual mounts, but then open a bash prompt instead of running fMRIPrep. From there, you can verify that you can write to the /scratch directory.

Flufferooo commented 2 years ago

Yes, sorry it should be -w. Thank you for your reply. The issue disappeared when changing the HDD's format type from exfat to APFS. Not sure why, but no errors since (without changing the my shellscript).

effigies commented 2 years ago

Likely exfat doesn't support common POSIX operations (my guess is file locking) that the database software expects to be available. Possibly Docker performs some translation that ends up confusing it, so it can't detect that correct operation will not work and use its own fallbacks.

I'm glad to hear that you found a solution. If you're feeling really ambitious, you could try reproducing the issue with a small script and reporting to Docker and/or SQLAlchemy.

nipreps / fmriprep