_prep_cache issue

[buus2]

For

neptune send --environment pytorch-0.2.0-gpu-py3 --worker gcp-gpu-medium -- dry_eval --problem whales

I obtained

114.465695 | Traceback (most recent call last):
-- | --
114.466053 | File "/usr/local/lib/python3.6/dist-packages/deepsense/neptune/job_wrapper.py", line 138, in <module>
114.466311 | execute()
114.466593 | File "/usr/local/lib/python3.6/dist-packages/deepsense/neptune/job_wrapper.py", line 134, in execute
114.466841 | execfile(job_filepath, job_globals)
114.467081 | File "/usr/local/lib/python3.6/dist-packages/past/builtins/misc.py", line 82, in execfile
114.467322 | exec_(code, myglobals, mylocals)
114.467577 | File "main.py", line 72, in <module>
114.467857 | action()
114.468074 | File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 722, in __call__
114.468385 | return self.main(*args, **kwargs)
114.468668 | File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 697, in main
114.46896 | rv = self.invoke(ctx)
114.46924 | File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 1066, in invoke
114.469555 | return _process_result(sub_ctx.command.invoke(sub_ctx))
114.469857 | File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 895, in invoke
114.470132 | return ctx.invoke(self.callback, **ctx.params)
114.470374 | File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 535, in invoke
114.47058 | return callback(*args, **kwargs)
114.470817 | File "main.py", line 28, in dry_eval
114.471031 | dry_run(problem, dev_mode, cloud_mode, train_mode=False)
114.471244 | File "main.py", line 40, in dry_run
114.471457 | pm.dry_run(sub_problem, train_mode, dev_mode, cloud_mode)
114.47169 | File "/neptune/minerva/whales/problem_manager.py", line 26, in dry_run
114.471911 | handle_empty_solution_dir(train_mode, config, pipeline)
114.472125 | File "/neptune/minerva/utils.py", line 54, in handle_empty_solution_dir
114.472339 | transformers_in_pipeline = set(pipeline(config).all_steps.keys())
114.472554 | File "/neptune/minerva/whales/pipelines.py", line 44, in alignment_pipeline
114.47277 | cache_dirpath=config['global']['cache_dirpath'])
114.472991 | File "/neptune/minerva/backend/base.py", line 183, in __init__
114.473226 | super().__init__(*args, **kwargs)
114.473485 | File "/neptune/minerva/backend/base.py", line 24, in __init__
114.473748 | self._prep_cache(cache_dirpath, save_outputs)
114.474027 | File "/neptune/minerva/backend/base.py", line 33, in _prep_cache
114.474336 | os.makedirs(os.path.join(cache_dirpath, dirname), exist_ok=True)
114.474574 | File "/usr/lib/python3.6/os.py", line 220, in makedirs
114.47482 | mkdir(name, mode)
114.475053 | OSError: [Errno 30] Read-only file system: '/public/minerva/resources/whales/solution/alignment/outputs'

I chose:

data_dir: /public/whales
solution_dir: /public/minerva/resources/whales/solution

[buus2]

Just want to add that fashion mnist with solution_dir: /public/minerva/resources/fashion_mnist/solution works fine.

[jakubczakon]

You need to specify neptune send ... cloud_mode flag in your command. However this issue pointed to a different problem that is fixed now. Thanks! Waiting for PR to merge.

[kamil-kaczmarek]

@jakubczakon @buus2 merged in #56