tasks must not change working directory to avoid "No such file or directory: '.doit.db.dat'"

[goretkin]

Running the dodo.py below:

import os

def task_hello():
    """hello"""

    def python_hello(targets):
        with open(targets[0], "a") as output:
            output.write("Python says Hello World!!!\n")
            os.chdir("..")

    return {
        'actions': [python_hello],
        'targets': ["hello.txt"],
        }

def task_goodbye():
    """goodbye"""

    def python_goodbye(targets):
        with open(targets[0], "a") as output:
            output.write("Python says Goodbye World!!!\n")

    return {
        'actions': [python_goodbye],
        'file_dep': ["hello.txt"],
        'targets': ["goodbye.txt"],
        }

if __name__ == '__main__':
    import doit
    doit.run(globals())

Gives the following output:

changing directory causes this issue:
.  hello
.  goodbye
DependencyError - taskid:goodbye
ERROR: Task 'goodbye' saving success: Dependent file 'hello.txt' does not exist

Traceback (most recent call last):
  File "[...]\python_environment\lib\site-packages\doit\doit_cmd.py", line 295, in run
    return command.parse_execute(args)
  File "[...]\python_environment\lib\site-packages\doit\cmd_base.py", line 151, in parse_execute
    return self.execute(params, args)
  File "[...]\python_environment\lib\site-packages\doit\cmd_base.py", line 605, in execute
    return self._execute(**exec_params)
  File "[...]\python_environment\lib\site-packages\doit\cmd_run.py", line 265, in _execute
    return runner.run_all(self.control.task_dispatcher())
  File "[...]\python_environment\lib\site-packages\doit\runner.py", line 260, in run_all
    self.finish()
  File "[...]\python_environment\lib\site-packages\doit\runner.py", line 239, in finish
    self.dep_manager.close()
  File "[...]\python_environment\lib\site-packages\doit\dependency.py", line 522, in close
    self.backend.dump()
  File "[...]\python_environment\lib\site-packages\doit\dependency.py", line 179, in dump
    self._dbm[task_id] = self.codec.encode(self._db[task_id])
  File "[...]\contrib\Python37\lib\dbm\dumb.py", line 208, in __setitem__
    self._addkey(key, self._addval(val))
  File "[...]\contrib\Python37\lib\dbm\dumb.py", line 164, in _addval
    with _io.open(self._datfile, 'rb+') as f:
FileNotFoundError: [Errno 2] No such file or directory: '.doit.db.dat'

What if the tasks themselves took care of restoring the working directory, since doit relies on it?

Upvote & Fund

• We're using Polar.sh so you can upvote and help fund this issue.
• We receive the funding once the issue is completed & confirmed by you.
• Thank you in advance for helping prioritize & fund our backlog.

[schettino72]

what if you have 2 different tasks running in different threads?

[goretkin]

@schettino72 , I am not sure I understand. Is that an argument against doing any process-wide state changes, like changing the current directory, within a task?

[schettino72]

Is that an argument against doing any process-wide state changes, like changing the current directory, within a task?

yes. of course you could do this on your own code on your own risk. but doit will not support that.

[rhubarb]

so what then, is the standard way of operating in a subdirectory of the root (which has dodo.py). E.g. I want to do a docker compose build in my deployment subdirectory. Manually, from the root, this is:

cd ./deployment
docker compose build

I'm sure I could do it without the cd using docker command-line args, but many similar tasks are neater when working from a specific cwd. What's the doit way of operating in a subdir? (I've scoured the docs for cd, directory, chdir, and can't find much here)

[schettino72]

What's the doit way of operating in a subdir?

None. You should use python's standard way of manipulating the current working directory. Just note that it is not thread safe, so you might have problems if executing tasks in parallel.

If you explicitly use CmdAction (not just pass a string) you could use the subprocess.Popen 's cwd parameter.