bench migrate stuck in docker

[saru2020]

Description of the issue

when running docker-compose exec -T backend bench --site <my_site> migrate, it is always stuck and nothing happens

Context information (for bug reports)

NOTE: I did a fresh docker setup with custom images and restored my site

this stuck issue is almost similar to this one where it happened during bench build and the solution there was to bench disable-production and then do the build/migrate and bench setup production <user>, which worked in that case but if we do bench disable-production within docker, this is the behaviour (when I execute these commands from within the container as frappe user: bench disable-production fails with WARN: superuser privileges required for this command

where as sudo bench disable-production fails with OCI runtime exec failed: exec failed: unable to start container process: exec: "sudo": executable file not found in $PATH: unknown

and docker-compose exec -u root -T backend bench disable-production fails with

ERROR: expected str, bytes or os.PathLike object, not NoneType
Traceback (most recent call last):
  File "/usr/local/bin/bench", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File "/usr/local/lib/python3.11/site-packages/bench/cli.py", line 132, in cli
    bench_command()
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/bench/commands/utils.py", line 150, in disable_production
    disable_production(bench_path=".")
  File "/usr/local/lib/python3.11/site-packages/bench/config/production_setup.py", line 96, in disable_production
    supervisor_conf = os.path.join(
                      ^^^^^^^^^^^^^
  File "<frozen posixpath>", line 76, in join
TypeError: expected str, bytes or os.PathLike object, not NoneType

and running bench setup production frappe fails with

Setting Up prerequisites...
$ sudo /usr/local/bin/python -m pip install ansible
ERROR: [Errno 2] No such file or directory: 'sudo'
Traceback (most recent call last):
  File "/usr/local/bin/bench", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File "/usr/local/lib/python3.11/site-packages/bench/cli.py", line 132, in cli
    bench_command()
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/bench/commands/setup.py", line 110, in setup_production
    setup_production(user=user, yes=yes)
  File "/usr/local/lib/python3.11/site-packages/bench/config/production_setup.py", line 37, in setup_production
    setup_production_prerequisites()
  File "/usr/local/lib/python3.11/site-packages/bench/config/production_setup.py", line 26, in setup_production_prerequisites
    exec_cmd(f"sudo {sys.executable} -m pip install ansible")
  File "/usr/local/lib/python3.11/site-packages/bench/utils/__init__.py", line 174, in exec_cmd
    return_code = subprocess.call(spl_cmd, cwd=cwd, universal_newlines=True, env=env)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/subprocess.py", line 389, in call
    with Popen(*popenargs, **kwargs) as p:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/subprocess.py", line 1026, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/local/lib/python3.11/subprocess.py", line 1950, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'sudo'

how do I disable production mode on the frappe app running inside a docker container within EC2 instance?

Steps to reproduce the issue

1. deploy custom app with frappe_docker
2. restore site with the backup files
3. try to run bench migrate

Observed result

the migration is stuck at Migrating <my_site>

Expected result

site migration should work

---

I feel like I've been going in circles trying to fix this issue for a long time now, any help would really appreciated

Edit: I have referred the FAQ page and know that we cannot use any build commands, but I can't get rid of the bench migrate issue, I'm stuck there and don't know what else to do except going back to VM but that's not a good choice for I have to write automate deployment and manage it manually, not a good place to be in and above are my attempts to just try & find & see if I could get rid of that issue.

[saru2020]

@revant can you help here?

[revant]

I can't really help. All my setup are migrating properly.

I can just suggest:

• don't use any build commands in production container
• don't use root user in production, sudo is not added for a reason.
• read faq
• switch to VM instead of containers, ssh and sudo everything! You've total control over VM.

[saru2020]

@revant thank you for your inputs, I hope your contributions get well rewarded. I'm going back to the hectic VM route for now.

[revant]

https://discuss.frappe.io/t/migrating-site-through-helm-issue/137131/2