Unify Popen calls

[bernt-matthias]

Discussion started in https://github.com/galaxyproject/galaxy/pull/9246.

There are many

p = subprocess.Popen(...)
stdout, stderr = p.communicate(...)

calls which should be unified with an already existing function from https://github.com/galaxyproject/galaxy/blob/9e76dcd9d381545ef627b63ec406dccf34bc9ae9/lib/galaxy/tool_util/deps/commands.py#L97

_wait needs to be unicodified since otherwise bytes are returned in py3 instead of str.

[bernt-matthias]

• [ ] close_fds
  • lib/galaxy/containers/init.py
  • lib/galaxy/visualization/plugins/interactive_environments.py
  • lib/galaxy/tools/expressions/evaluation.py
  • lib/galaxy/tool_shed/galaxy_install/tool_dependencies/recipe/install_environment.py + bufsize + cwd
• [x] use stdin
  • lib/galaxy/datatypes/converters/bgzip.py: https://github.com/galaxyproject/galaxy/issues/9672
• [ ] use returned process
  • lib/galaxy/datatypes/dataproviders/external.py
  • lib/galaxy/util/lazy_process.py
  • lib/galaxy/jobs/runners/util/sudo.py
  • lib/galaxy_test/driver/driver_util.py + preexec_fn + cwd
• [ ] use env -> maybe shell_process
  • lib/galaxy/tool_util/deps/brew_exts.py
  • lib/galaxy/tool_util/deps/commands.py + shell
  • lib/galaxy/tool_util/deps/resolvers/modules.py
  • lib/galaxy/tool_util/deps/resolvers/lmod.py
  • lib/galaxy/util/pastescript/serve.py
  • lib/galaxy/jobs/runners/init.py + shell clse_fds, preexec_fn
  • lib/galaxy/jobs/runners/condor.py + -"-
  • lib/galaxy/jobs/runners/local.py + -"-
  • lib/galaxy/jobs/runners/util/cli/shell/local.py + -"-
• [ ] use shell=True
  • lib/galaxy/tool_util/deps/container_classes.py
  • lib/galaxy/tool_util/verify/test_data.py (because it uses strings and command chaining with &&)
  • lib/galaxy/job_metrics/collectl/cli.py
• [ ] read ony begin of stdout
  • lib/galaxy/tool_shed/galaxy_install/migrate/check.py
  • lib/galaxy/jobs/transfer_manager.py