oamg / leapp-repository

Leapp repositories containing actors for the Leapp framework (https://github.com/oamg/leapp). Currently provides leapp repositories for in-place upgrades of RHEL systems.
Apache License 2.0
48 stars 144 forks source link

"Leapp command Error has occurred" instead of proper upgrade inhibition in case not enough space available at / #332

Open fernflower opened 5 years ago

fernflower commented 5 years ago

To reproduce on an openstack vm: fallocate -l $(($(df --output=avail /boot | tail -1) / 1025))M /boot/dump

and run leapp preupgrade

Instead of upgrade inhibition with human-readable report and console logging you'll get

rocess Process-172:
Traceback (most recent call last):
  File "/usr/lib64/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/lib64/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python2.7/site-packages/leapp/repository/actor_definition.py", line 63, in _do_run
    target_actor(logger=logger, messaging=messaging, config_model=config_model).run(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/leapp/actors/__init__.py", line 320, in run
    self.process(*args)
  File "/usr/share/leapp-repository/repositories/system_upgrade/el7toel8/actors/targetuserspacecreator/actor.py", line 25, in process
    userspacegen.perform()
  File "/usr/share/leapp-repository/repositories/system_upgrade/el7toel8/actors/targetuserspacecreator/libraries/userspacegen.py", line 182, in perform
    prepare_target_userspace(context, constants.TARGET_USERSPACE, target_repoids, list(packages))
  File "/usr/share/leapp-repository/repositories/system_upgrade/el7toel8/actors/targetuserspacecreator/libraries/userspacegen.py", line 36, in prepare_target_userspace
    callback_raw=utils.logging_handler
  File "/usr/share/leapp-repository/repositories/system_upgrade/el7toel8/libraries/mounting.py", line 153, in call
    return run(self.type.make_command(cmd), *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/leapp/libraries/stdlib/__init__.py", line 178, in run
    result=result
CalledProcessError: A Leapp Command Error occurred. 

While preupgrade.log will have

2019-09-09 11:08:08.338 DEBUG    PID: 10967 leapp.workflow.FactsCollection.target_userspace_creator: Error: Transaction check error:
...
2019-09-09 11:08:08.404 DEBUG    PID: 10967 leapp.workflow.FactsCollection.target_userspace_creator: Error Summary
2019-09-09 11:08:08.405 DEBUG    PID: 10967 leapp.workflow.FactsCollection.target_userspace_creator: -------------
2019-09-09 11:08:08.407 DEBUG    PID: 10967 leapp.workflow.FactsCollection.target_userspace_creator: Disk Requirements:
2019-09-09 11:08:08.409 DEBUG    PID: 10967 leapp.workflow.FactsCollection.target_userspace_creator:    At least 40MB more space needed on the / filesystem.
bocekm commented 5 years ago

Are you using the latest build from leapp master? The error message has been improved a bit: https://github.com/oamg/leapp/pull/552 EDIT: Of course, it does not solve the issue that the upgrade is not inhibited.

fernflower commented 5 years ago

Probably not the latest, some from last week. But thanks.

fernflower commented 5 years ago

Another variant of trace

  File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 994, in _getFile
    ) % (os.path.dirname(local), format_number(avail), format_number(long(size)))
RepoError: Insufficient space in download directory /var/cache/yum/x86_64/7Server/rhel-7-server-rpms
    * free   37 M
fernflower commented 5 years ago

As long as out of memory issue can affect several actors in some fairly unexpected ways (like subcommand invocation trouble in the middle of the processing) it was decided not to rush to develop a quickfix\workaround but to think about a global approach like introducing new stage\stage(s) are reviewing\refactoring the ones we already have.