search

dmwm / WMCore

Core workflow management components for CMS.
Apache License 2.0
46 stars 107 forks source link

WorkflowUpdater: Renaming files and moving them to a different FS location via os.rename fails with Err 18 #12112

Open khurtado opened 1 month ago

khurtado commented 1 month ago

Impact of the bug WorkflowUpdaterPoller crashes when running unit tests where /tmp is binded inside the container from the host.

Describe the bug When writing the pileup json, in the following unit test (EL9) tries to rename and move the file to /tmp, failing with an Invalid cross-device link error.

 File "/home/cmsbld/WMCore/test/python/WMComponent_t/WorkflowUpdater_t/WorkflowUpdaterPoller_t.py", line 147, in testWritePileupJson
    writePileupJson(self.sandbox1, jdict, self.logger, tmpFile)
  File "/home/cmsbld/WMCore/src/python/WMComponent/WorkflowUpdater/WorkflowUpdaterPoller.py", line 223, in writePileupJson
    os.rename(ofile, dest)
[Errno 18] Invalid cross-device link: '/home/cmsbld/WMCore/test/python/WMComponent_t/WorkflowUpdater_t/new-SC_MultiPU_Agent227_Val_240110_215719_7133-Sandbox.tar.bz2' -> '/tmp/WMComponent-Sandbox.tar.bz2'

How to reproduce it Run testWritePileupJson unit test above, inside docker.

Expected behavior File should be placed at the destination successfully.

Additional context and error message An alternative solution would be to try shutil.move or shutil.copy and shutil.remove accordingly.

Related meta issue: https://github.com/dmwm/WMCore/issues/11978