scikit-hep / uproot3

ROOT I/O in pure Python and NumPy.
BSD 3-Clause "New" or "Revised" License
314 stars 67 forks source link

Issue writing root files to cernbox with xrootd #386

Open marinang opened 4 years ago

marinang commented 4 years ago

I am unable to write a tfile using Xrootd on cernbox/eos for instance, it says FileNotFoundError. I attached an example of an existing file that I tried to overwrite.


In [1]: import uproot

In [2]: f = uproot.open("root://eosuser.cern.ch//eos/user/m/mmarinan/example.root")

In [3]: f.keys()
Out[3]: [b't;1']

In [4]: uproot.recreate("root://eosuser.cern.ch//eos/user/m/mmarinan/example.root")
---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
<ipython-input-4-d3d6e0d9c1a2> in <module>
----> 1 uproot.recreate("root://eosuser.cern.ch//eos/user/m/mmarinan/example.root")

~/packages/anaconda3/envs/pyana/lib/python3.7/site-packages/uproot/write/TFile.py in __init__(self, path, compression)
    202 class TFileRecreate(TFileUpdate):
    203     def __init__(self, path, compression=uproot.write.compress.ZLIB(1)):
--> 204         self._openfile(path, compression)
    205         self._writeheader()
    206         self._writerootdir()

~/packages/anaconda3/envs/pyana/lib/python3.7/site-packages/uproot/write/TFile.py in _openfile(self, path, compression)
     43         self._treedict = {}
     44
---> 45         self._sink = uproot.write.sink.file.FileSink(path)
     46         self._path = path
     47         self._filename = os.path.split(path)[1].encode("utf-8")

~/packages/anaconda3/envs/pyana/lib/python3.7/site-packages/uproot/write/sink/file.py in __init__(self, path)
     11     def __init__(self, path):
     12         self._path = path
---> 13         self._sink = open(path, "wb+")
     14         # self._pos = 0
     15

FileNotFoundError: [Errno 2] No such file or directory: 'root://eosuser.cern.ch//eos/user/m/mmarinan/example.root'
jpivarski commented 4 years ago

We may need to document this better, but uproot can't write remote files. Writing a ROOT file involves a lot of backtracking—we made strong assumptions about having a local file handle. It wouldn't be impossible to handle remote writing, although it might be difficult to do it efficiently.

To not get your hopes up, I'll say that we don't plan on adding this feature—it's beyond the scope we had been considering. (Although @reikdas, if you think it's not too hard, let me know.) I'm going to keep this open for better documentation: we should state clearly that uproot can only write to local files, in the README, the docstrings, and in the error message you get if you try it.

marinang commented 4 years ago

Ok that explains why. It's not a big deal I can write it locally and send it to cernbox with xrootd commands. Thanks.

reikdas commented 4 years ago

I plan to take a closer look at this. Reopening the issue as a reminder.