Open NiclasEich opened 3 years ago
added the root-version used
I encountered some interesting/weird behaviour while trying to even reduce the example:
The following script only SOMETIMES failes with a segmentation-fault. Thus far this looks like a stochastic behaviour to me, with the script failing in 50% of the cases and the rest running fine.
import numpy as np
import awkward as ak
import uproot3 as ur
import root_numpy as rn
f_path = "/tmp/test_uproot_writing.root"
keys = ["key-a"]
print("Creating New root-file with uproot")
with ur.recreate( f_path ) as out_file:
arr = ak.to_awkward0( ak.Array( [np.zeros( (20))[:np.random.randint(4, 20)] for i in range(100)] ) )
out_file["ttree"] = ur.newtree( {"key-a": ur.newbranch(np.dtype("f8"), size="n")})
out_file["ttree"].extend({"key-a": arr, "n": arr.counts})
print("Trying to load the file with root_numpy")
nparr = rn.root2array( [f_path], treename="ttree", stop=None, branches=keys)
print(nparr)
Further, when substituting
arr = ak.to_awkward0( ak.Array( [np.zeros( (20))[:np.random.randint(4, 20)] for i in range(100)] ) )
with arr = ak.to_awkward0( ak.Array( [np.zeros( (20)) for i in range(100)] ) )
, the latter array ALWAYS fails, so it might have to do something with how the awkward-Array is handled...
Letting both script-versions run 50 times:
The
with ur.recreate( f_path ) as out_file:
arr = ak.to_awkward0( ak.Array( [np.zeros( (20))[:np.random.randint(4, 20)] for i in range(100)] ) )
out_file["ttree"] = ur.newtree( {"key-a": ur.newbranch(np.dtype("f8"), size="n")})
out_file["ttree"].extend({"key-a": arr, "n": arr.counts})
Version failed 26/50 times
and the
with ur.recreate( f_path ) as out_file:
arr = ak.to_awkward0( ak.Array( [np.zeros( (20)) for i in range(100)] ) )
out_file["ttree"] = ur.newtree( {"key-a": ur.newbranch(np.dtype("f8"), size="n")})
out_file["ttree"].extend({"key-a": arr, "n": arr.counts})
failed 50/50 times
Same problem with the newest verrsion:
Version:
uproot: 3.14.4
awkward: 1.3.0
root_numpy: 4.8.0
When the script is succesful in loading the uproot-written tree with root, I encounter vastly random values in all ranges of magnitude, so I suspect that succes is still corrupted somehow internally.
I'm following this; thanks for submitting it! (Sorry for my silence so far.)
My action is going to be to ensure that this case is not broken in Uproot 4 (follow it on https://github.com/scikit-hep/uproot4/discussions/321), which is my highest programming priority right now, though non-programming priorities (summer conferences) have prevented me from updating it for the last few weeks. It will get done this summer, though.
What you're seeing with non-deterministic segfault vs wrong values is typical of ROOT file-writing: if one byte is out of place, reading it back might trip up in different ways. (It might depend on uninitialized values in some array somewhere.)
Your example is an indication that the file is not being written correctly, though I don't think it has ever been tested with root-numpy before. (I think the unit tests use PyROOT to test read-back, other than Uproot itself.) It could be that root-numpy is revealing an error that hadn't been revealed before by reading it in a different way. All of the information needed to read a jagged branch is contained in that branch—although it is usually linked to a "count leaf" in another branch (size
in the Uproot 3 API), it is not necessary to read the counts to get the jagged array. It could be that the count leaf is incorrectly written, but only root-numpy attempts to read the count leaf. (Uproot definitely doesn't; I don't know about PyROOT.)
Thanks for the answer! Is there already a version/branch of uproot4 that would allow writing trees?
As mentioned earlier, opening the broken branches in a TBrowser also results in a seg-fault
See https://github.com/scikit-hep/uproot4/discussions/321 — the Uproot 4 feature is in progress.
By testing the Uproot 3 feature in PyROOT, we also didn't see what it would do in the TBrowser, which might try to access more, like root-numpy.
Hello,
I am trying to write and then read a tree with uproot. Writing the tree and reading it again with uproot is no problem but I encounter a segmentation fault, when trying to open Jagged-Array branches of the tree with other root-tools like a TBrowser or the python root_numpy module. I used the awkward-array documentation on how to write the tree.
A minimal Example of how I write and how it fails would be this:
With the stack-trace:
Further:
using root-version 6.18/04