search

JuliaHEP / UpROOT.jl

Julia package to access CERN ROOT files, wraps Python package uproot
Other
15 stars 3 forks source link

Can plotting recipies for TH1D, TGraph, TGraphError be a part of the package? #10

Open mmikhasenko opened 4 years ago

mmikhasenko commented 4 years ago

Here is how I plot TGraphError extracted from TFile, 

f = TFile("test.root")
c = f["myTGE"]  # typo of object PyObject <b'TGraphErrors' b'myTGE' 0x00003a8740f0>
plot(
    vcat(c.xvalues...),
    vcat(c.yvalues...),
    xerr = vcat(c.xerrors...), yerr = vcat(c.yerrors...),
    lab="",
    xlab = latexstring(replace(c.xlabel,"text" => "mathrm")),
    ylab = latexstring(replace(c.ylabel,"text" => "mathrm")),
    title = c.title
)

I think it would make a lot of sense to write a few recipes. I am just not sure how to check subtypes of these PyObject without writing the parser

oschulz commented 4 years ago

Well, we should definitely add support for THxx and implement a conversion to StatsBase.Histogram.

And you're right, Plots recipes for some things would be useful, too.

mmikhasenko commented 4 years ago

do you know how to check subtypes of the PyObject? I am afraid it cannot be done by the dispatch, i.e. requires conversion to Julia structure upfront.

I saw you do something similar in the py2jl function https://github.com/JuliaHEP/UpROOT.jl/blob/master/src/pyjlconv.jl#L45

how will this work for TH, TGraph?

oschulz commented 4 years ago

I think it will just be done via further additions to py2jl, checking the type on the python side and then selecting an appropriate Julia wrapper or type to convert to.

mmikhasenko commented 4 years ago

any experience with reading TCanvas in TFile using uproot? I cannot find the way to extract content from PyObject <Undefined_TCanvas at 0x00005067a0f0>.

o._fields # Vector{Any} with 0 elements
o._members # PyObject <bound method ROOTStreamedObject._members of <class 'uproot.rootio.Undefined_TCanvas'>>
o.__getattribute__ # PyObject <method-wrapper '__getattribute__' of Undefined_TCanvas object at 0x000000005067A0F0>
oschulz commented 4 years ago

any experience with reading TCanvas in TFile using uproot?

Haven't tried yet - but if Python can read it, there must be a way to get at it. :-)

mmikhasenko commented 2 years ago

I have got the next opportunity (an urgent need) to get the extraction of the canvas data working.

julia> c0 = TFile("file.root")["page0"]
PyObject <Undefined_TCanvas at 0x00000256f6d0>
julia> keys(c0)
39-element Vector{Symbol}:
 :__class__
 :__delattr__
 :__dict__
 :__dir__
 :__doc__
 :__eq__
 :__format__
 :__ge__
 :__getattribute__
 :__gt__
 :__hash__
 :__init__
 :__init_subclass__
 :__le__
 :__lt__
 :__metaclass__
 :__module__
 :__ne__
 :__new__
 :__reduce__
 :__reduce_ex__
 :__repr__
 :__setattr__
 :__sizeof__
 :__str__
 :__subclasshook__
 :__weakref__
 :_classname
 :_classversion
 :_copycontext
 :_cursor
 :_fields
 :_members
 :_postprocess
 :_readinto
 :_recarray
 :_recarray_dtype
 :classname
 :read

any idea where to look for the right methods?

oschulz commented 2 years ago

Oh dear ... uhm, not really, without digging depper into ROOT myself ...

@tamasgal , @Moelf you're much deeper in ROOT's internals than me right now - any idea? Or can UnROOT do something that help Misha with this already (or provide an easier path to it at least)?

tamasgal commented 2 years ago

I am a bit confused. What is the main issue? Accessing the data from a TCanvas object?

mmikhasenko commented 2 years ago

yes, indeed. To get the data points and the curves from the plot image

tamasgal commented 2 years ago

Is it possible to upload the ROOT-file somewhere? (e.g. here, as a ZIP file)

mmikhasenko commented 2 years ago

sure plot.ReIm.pre.att0911.pdf.zip you get the canvas by

c0 = TFile("plot.ReIm.pre.att0911.pdf.root")["page0"]
mmikhasenko commented 2 years ago

In the ROOT I would do something like this:

root> c1 = (TCanvas*)page0->GetListOfPrimitives()->First()
root> l1 = c1->GetListOfPrimitives()->FirstLink()
root> m1 = (TMultiGraph*)l1->Next()->GetObject()
root> m1->GetListOfGraphs()->GetEntries()
(int) 11
root> g1 = (TGraph*)m1->GetListOfGraphs()->First()
root> N = g1->GetN()
(int) 85
root> xv = g1->GetX()
root> yv = g1->GetY()

ah, root

tamasgal commented 2 years ago

I fear that this stuff is deeply connected with the underlying C++ function mapping. While it might be possible to retrieve the data points if there is valid streamer info (it seems uproot already fails to understand the primitives, that's why it's an Undefined_TCanvas, but that's more of a guess), I am pretty sure that the other information (curves) is pretty much useless without the C++ origins.

I assume it's worthless to ask if there is a possibility to change the output ROOT format to something which is less dependent of the C++ framework...

mmikhasenko commented 2 years ago

I guess, this points to the same direction

julia> pycall(c0, PyObject)[:GetListOfPrimitives]()
PyError ($(Expr(:escape, :(ccall(#= .julia\packages\PyCall\3fwVL\src\pyfncall.jl:43 =# @pysym(:PyObject_Call), PyPtr, (PyPtr, PyPtr, PyPtr), o, pyargsptr, kw))))) <class 'TypeError'>                                                                                                                         
TypeError("'Undefined_TCanvas' object is not callable")

Does it?

I assume it's worthless to ask if there is a possibility to change the output ROOT format to something which is less dependent of the C++ framework... haha.

No way around unwrapping the objects in the root terminal ... :(

Moelf commented 2 years ago

unwrapping the objects in the root terminal

call root from shell> and parse the stdout :(((

mmikhasenko commented 2 years ago

AAA! TBufferJSON::ToJSON(page0) works as well as TBufferJSON::ExportToFile("page0.json", page0)

mmikhasenko commented 2 years ago

BTW, the root structures in JSON look so simple. I thought julia structures can hold it very well. One can even generate the structures automatically from json file.

            }, {
              "_typename" : "TGraph",
              "fUniqueID" : 0,
              "fBits" : 1032,
              "fName" : "Graph",
              "fTitle" : "",
              "fLineColor" : 2,
              "fLineStyle" : 2,
              "fLineWidth" : 1,
              "fFillColor" : 1,
              "fFillStyle" : 1001,
              "fMarkerColor" : 1,
              "fMarkerStyle" : 1,
              "fMarkerSize" : 1,
              "fNpoints" : 100,
              "fX" : [0.81, 0.826969696969697, 0.843939393939394, 0.860909090909091, 0.877878787878788, 0.894848484848485, 0.911818181818182, 0.928787878787879, 0.945757575757576, 0.962727272727273, 0.97969696969697, 0.996666666666667, 1.01363636363636, 1.03060606060606, 1.04757575757576, 1.06454545454545, 1.08151515151515, 1.09848484848485, 1.11545454545455, 1.13242424242424, 1.14939393939394, 1.16636363636364, 1.18333333333333, 1.20030303030303, 1.21727272727273, 1.23424242424242, 1.25121212121212, 1.26818181818182, 1.28515151515152, 1.30212121212121, 1.31909090909091, 1.33606060606061, 1.3530303030303, 1.37, 1.3869696969697, 1.40393939393939, 1.42090909090909, 1.43787878787879, 1.45484848484849, 1.47181818181818, 1.48878787878788, 1.50575757575758, 1.52272727272727, 1.53969696969697, 1.55666666666667, 1.57363636363636, 1.59060606060606, 1.60757575757576, 1.62454545454545, 1.64151515151515, 1.65848484848485, 1.67545454545455, 1.69242424242424, 1.70939393939394, 1.72636363636364, 1.74333333333333, 1.76030303030303, 1.77727272727273, 1.79424242424242, 1.81121212121212, 1.82818181818182, 1.84515151515152, 1.86212121212121, 1.87909090909091, 1.89606060606061, 1.9130303030303, 1.93, 1.9469696969697, 1.96393939393939, 1.98090909090909, 1.99787878787879, 2.01484848484849, 2.03181818181818, 2.04878787878788, 2.06575757575758, 2.08272727272727, 2.09969696969697, 2.11666666666667, 2.13363636363636, 2.15060606060606, 2.16757575757576, 2.18454545454545, 2.20151515151515, 2.21848484848485, 2.23545454545455, 2.25242424242424, 2.26939393939394, 2.28636363636364, 2.30333333333333, 2.32030303030303, 2.33727272727273, 2.35424242424242, 2.37121212121212, 2.38818181818182, 2.40515151515152, 2.42212121212121, 2.43909090909091, 2.45606060606061, 2.4730303030303, 2.49],
              "fY" : [12.9682208066244, 12.2094997483641, 11.4953288592651, 10.6288623242888, 9.72012339765948, 9.13094068009057, 8.61332999571456, 8.2844855461918, 8.40400332795204, 8.94647394275909, 9.99194057534541, 11.8358371437289, 14.5303443527173, 18.1650509327863, 23.2639603667477, 29.8207803869633, 38.0170630705138, 48.9372324281355, 62.2211665324215, 78.8905861575526, 99.9940144865317, 125.257394457864, 158.260701113804, 198.190520728579, 248.27298647972, 312.274175721447, 389.123385331609, 494.343851590691, 621.108034680253, 793.036787487114, 1010.28276127011, 1299.76649271051, 1685.09530180919, 2185.00350639352, 2874.72900835389, 3738.74127009556, 4914.01136960223, 6343.42029830824, 8166.09847664562, 10331.3897775627, 12912.6300304604, 15905.4079903201, 19300.4365309511, 23096.6131718114, 27170.6422072014, 31425.8047139083, 35539.159595438, 39263.7643207574, 42100.977290294, 43796.2938246386, 44075.7183365252, 42966.4422322125, 40721.5586158681, 37625.9638532562, 34086.3892159137, 30321.3388155636, 26545.8464041568, 22839.3549352592, 19267.2598756726, 15935.5812340163, 12976.276746148, 10554.4969779581, 8761.0310039344, 7565.83540280415, 6841.49325553332, 6429.43173106948, 6194.21758970629, 6044.95849612353, 5926.77485317235, 5809.50465962095, 5679.59357571744, 5531.54077350329, 5364.45272664233, 5179.745501022, 4980.11037861504, 4768.72837959997, 4548.64458271685, 4323.20975051944, 4095.35370699651, 3867.81672514108, 3642.62411929312, 3422.03400541244, 3207.79112899683, 3001.07132780383, 2802.95685231493, 2614.34877244856, 2435.80212226489, 2267.64370902316, 2109.8181280774, 1962.44290466261, 1825.41550485699, 1698.73959196222, 1581.28791397172, 1473.58373042074, 1374.86265862507, 1283.7273882579, 1201.24301989937, 1126.2575418734, 1058.33190043502, 997.02766957764],
              "fFunctions" : {
                "_typename" : "TList",
                "name" : "TList",
                "arr" : [],
                "opt" : []
              },
              "fHistogram" : {
                "_typename" : "TH1F",
                "fUniqueID" : 0,
                "fBits" : 512,
                "fName" : "Graph",
                "fTitle" : "",
                "fLineColor" : 602,
                "fLineStyle" : 1,
                "fLineWidth" : 1,
                "fFillColor" : 0,
                "fFillStyle" : 1001,
                "fMarkerColor" : 1,
                "fMarkerStyle" : 1,
                "fMarkerSize" : 1,
                "fNcells" : 102,
                "fXaxis" : {
                  "_typename" : "TAxis",
                  "fUniqueID" : 0,
                  "fBits" : 0,
                  "fName" : "xaxis",
                  "fTitle" : "",
                  "fNdivisions" : 510,
                  "fAxisColor" : 1,
                  "fLabelColor" : 1,
                  "fLabelFont" : 42,
                  "fLabelOffset" : 0.005,
                  "fLabelSize" : 0.035,
                  "fTickLength" : 0.03,
                  "fTitleOffset" : 1,
                  "fTitleSize" : 0.035,
                  "fTitleColor" : 1,
                  "fTitleFont" : 42,
                  "fNbins" : 100,
                  "fXmin" : 0.642,
                  "fXmax" : 2.658,
                  "fXbins" : [],
                  "fFirst" : 0,
                  "fLast" : 0,
                  "fBits2" : 0,
                  "fTimeDisplay" : false,
                  "fTimeFormat" : "",
                  "fLabels" : null,
                  "fModLabs" : null
                },
                "fYaxis" : {
                  "_typename" : "TAxis",
                  "fUniqueID" : 0,
                  "fBits" : 0,
                  "fName" : "yaxis",
                  "fTitle" : "",
                  "fNdivisions" : 510,
                  "fAxisColor" : 1,
                  "fLabelColor" : 1,
                  "fLabelFont" : 42,
                  "fLabelOffset" : 0.005,
                  "fLabelSize" : 0.035,
                  "fTickLength" : 0.03,
                  "fTitleOffset" : 1,
                  "fTitleSize" : 0.035,
                  "fTitleColor" : 1,
                  "fTitleFont" : 42,
                  "fNbins" : 1,
                  "fXmin" : 0,
                  "fXmax" : 48482.4617216231,
                  "fXbins" : [],
                  "fFirst" : 0,
                  "fLast" : 0,
                  "fBits2" : 0,
                  "fTimeDisplay" : false,
                  "fTimeFormat" : "",
                  "fLabels" : null,
                  "fModLabs" : null
                },
                "fZaxis" : {
                  "_typename" : "TAxis",
                  "fUniqueID" : 0,
                  "fBits" : 0,
                  "fName" : "zaxis",
                  "fTitle" : "",
                  "fNdivisions" : 510,
                  "fAxisColor" : 1,
                  "fLabelColor" : 1,
                  "fLabelFont" : 42,
                  "fLabelOffset" : 0.005,
                  "fLabelSize" : 0.035,
                  "fTickLength" : 0.03,
                  "fTitleOffset" : 1,
                  "fTitleSize" : 0.035,
                  "fTitleColor" : 1,
                  "fTitleFont" : 42,
                  "fNbins" : 1,
                  "fXmin" : 0,
                  "fXmax" : 1,
                  "fXbins" : [],
                  "fFirst" : 0,
                  "fLast" : 0,
                  "fBits2" : 0,
                  "fTimeDisplay" : false,
                  "fTimeFormat" : "",
                  "fLabels" : null,
                  "fModLabs" : null
                },
                "fBarOffset" : 0,
                "fBarWidth" : 1000,
                "fEntries" : 0,
                "fTsumw" : 0,
                "fTsumw2" : 0,
                "fTsumwx" : 0,
                "fTsumwx2" : 0,
                "fMaximum" : 48482.4617216231,
                "fMinimum" : 0,
                "fNormFactor" : 0,
                "fContour" : [],
                "fSumw2" : [],
                "fOption" : "",
                "fFunctions" : {
                  "_typename" : "TList",
                  "name" : "TList",
                  "arr" : [],
                  "opt" : []
                },
                "fBufferSize" : 0,
                "fBuffer" : [],
                "fBinStatErrOpt" : 0,
                "fStatOverflows" : 2,
                "fArray" : [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
              },
              "fMinimum" : -1111,
              "fMaximum" : -1111
            }, {
Moelf commented 2 years ago

It's not the data structure that is the problem, it's the parsing of nested C++ classes that lead to the final result