Uniform generator for time series and histograms

[tsengj10]

Generator code for TimeSeries and TimeHist objects, starting with just a uniform event rate over a given span.

One can create a TimeSeries or TimeHist using the NewTimeSeries or NewTimeHist generator. Uniform then adds to the object. One can then string together further generators to add to the objects.

We'll need to add some more utilities, like setting new start times and adding objects together, to make this fully functional. This would be an alternative to creating a bunch of objects in the data['gen'], with a Combine at the end, which was getting a bit clumsy.

[tsengj10]

Need to pull this on top of master for it to work, because renderers/init.py refers to PickleOutput now

[tsengj10]

This PR mostly stands on its on (and I think shouldn't cause trouble for others), so going to merge soon.

[mcolomermolla]

I see that for the pickle implementation (still looking at it), you have come back to old ".py" config files instead of csv: snewpdag/data/test-pickle-in.py

Shouldn't it be also a csv?

[tsengj10]

Sometimes I find the old .py form easier for simple graphs. But happy to make csv versions in a new PR!

[mcolomermolla]

Some comments:

• NewTimeHist: I would replace time_span by an end_time
• NewTimeSeries: why it one returns the start_time instead of all event times? I think I don't get the idea of this script
• TimeHist: I would not use default bins to zero to a non-zero value, as indeed it throws and error, and I think by default it should run for an arbitrary number of bins
• TimeSeries and TimeHist : I don't like the idea of having times as an array of (s,ns) times, it is just a factor 1000 and will be a lot more of data (and tuple structure) to handle Question: I don't reallysee how this connect to snewpdag/plugins/gen/TimeSeries.py after all

[tsengj10]

Thanks @mcolomermolla for the comments. A few replies:

• I was thinking that for supernova data we have a start time (or something related to the burst), and an amount of time we record after that.
• A time series doesn't have an end in principle. In fact, it doesn't really even have a start, but just a reference time. All the other times are just offsets from that time. Would we prefer to have the object discard new entries because it fell outside a window (which is what would happen in a histogram)?
• I could go either way on a default (valid) nbins value 4for TimeHist.
• (s,ns) came from the fact that most times from experiments are counted in blocks of ns or possibly microseconds, but unfortunately 0.1 or such fractions in base 2 are continued fractions rather than exactly represented. On the other hand, the loss of precision in float64 (as long as the integer part is reasonable) is pretty minimal.
• Ah, I forgot about the gen.TimeSeries plugin... In any case, I was trying to tidy things up not by replacing code at first, but by prototyping the new classes on the side, independent of the working code base. Then we'd integrate the new classes after we've worked out the kinks and (hopefully) find them useful.

[mcolomermolla]

• I agree with that, I was just not seeing that "an amount of time after that" in the script that is defined/recorded
• You're right in theory for the experiment continuous data taking, there is no start and end for the event times (time series). However, for the practical case of a CCSN trigger, the experiment will define a start an an end that they will send to SNEWS as "the window the the supernova happens", and not hours or days of data. So I think we will get a start an end from experiments data first and last event they send.
• Yep, just a default value different from zero.
• (s,ns): I see, in any case yes, I think the amount of information lost is minimal (as in reality we won't achieve ns recision on the deltat or the T0). Also, experiments like IceCube and KM3NeT won't have a ns timing of events.
• Yes, indeed, I see that the values.TimeSeries is defining a TimeSeries class, not really doing the "generator" part by randomising as in gen.TimeSeries

[tsengj10]

Good point about the duration - I'll add it to values.TimeSeries. And possibly call it "duration" rather than "time_span".

Also will try to add the non-zero default nbins value - though I now recall why I didn't, which is that I had allowed the nbins parameter to override (i.e., shorten) the length of the data array in determining how many bins the histogram was to have. Having a non-zero default value makes it impossible to specify "I want my histogram to have 100 bins, so truncate my array which happens to have 110 bins". On the other hand, there are other ways to specify that outside of the arguments, but in any case it'll change the semantics of the nbins vs data.

Finally, I remembered that we automatically lose precision in the timestamp, so we may need to keep (s,ns) for anything which could be a full timestamp. The reason is that the usual Unix-epoch timestamp uses nearly its full 32 bits - if I remember correctly, it turns over in the 2030's, possibly a bigger "Y2K" problem than the one in 1/1/2000. A 64-bit float has 53 significant binary digits, so we're down to about 21 bits left for the fraction. This only gets us down to microsecond granularity.

The detectors probably won't get down to ns timing granularity, but they do go below microseconds. The main SNO+ trigger window, for instance, is 400ns, so if we cut the timestamp off at the microsecond level, it's possible to have two events with the same timestamp. That could be a problem if we have to do some elementary data reduction within snewpdag.

Anyway, I'll see if I can reduce the number of ways one must use (s,ns), or indeed nanoseconds. For instance, by adding methods which take floats instead of long ints.

That will be another PR, because I tangled up the git history a little and need to merge to straighten it out...