Closed KaraMelih closed 2 years ago
@Storreslara @joesmolsky could you guys please review the commits? I'm also going to make a small PR on the CS (to Sebastian's stats branch)
I also updated the requirements.txt
because one package was missing and it is needed for the docs.
I updated some other packages, and made flexible dependencies (or whatever it is called when you set package~=x.y.z
with a tilde). I set the same dependencies for both snews_pt and snews_cs so that both can be run with the same environment.
They don't have to be exactly the same, but the mutual package's versions have to be the same.
ipython
actuall has a latest version 8.0.2
but for some reason, sphinx (readthedocs) can't find anything greater than 7.32.0
so I set it to this.
What changed;
--firedrill/--no-firedrill
options and has docs seesnews_pt --help
(defaults True)snews_pt run-scenarios
also has the firedrill option (default True)utcnow
and appends to all messages'sent_time'
field.'sent_time'
is used to stamp the'_id'
This
'sent_time'
is used to stamp the'_id'
because it is always created whereas before we were using'machine_time'
and if nothing is passed this wasNone
by default. Thus, all the messages from the same detector had the same id e.g."4_CoincidenceTier_None"
we did not notice this problem before but now that we have mongo it complains about messages having the exact same id. Replacing this with a utcnow timestamp fixes it. Accordingly, the tests had to be fixed. We also missed it there, all the test cases were testing for"_id"=="XX_CoincidenceTier_None"
. Now I just check the rest of the keys and not the timestamp. Furthermore, thesnews_cs
has some functions that were checking the'sent_time'
, apparently we removed it at some point from pt but kept it in cs.Now that we have a stamp when it is sent (if user also provides a
machine_time
with their timestamp) we can measure the latency between the user submitting and snews_pt publishing. As far as I see this is ~ms levels.On the
snews_cs
side we have"received_time"
and this can tell us the latency between the message being published, and after kafka flight it being received at Purdue.So we have;
"neutrino_time"
- time of the neutrino burst "machine_time
" - the time experiment thinks they sent (optional) "sent_time
" - the time we stamp right beforehop.Stream.write()
"received_time
" - the timesnews_cs
reads it from the hop streamThus, we can infer several latencies.