search

cybergreen-net / pm

Tech project management repo (issue tracker only)
2 stars 1 forks source link

Do a data load (with new UPnP/SSDP data) #95

Closed rufuspollock closed 7 years ago

rufuspollock commented 7 years ago
aaronkaplan commented 7 years ago

On 08 Mar 2017, at 16:06, Rufus Pollock notifications@github.com wrote:

• Clarify whether this is new risk or continues the SSDP data (and if this is new data source what will happen with time series etc)

--> according to Cosive : continue the SSDP data. We will switch over to that data after a TBD cut-off day.

• Update https://github.com/cybergreen-net/refdata-risk with new risk info (@kxyne or @aaronkaplan ) Same SSDP

• Run new load after data is live in private store • Check good in staging site — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

rufuspollock commented 7 years ago

@aaronkaplan thanks for clarification - what about sub-bullet about same risk / different data source question.

aaronkaplan commented 7 years ago

@rufuspollock regarding the lack of continuity: we checked the data. The info I got from @kxyne is that we can move in the new SSDP data per 16th of Jan 2017. That can be the cut-off day. We will continue to keep the old data . And yes, that means there will be a "stair"/jump in the graph. But for that we would like to have an annotiation in the graph. Or at least a text somewhere explaining the jump ("on the 16th of January we moved to a different measurement method - hence the jump in the graph").

rufuspollock commented 7 years ago

@aaronkaplan we have no facility to do annotations at the moment -- that would be next phase of work.

Re the text: definitely something to flag on ux review call tomorrow.

aaronkaplan commented 7 years ago

On 08 Mar 2017, at 19:10, Rufus Pollock notifications@github.com wrote:

@aaronkaplan we have no facility to do annotations at the moment -- that would be next phase of work.

Re the text: definitely something to flag on ux review call tomorrow.

Hereby flagged :)

I am also fine (for now) with finding a nice place on the stats site where we explain this as text.

ok?

kxyne commented 7 years ago

Also worth noting that censys does have missing weeks so we still need to account for this and interpolate or whatever.

zelima commented 7 years ago

@kxyne Is data already in private store? I noticed only three file in new folder named censys-upnp and plus 2 file uploads for Mirai, is that all?

So my general question would be are we ready to go? should I run script?

kxyne commented 7 years ago

I was going to fix up the automation for the data package. There are days missing from censys but we're current now for it and mirai I believe.

On Thu, 9 Mar 2017, 22:29 Irakli Mchedlishvili notifications@github.com wrote:

@kxyne https://github.com/kxyne Is data already in private store? I noticed only three file in new folder named censys-upnp and plus 2 file uploads for Mirai, is that all?

  • Also datapackage.json is not updated (which I can do manually for now as there are very few changes)
  • @aaronkaplan https://github.com/aaronkaplan Running new load as is (with only this updates) will result with in empty tables on country page as there will no new data for DNS, NTP etc...

So my general question would be are we ready to go? should I run script?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/cybergreen-net/pm/issues/95#issuecomment-285327276, or mute the thread https://github.com/notifications/unsubscribe-auth/AELd3erB2Q6M-QLRg_u2EpS-ikN8HGLUks5rj-KRgaJpZM4MW5F1 .

-- Kayne Naughton | Managing Director

M: +61-418-695-199 E: kayne.naughton@cosive.com W: www.cosive.com

zelima commented 7 years ago

@kxyne could you please update. What is status for this? Is this going to happen in this milestone?

rufuspollock commented 7 years ago

Moving to backlog now - it won't happen for this release and i now outside of SoW3.

zelima commented 7 years ago

FIXED data is live http://stats.cybergreen.net/ Note: there is gap in January and February.

aaronkaplan commented 7 years ago

The gap is ok and we can explain it. Thanks. However, please note that the data that Irakli loaded just now was not the UPnP data. It was open NTP project.