search

NSLS-II / docs

Standards Documentation for NSLS-II DAQ and Analysis
https://nsls-ii.github.io
BSD 2-Clause "Simplified" License
2 stars 13 forks source link

Draft requirements. #54

Closed danielballan closed 7 years ago

danielballan commented 7 years ago

A quick start on the list requested by Richard this morning. I already have 20 points and have not yet addressed several topics. Please comment and add.

attn @stuartcampbell

tacaswell commented 7 years ago

attn @NSLS-II/dama

cowanml commented 7 years ago

It's a deployment detail, but one with significant implications for all sorts of stuff. Maybe it doesn't belong in requirements, but something like these bullets do?

 - backups

 - how easy or difficult it will be to do adhoc queries across beamlines
   for annual reporting, usage data, etc

 - how easy or difficult it will be to create a syncweb/ispyb type
   web interface that can show users all their work, regardless of beamline

 - etc

The mongo way of doing the 2nd and 3rd items could very well be to have both an instance at each beamline and a central store?

tacaswell commented 7 years ago

I agree those 3 points should be in the requirements. Probably want to say something about feeding usage data back into the data retention policy?

klauer commented 7 years ago

Who's the target audience of this document? What's the interest in generating these points so late in the game?

danielballan commented 7 years ago

Who's the target audience of this document? What's the interest in generating these points so late in the game?

It's partly to help on-board new hires, partly to explain ourselves to the outside world, and partly to ensure that we all are marching in the same direction. Whether or not it would have been possible/desirable to do this earlier in the game is arguable but moot.

And directly, it's because Richard asked us to do it. ;- ) But I'm not sure yet that this is what he had in mind.

cowanml commented 7 years ago

On Wed, 14 Dec 2016, Thomas A Caswell wrote: ...

Probably want to say something about feeding usage data back into the data retention policy?

I'm not sure I understand? I guess it should feedback into almost everything we do to some extent?

tacaswell commented 7 years ago

@cowanml Something like (I am making this up as I type so not a well thought out proposal, just something of the flavor)

with similar policies for flushing data out of the lower teirs.

cowanml commented 7 years ago

On Wed, 14 Dec 2016, Li Li wrote:

In source/requirements.rst: ... +Data Model +========== ... +* A facility or a beamline can impose extra required fields in the schema at

  • configuration time (e.g. requiring username, SAF number, proposal ID, or some
  • beamline-specific metadata). +* At NSLS-II, there is yet no facility-wide standard for collecting information
  • identifying the owner of a data set. We await guidance from the user task
  • force on this issue. Ultimately, we expect that a set of required fields will
  • likely be imposed at the facility level, once the tooling is in place to
  • authenticate the user and validate their inputs.

Agree. And proposal_id is already mentioned in the previous point. Basically user information is not critical for now and can be waited for future direction.

I would argue very strongly that nsls2 should be imposing saf# and proposal_id as required fields for all beamlines. They're kinda critical to be able to do any sort of reasonable reporting. They're very basic fields; I don't think we should be waiting to have a means to validate them.

-matt

tacaswell commented 7 years ago

The discussion of saf#/proposal_id is getting too much into the weeds, I think the 3 (nested?) bullet points are:

stuartcampbell commented 7 years ago

Hi, what state would you say this is in ? Safe to merge as is for now and then work to refine ? Or do we need to work on it before merging ?

tacaswell commented 7 years ago

Probably safe to merge as-is. We were waiting on feedback from Richard.