search

xpdAcq / mission-control

Releases, Installers, Specs, and more!
0 stars 4 forks source link

ENH: plot QOIs #27

Open CJ-Wright opened 7 years ago

CJ-Wright commented 7 years ago

@CJ-Wright commented on Sat Sep 02 2017

Now that we have some reduced data in the pipeline we are ready to start plotting some interesting QOIs.

CJ-Wright commented 7 years ago

This will require:

sbillinge commented 6 years ago

UC that will deliver this cycle:

  1. software automatically finds the locations of the first 3 "useful" (i.e., not the beamstop) peak maxima
  2. software plots peak position vs. independent process variable (index or temperature depending on the scan)

@sghose @dooryhee we want this initial version to work automatically (no user input) but we could extract a different dependent variable QOI if you let us know what you prefer.

Later there can be user-requested QOIs but we can't deliver that right now.

dooryhee commented 6 years ago

Very useful. Peak position … and amplitude ? E

From: Simon Billinge [mailto:notifications@github.com] Sent: Wednesday, May 16, 2018 6:02 PM To: xpdAcq/mission-control Cc: Dooryhee, Eric; Mention Subject: Re: [xpdAcq/mission-control] ENH: plot QOIs (#27)

UC that will deliver this cycle:

  1. software automatically finds the locations of the first 3 "useful" (i.e., not the beamstop) peak maxima
  2. software plots peak position vs. independent process variable (index or temperature depending on the scan)

@sghosehttps://github.com/sghose @dooryheehttps://github.com/dooryhee we want this initial version to work automatically (no user input) but we could extract a different dependent variable QOI if you let us know what you prefer.

Later there can be user-requested QOIs but we can't deliver that right now.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/xpdAcq/mission-control/issues/27#issuecomment-389679489, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ALyE3cbpsy1VkVVBrvYHrNb_-FNMRlWbks5tzKG_gaJpZM4PK-Fs.

CJ-Wright commented 6 years ago

The main issue here is that we need to merge (at least) two document streams, the stream which contains the peak positions from the analysis, and the stream which has the independent variable from the raw data. I think the merging can be done with ChainDB from regolith which is a recursive version of ChainMap. (@sbillinge see what I did there)

The merged streams will create a single document stream which can be passed to a plotting callback or even BestEffortCallback if I can get the hinting correct.

CJ-Wright commented 6 years ago

TODO:

CJ-Wright commented 6 years ago

Relevant PRs https://github.com/xpdAcq/SHED/pull/142 https://github.com/xpdAcq/xpdtools/pull/35 https://github.com/xpdAcq/xpdAn/pull/208

sbillinge commented 6 years ago

Here are some more precise UCs, which should go from easier (I hope) but still useful on down.

UC1:

  1. user starts a scan
  2. software automatically finds peak positions of all the peaks in I(Q) in a range up to 6 A-1 using a simple local maximum finder, with data on a finish grid.
  3. software determines how many plot windows to open
    1. If peaks are within 0.25 A-1 of each other they are in the same plot window, if they are further apart than this, they are in a separate plot window
  4. software opens this many plot windows, preferrably arranged in a reasonable grid (but separate windows, not subplots of the same figure).
  5. software plots the position vs. index of each peak in its window as the stream comes down. Since we don't know the correct range, have the plots autoscale. This may look bad if the data are coming quickly, but let's try it.
  6. at the end of the scan, save the QOI data in an easily replottable form (xy column format?)

UC1A:

  1. scientist wants to replay a previous scan
  2. scientist types replay_scan and gives a scan id and all the plot windows open up from that scan, including the QOI scans. This doesn't compete with live data, so would have to be working in a separate process.

UC1B

  1. scientist just wants to replot one or a few of the QOI scans.
  2. scientist double-clicks the qoi.xyz file that she is interested in and these plots open up. This doesn't compete with data acquisition or live plotting.

UC2: as UC1 but it plots the peak intensity

UC3 as UC1 but it is the PDF data. Let's try not doing it on Nyquist grid but doing it on a 0.01 gird. for the ranges, we could try harvesting all the peaks in the 1.3-6 A range and again, plot in the same figure if they are within 0.25A of each other

sbillinge commented 6 years ago

UC4: As UCs 1-4 but peak positions and intensities are extracted by fitting using and automated fitting program. We could code up Ran's peak finding program for example.

UC5: as UC2, but now plotting the PDF peak integrated area.

UC6: as UC5 but plot the PDF peak widths. Plot them all on the same axes as they should be all close in value.

UC5: As UCs 1-4 but after the 4th point, put a polynomial or a spline through the data on the plot

dooryhee commented 6 years ago

From: Simon Billinge [mailto:notifications@github.com] Sent: Tuesday, May 29, 2018 7:03 AM To: xpdAcq/mission-control Cc: Dooryhee, Eric; Mention Subject: Re: [xpdAcq/mission-control] ENH: plot QOIs (#27)

Here are some more precise UCs, which should go from easier (I hope) but still useful on down.

UC1:

  1. user starts a scan
  2. software automatically finds peak positions of all the peaks in I(Q) in a range up to 6 A-1 using a simple local maximum finder, with data on a finish grid. – that would be really awesome. A remark: people usually “know” their diffraction pattern by the d-spacing values.
  3. software determines how many plot windows to open
  4. If peaks are within 0.25 A-1 of each other they are in the same plot window, if they are further apart than this, they are in a separate plot window
  5. software opens this many plot windows, preferrably arranged in a reasonable grid (but separate windows, not subplots of the same figure). – can’t we track all Q positions in one plot? Or better DeltaQ/Q? or maybe 2-3 plots max if overcrowded.
  6. software plots the position vs. index of each peak in its window as the stream comes down. Since we don't know the correct range, have the plots autoscale. This may look bad if the data are coming quickly, but let's try it. YES.
  7. at the end of the scan, save the QOI data in an easily replottable form (xy column format?) Same for peak intensity and peak width ?

UC1A:

  1. scientist wants to replay a previous scan
  2. scientist types replay_scan and gives a scan id and all the plot windows open up from that scan, including the QOI scans. This doesn't compete with live data, so would have to be working in a separate process.

UC1B

  1. scientist just wants to replot one or a few of the QOI scans.
  2. scientist double-clicks the qoi.xyz file that she is interested in and these plots open up. This doesn't compete with data acquisition or live plotting.

UC2: as UC1 but it plots the peak intensity

UC3 as UC1 but it is the PDF data. Let's try not doing it on Nyquist grid but doing it on a 0.01 gird. for the ranges, we could try harvesting all the peaks in the 0-6 A range and again, plot in the same figure if they are within 0.25A of each other

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/xpdAcq/mission-control/issues/27#issuecomment-392737549, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ALyE3eFBawV8DAqaafuZSkFNI1t4_En8ks5t3SrigaJpZM4PK-Fs.

CJ-Wright commented 6 years ago

Rerendering the email reply in markdown

Here are some more precise UCs, which should go from easier (I hope) but still useful on down.

UC1:

  1. user starts a scan
  2. software automatically finds peak positions of all the peaks in I(Q) in a range up to 6 A-1 using a simple local maximum finder, with data on a finish grid.

that would be really awesome. A remark: people usually “know” their diffraction pattern by the d-spacing values.

  1. software determines how many plot windows to open
  2. If peaks are within 0.25 A-1 of each other they are in the same plot window, if they are further apart than this, they are in a separate plot window
  3. software opens this many plot windows, preferrably arranged in a reasonable grid (but separate windows, not subplots of the same figure).

can’t we track all Q positions in one plot? Or better DeltaQ/Q? or maybe 2-3 plots max if overcrowded.

  1. software plots the position vs. index of each peak in its window as the stream comes down. Since we don't know the correct range, have the plots autoscale.

This may look bad if the data are coming quickly, but let's try it. YES.

  1. at the end of the scan, save the QOI data in an easily replottable form (xy column format?)

Same for peak intensity and peak width ?

dooryhee commented 6 years ago

From: Christopher J. Wright notifications@github.com Sent: Wednesday, July 25, 2018 12:14 PM To: xpdAcq/mission-control mission-control@noreply.github.com Cc: Dooryhee, Eric edooryhee@bnl.gov; Mention mention@noreply.github.com Subject: Re: [xpdAcq/mission-control] ENH: plot QOIs (#27)

Rerendering the email reply in markdown

Here are some more precise UCs, which should go from easier (I hope) but still useful on down.

UC1:

  1. user starts a scan
  2. software automatically finds peak positions of all the peaks in I(Q) in a range up to 6 A-1 using a simple local maximum finder, with data on a finish grid.

that would be really awesome. A remark: people usually “know” their diffraction pattern by the d-spacing values.

  1. software determines how many plot windows to open
  2. If peaks are within 0.25 A-1 of each other they are in the same plot window, if they are further apart than this, they are in a separate plot window
  3. software opens this many plot windows, preferrably arranged in a reasonable grid (but separate windows, not subplots of the same figure).

can’t we track all Q positions in one plot? Or better DeltaQ/Q? or maybe 2-3 plots max if overcrowded.

  1. software plots the position vs. index of each peak in its window as the stream comes down. Since we don't know the correct range, have the plots autoscale.

This may look bad if the data are coming quickly, but let's try it. YES.

  1. at the end of the scan, save the QOI data in an easily replottable form (xy column format?)

Same for peak intensity and peak width ? Yes, if possible at all…

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/xpdAcq/mission-control/issues/27#issuecomment-407810883, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ALyE3UXqGLKL-siBC_8lYDUH3lKBXL7vks5uKJlPgaJpZM4PK-Fs.

st3107 commented 2 years ago

I will move this issue to the current milestone. I feel that it is better to provide some easy-to-code infrastructure, e. g. a python class or just a script leaving someplace (for example, a method that takes in data and returns results) for beamline scientists to code. Then, they run the script and a server will start running and visualize the stuff they would like to see. This is a more long-term solution and offers people more freedom to see what they want.