CIRDLES / Squid

Squid3 is being developed by the Cyber Infrastructure Research and Development Lab for the Earth Sciences (CIRDLES.org) at the College of Charleston, Charleston, SC and Geoscience Australia as a re-implementation in Java of Ken Ludwig's Squid 2.5. - please contribute your expertise!
http://cirdles.org/projects/squid/
Apache License 2.0
12 stars 24 forks source link

Ratio mode #431

Closed cwmagee closed 4 years ago

cwmagee commented 4 years ago

Hi all, If we aresquidapoloozaing, can we get ratio mode running?

This involves allowing arbitrary run tables which process as far as the mean ratio calculations, with no geochron calculations performed. Custom expressions still need to be importance and/or editable.

bowring commented 4 years ago

@cwmagee - We can look at this, but you will have to drive the process, which means providing some files and tasks and detailing the processes and then reviewing the results, etc.

NicoleRayner commented 4 years ago

I'm inserting myself here, but here is a xml and a task for that we use when we want to determine our deadtime. Nice and simple, 2 isotopes. Deadtime xml and task.zip

Here is another xml for Boron (3 isotopes), I don't have a task for that one though.
Boron analysis 3P.zip

cwmagee commented 4 years ago

OK, here is a trace element xml file. Trace elements in zircon: 30ish peaks. 200010_GA6440_20021115.41.zip I don't have a viable task for this at home, but I can upload one tomorrow from work.

bowring commented 4 years ago

We will need more than a task! We need a detailed description of the workflow and anticipated results, as we have no idea what this means.

cwmagee commented 4 years ago

OK, the point of ratio mode is to calculate ratios of arbitrary peak positions. This is basically the same as the beginning part of geochron mode- I think similar to the four step document Simon wrote years ago that might still be floating around this space (I just had a look and can't find it). So the ratio mode squid needs to identify peaks and backgrounds, subtract background(s) from peaks, calculate ratios from background subtracted peaks, and then either interpolate to a fixed time, a mid time, or else just use a weighted mean. Users may choose to write additional tasks beyond that, but the main point of ratio mode is calculating and presenting ratios for any old peaks you wish to analyse.

I know there was some talk about specifying ratios vs simply calculating all permutations- the latter could be complicated by the fact that large run tables will have over a thousand possible permutations.

The ability to assign different background peaks to different peaks would be really handy, but we do not have that capability yet today.

Additional user-specified calculations are also sometimes used. For example, Stanford has a U-Th dating task which uses decay constants for the U series isotopes to turn isotopic ratios into activity ratios.

So I guess in the first instance, we need the ability to output background subtracted ratios, so that we can compare them to excel squid books.

NicoleRayner commented 4 years ago

Probably another thing it would be good to have is the ability to define a RM for these types of squid projects (using the same Manage spots and RM tools). Then the question becomes what to do with the model for that RM. Have the ability to make a fully customized model where the parameters are defined by the user? Or create custom constants in the expression manager?

bowring commented 4 years ago

@cwmagee - your "200010....zip" file loads with this info: Data Source File provides 32 Species: 16.01259, 19.02066, 30.00555, 31.01105, 35.03857, 44.00253, 44.01953, 48.99034, 88.96535, 97.96383, 134.01855, 139.07346, 139.62346, 140.07520, 141.07959, 146.07795, 147.07886, 153.05778, 157.04155, 159.03015, 163.01383, 165.00892, 166.00893, 167.00781, 169.00637, 171.01043, 172.01062, 175.02636, 178.04953, 180.07258, 232.19093, 238.21044

Do you have a task identifying 32 masses for specifying ratios and the ratios of interest? If not, Squid could auto-generate a set, but what values would we use for the names of the masses - the listed ones, or the "label" field from the prawn file, as for example:

<par name="label" value="16O" />
<par name="amu" value="16.012586" />
<par name="label" value="19F" />
<par name="amu" value="19.020662" />

Then the ratio would be named "16O/19F".

If we auto-selected all ratios by choosing unique pairs, then "32 choose 2 = 496" would include "16O/19F" but not "19F/16O" or vice-versa.

Thoughts?

cwmagee commented 4 years ago

Hi Jim, I don't have a 32 mass task here at home, and my time in at work has mostly been hands on SHRIMP hardware stuff. So I don't have a task on hand, unless it can be copied off the first sheet of a squided workbook (Do @sbodorkos or @NicoleRayner know if that works).

As a more general point, I recall in early versions there was a matrix that allowed the user to select ratios. Is that still possible? If the xml records which isotopes are labeled "reference" in the run table, than ratioing everything against the reference element(s) with the reference as a denominator would be a good place to start.

Otherwise we could calculate all 496 one sided ratios, but then it might be cumbersome for people to find the ratios they want.

Another feature we need (but don't have in SQUID 2) is the ability to assign different background peaks to different mass stations, in tasks with multiple background peaks. I haven't looked at this in years, but I have an emailed task and example sheet from the USGS/Stanford lab for U/Th disequilibrium which may have multiple backgrounds. reuthdisequilibrium.zip

cheers, Chuck

sbodorkos commented 4 years ago

@cwmagee I think it does work... if you Copy the Task week to a new Excel 2003 workbook, and then Save that workbook with a name that matches the 'real' Tasks in your squiduser folder (i.e. preface the filename 'SquidTask_' etc., look at a real Task in your squiduser folder for the naming convention). Then you should be able to place the new workbook in your squiduser folder, open the SQUID 2.50 Task Editor, and press Rebuild Task Catalog. It should then be functional and available for use.

sbodorkos commented 4 years ago

@cwmagee I forgot about multi-backgrounds... at the moment, that really rests with us in terms of defining a conceptual framework and some 'business rules', which we would then inflict on the rest of the community... For example, I think we would need to stipulate that any peak nominated as a 'background' could never itself be used as a numerator or denominator in any of the user-nominated isotopic ratios of interest. (SQUID 2.50 does allow background to be used in ratios, which is nonsensical arithmetically as both numerator and denominator are background-corrected during the calculation. Squid3 prohibits the use of background in ratios.) Then we would need to make sure that everyone interested in multi-backgrounds is OK with this sort of limitation, because if you can background-correct some backgrounds using other backgrounds, the universe collapses pretty quickly.

From there, we would need a way (checkboxes perhaps?) for the user to identify "all the backgrounds" from a list of the Prawn XML mass-stations, and register those selections. On that basis, Squid3 could then run a simple "auto-guess" of which of the remaining peaks would be associated with which background, by choosing the background with the closest mass in each case. However, the results of that process would then need to be presented back to the user to provide the opportunity for manual override if needed, prior to finalisation.

But this sort of enhancement also requires a first-order re-examination of the code, because Ken's implementation (which we replicated) heavily exploits the idea that all peaks have the same unique background. In essence, we would need to reimagine the fundamental arithmetic, where instead of using mass-stations in isolation and invoking generalised background subtraction where needed, we instead consider the mass-stations as a series of couplets, each comprising the mass-station itself and its nominated background.

It's doable, but not straightforward. Because recasting the logic and arithmetic is probably the easy part. It would be up to Jim @bowring to then support that functionality through the graphical user interface in a way that normal users can understand it (bearing in mind that no-one has done this before), and we'd need to work out what that would look like. It would take time.

bowring commented 4 years ago

Good discussion. I still need to solve the basic problem of how we will identify the numerators and denominators in the absence of an imported task! I am repeating my example below for clarity:

Do you have a task identifying 32 masses for specifying ratios and the ratios of interest? If not, Squid could auto-generate a set, but what values would we use for the names of the masses - the listed ones, or the "label" field from the prawn file, as for example:

<par name="label" value="16O" />
<par name="amu" value="16.012586" />
<par name="label" value="19F" />
<par name="amu" value="19.020662" />

Then the ratio would be named "16O/19F".

cwmagee commented 4 years ago

Dear Y'all, Using the label field is a very good idea. For example, for the peak 30Si, we have:

<!-- Single Collector can be "none" "reference" "background" -->
<par value="reference" name="sc_reference"/>

Unfortunately, for this run table the user forgot to specify the background peak as a background in the run table, so making SQUID 3 smart enough to autodetect the backgrounds won't help here. It should say:

<par value="background" name="sc_reference"/>
but it says:
<par value="none" name="sc_reference"/>

Because I was careless building the acquisition table. Still, it should check this tag, since people who make their acquisition tables correctly should be rewarded. As a backup, though, you might want to search the label value for at least three of the consonants in "Background", in order, ignoring the vowels. There is an exceedingly slim chance that this could be an analytical peak ( e.g. if somebody discovered and wants to analyze potassium-bearing rare earth minerals), but most of the time it will probably be some sort of abbreviation for background, if not the word itself. Coding the search is left as an exercise for the undergrads.

Anyway, for background selection and assignment, I envisage a table with dimensions x-y by y, where x is the total number of peaks and y is the total number of background peaks. (31 by 1 in the case of this run table). The number of background peaks can be anything from zero to X-1 (although it would rarely by more than x/2).

So, if you autoassign background peaks based on the criteria above (the reference setting, or the label letter search), you end up with a table. Let's just say for example that you have 4 background peaks and 10 other peaks, and that autofind gives you a 10x4 table. Each background gets a radio butting: not a background, which redraws the table as 11x3. Each peak has a "background" button which redraws the table as a 9x5 with that peak listed as a background. Peak order is preserved from the run table (the order the peaks appear in the xml file)

When the table is first generated, backgrounds are default assigned based on which is numerically closest to each numeric mass as specified in the first spot of the xml.

The user can then change this, move peaks from background to analyte, reassign, and then save the settings to a task so that they are the default when loading any run table with the same set of peaks (same titles in the same order).

Not all peaks need to have a background assigned, in fact, some run tables won't even have a background peak, For example, @NicoleRayner's two peak run table.

All of @sbodorkos issues with ratioing backgrounds and dividing by zero go away if we apply this simple rule: Background peaks are not background subtracted. Then you can ask for ratios such as 204/background to get a feeling for the signal/background for a run, for example. As long as those background peaks are flagged as being background, and therefore NOT having been background subtracted, then everything should be fine.

Note that this does result in being a improvement over the original Ludwig code, but reducing data where multiple backgrounds are necessary in Squid 2.5 is a nightmare, so that's good.

cwmagee commented 4 years ago

Um, do you have to put a and and around any cut and pasted xml to make it appear in this text box?

NicoleRayner commented 4 years ago

Here's a 27 peak task, could we start with this one @bowring and then modify? Is this a helpful starting place for you to try out ratio mode? I don't have a 32 peak one on hand, but I guess I could make one.

SquidTask_Trace 27P ZrSiO2.GSC.zip

bowring commented 4 years ago

yes, @cwmagee,

blah
bowring commented 4 years ago

@NicoleRayner - I will study your example. I was trying to say also that Squid3 could invent a task using rules like you proposed and then user could edit it.

NicoleRayner commented 4 years ago

it won't match up except for a couple of peaks (a stopped clock is right twice a day), but it is a place holder...

bowring commented 4 years ago

thought Canada was on military time? = once a day ;-)

cwmagee commented 4 years ago

Note that not requiring exact data-task matching would be a good thing. Writing a whole new task every time you try a new peak is a pain in the butt.

On 01:36, Sat, 11/07/2020 Jim Bowring notifications@github.com wrote:

thought Canada was on military time? = once a day ;-)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/CIRDLES/Squid/issues/431#issuecomment-656709450, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACXDTBZKOAHYNCVEEVR6YODR24RQNANCNFSM4LAGVWFA .

bowring commented 4 years ago

My first pass plan is to auto generate tasks that can then be edited and saved and forget about squid 2 tasks as they seem of little use.

cwmagee commented 4 years ago

How do you autogenerate tasks if you don't know what the user is trying to accomplish? For example, the U-Th task and finished datasheet I posted above generates activity ratios, not isotopic ratios. Once user tasks generate ratios, there my be user defined equations that then work with those ratios (e.g. by multiplying them by activity rations, as is done above).

However, this may be an example that is a bit more advanced than what we need to deal with at the moment.

cwmagee commented 4 years ago

Attached are xml, run table, task, and squidded workbook files for a K/Ca doubley charged isochron dating. This run table has four background peaks, at least some of which are indicated with different names. If you are teaching SQUID3 to autoidentify background peaks, this is a good starting place.

bowring commented 4 years ago

One step at a time. FIrst step is to modify Squid3 to generate an empty task that can provide the ratios as first requested, which will give us a more concrete process for discussion and modification: "This involves allowing arbitrary run tables which process as far as the mean ratio calculations, with no geochron calculations performed. Custom expressions still need to be importance and/or editable."

Also, no attachment found in last post.

cwmagee commented 4 years ago

Yeah, Github blew up and started showing 400 and 500-something errors and cute cartoon pictures when I tried uploading. IT said or internet was wonky. I will try again in the morning.

On 19:28, Mon, 13/07/2020 Jim Bowring notifications@github.com wrote:

One step at a time. FIrst step is to modify Squid3 to generate an empty task that can provide the ratios as first requested, which will give us a more concrete process for discussion and modification: "This involves allowing arbitrary run tables which process as far as the mean ratio calculations, with no geochron calculations performed. Custom expressions still need to be importance and/or editable."

Also, no attachment found in last post.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/CIRDLES/Squid/issues/431#issuecomment-657405325, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACXDTB5C6YM774C5XLSXD43R3LARZANCNFSM4LAGVWFA .

cwmagee commented 4 years ago

K-Ca data, run table, task, final results attached (I hope) K-Ca.zip

bowring commented 4 years ago

handled in v1.5.5. Please create new issue to advance ratio-mode.