PEP: new measurement types

[smXplorer]

New measurement types

The following measurement types should be supported in the next Photon-HDF5 version:

1. smFRET (TCSPC) Formally, it is possible to study FRET using pulsed excitation by looking at the donor lifetime only. An acceptor molecule is present in these measurements, and its signal can be recorded, but is not indispensable. The suggestion is to introduce a "smFRET (TCSPC)" measurement type for these cases. In the case where only a donor channel is used for FRET measurements, there would be no "detectors" field in the file (since there is only one channel per spot) and no spectral channel definition (it is implicit that all channels are donor channels in this measurement type). In the case where both donor and acceptor channels are used, the structure would be similar to the smFRET-nsALEX measurement type, with the exception that there would be no definition of a second laser excitation, and no alternation period definitions. The distinction between these two cases would be easy to perform based on the presence or absence of a detectors field, although setup information on the single spectral channel used would be of interest.
2. Polarization Anisotropy Measurements involving linearly polarized excitation and polarization-sensitive detection are used to study the motional freedom of dyes (generally on the ns time scale, but possibly on the us to ms time scale) or the macromolecule they are attached to. The suggestion is to introduce a measurement type (possibly two: one for CW excitation, one for pulsed excitation) for measurement involving a single spectral channel and two polarization channels. Information on the spectral channel could be useful and might require to introduce a single spectral channel for that purpose.
3. smFRET with polarization Measurements with donor and acceptor signals split into s and p polarization are common but currently, it is not clear whether they are supported within the smFRET or smFRET-usALEX, smFRET-nsALEX or not. Formally, it appears possible to provide polarization channel information in any of those measurement types. This needs to be clarified. New measurement types exclusively reserved for these situations might be needed.

[tritemio]

Yes, all these 3 cases need a new measurement_type definition. We still use the same "logic" and field names for spectral, polarization and split channels. But we need to decide what is required for each of these new measurement_types.

In the next "release" Photon-HDF5 0.5 we can both include these new measurement types and the addition proposed in PEP #33.

As for the other proposals, comments and inputs from anybody interested are welcome!

[lampo808]

Hi all. I am currently taking measurements monitoring the aggregation process of a protein labeled with a single fluorophore via FCS and photon count histogram analysis, measuring the fluorescence photons with two detectors after a non-polarizing beam splitter. I think a new measurement type should also be included for this kind of experiments, where the fluorescence signal can be splitted among an arbitrary number of detectors with non-polarizing or polarizing beam splitters and dichroic mirrors, but in principle no FRET is involved. The excitation could be CW or pulsed.

Maybe polarization anisotropy experiments could be defined as a special case of this kind of measurements.

[tritemio]

@lampo808, thanks for joining the conversation! And good input!

Your post gives me the opportunity to clarify some of the design philosophy behind Photon-HDF5. We had a discussion with @smXplorer a few days ago and this point is probably not very clear. This is going to be longish.

New measurement type vs "variants"

The way Photon-HDF5 works, at least at the moment, is that a measurement type univocally specifies the measurement, i.e. there are no "variants". This has a strong advantage for people writing programs/scripts to analyze the data. Data analysis is very specific for each type of measurements (even though there may be similar algorithms). If I write a script to do polarization anisotropy analysis, I would read the measurement type and if it matches polarization_anysotropy (hypothetical name) I know all the fields that I need are present in the file. There are no if or special cases. Meaning that if you can analyze one polarization_anysotropy file you can analyze them all. On the contrary, if we add variants, I need to check which fields are present and which are not, and adjust the analysis accordingly. What can likely happen is that a script to analyze the data that works with one file, will work with another that has the same measurement type!

This approach has also a disadvantage: it makes the format design more laborious requiring new measurement types for each "variant" of a measurement. For example, we have separate measurement types for 2-colors and 3-colors smFRET. This issue can be mitigated if defining new measurement types is very easy.

In practice, "similar" measurement types will have similar field names. For example, 2-color smFRET have:

• spectral_ch1
• spectral_ch2

and 3-color smFRET has:

• spectral_ch1
• spectral_ch2
• spectral_ch3

So, defining a new measurement type should be just a matter of adding (or removing!) some relevant fields from another similar measurement type.

The bottom line is: we need to decide a set of measurement types to add to Photon-HDF5, using a similar structure for measurement types that are similar.

As a counter argument, an analysis program for a "class" of measurement (let's say all smFRET measurement, with one or two lasers, pulsed or cw) would need to handle many measurement types. I don't think this would cause additional complexity in the program. If anything it would make exact identification of the measurement immediate. The program can be structured to apply the same algorithm to different measurement types. For example the program can allow burst search or CCF on any measurement type, but it would allow FRET analysis or some specialized 2-color CCF analysis only for smFRET measurement types.

These are my thoughts. If anybody strongly opposes to it, i.e. if you think that we should definitely allow variants within the same measurement type, I'd like to hear your argument :).

[tritemio]

Here I'm starting a draft for the new measurement types. I'll keep editing/fixing this comment iteratively as the discussion goes on.

"smFRET-TCSPC"

2-color smFRET with single pulsed laser excitation and TCSPC acquisition.

measurement_specs/:

• detectors_specs/spectral_ch1
• detectors_specs/spectral_ch2
• laser_repetition_rate

Relevant fields in /setup/:

• num_spectral_ch = 2
• lifetime = True
• excitation_cw = False

"smFRET_2polarization_tcspc"

Single pulsed excitation, 2 spectral detection channels, each split in 2 polarization channels (4 detectors). With TCSPC acquisition.

measurement_specs/:

• detectors_specs/spectral_ch1
• detectors_specs/spectral_ch2
• detectors_specs/polarization_ch1
• detectors_specs/polarization_ch2
• laser_repetition_rate

Relevant fields in /setup/:

• num_spectral_ch = 2
• num_polarization_ch = 2
• num_split_ch = 1
• lifetime = True
• excitation_cw = False

"2polarization_tcspc"

Single pulsed excitation with 2 detectors for 2 polarizations and TCSPC acquisition.

measurement_specs/:

• detectors_specs/polarization_ch1
• detectors_specs/polarization_ch2
• laser_repetition_rate

Relevant fields in /setup/:

• num_spectral_ch = 1
• num_polarization_ch = 2
• num_split_ch = 1
• lifetime = True
• excitation_cw = False

"2split"

Single CW excitation with two beam-spit detectors.

measurement_specs/:

• detectors_specs/split_ch1
• detectors_specs/split_ch2

Relevant fields in /setup/:

• num_spectral_ch = 1
• num_polarization_ch = 1
• num_split_ch = 2
• lifetime = False
• excitation_cw = True

[tritemio]

@lampo808 wrote

I think a new measurement type should also be included for this kind of experiments, where the fluorescence signal can be splitted among an arbitrary number of detectors with non-polarizing or polarizing beam splitters and dichroic mirrors, but in principle no FRET is involved. The excitation could be CW or pulsed.

Based on my argument we should define a measurement type for each of these combinations. It is a good point that no FRET needs to be involved even when we have several spectral channels (i.e. dichroics).

Thinking aloud, if we limit ourself to 2 split, polarization or spectral channels what would be the most reasonable types?

Trying to build a list:

4 detectors configurations

• 2spectral_2polarization this may also cover @smXplorer proposal of "smFRET with polarization"
• 2polarization_2split
• 2spectral_2split

2 detectors configurations

• 2polarization
• 2split this is what I call "FCS_2split" in the previous comment
• 2spectral, we don't need this. It is already defined as smFRET. I think it is fair to use it even is there is no FRET in the sample.

Food for thoughts.

[lampo808]

After a rapid look through some literature, I think we can limit ourselves to the following lists as proposed by @tritemio:

4 detectors:

• 2spectral_2polarization
• 2spectral_2split

2 detectors:

• 2polarization. can also cover the case for polarization anisotropy. Indeed I haven't found anyone splitting the signal again after the polarizing beam splitter
• 2split

I agree that 2spectral-kind of measurements can be treated as smFRET.

Oh, and maybe we can drop FCS from the name, if we want to use the same measurement types also in other cases (e.g. 2polarization both for polarization-sensitive FCS and polarization anisotropy measurements).

[tritemio]

@lampo808 thanks! What about TCSPC? Should we define a TCSCP version of each of those measurement types? Which of these configurations are most commonly used with TCSPC?

[tritemio]

Is there anybody doing polarization measurement without TCSPC? It seems natural that if you do polarization anisotropy you want to look at rotational diffusion that requires sub-ns resolution, therefore TCSPC.

If this is the case we can define all measurement types having 2 polarization channels as also having TCSPC.

[lampo808]

I have looked through some literature, and it seem to me that all the people doing polarization measurements use TCSPC.

[smXplorer]

As a matter of fact, we did some polarization anisotropy studies of slowly rotating nanorods in the past (doi: 10.1364/BOE.7.002441) using CW lasers, so I would not exclude CW or us-ALEX polarization studies off the bat.

[tritemio]

@talaurence do you have any inputs on measurement types to be defined?

[tritemio]

@AndersBarth do you want to give your opinion on the new measurement types?

[smXplorer]

I would suggest "2color" rather than "2spectral", which doesn't mean anything unless you add "channel" or "ch" to it (if you want to stick with the name given to the corresponding parameter). Likewise, I don't believe that adding "2" to "split" adds much to the fact that the signal is split into two fractions and as before, it doesn't correspond to anything in the English language. Or add "channel" or "ch" to make it more human-readable.

My suggestion would therefore be to stick to the shorter (and understandable): CW_split CW_2polarization CW_split_2polarization and the same with TCSPC.

Of course, the 3rd type of measurement (and its TCSPC counterpart) leave the ambiguity of whether there is first a split into 2 polarizations, followed by a non-polarizing beamsplitter or the reverse, but if the setup is well designed, this should not matter.

[tritemio]

@smXplorer they sound like good names to me.

[lampo808]

@smXplorer good point on the names.

BTW, I just came across a couple of other papers doing CW fluorescence anisotropy... Discovered a new world.

[tritemio]

We can drop the 2 from the name in all cases. I'm currently implementing the following name scheme:

cw_split
cw_polarization
cw_split_polarization
tcspc_split
tcspc_polarization
tcspc_split_polarization

These are all single-excitation measurements.

[tritemio]

I started a new branch of the documentation for 0.5:

http://photon-hdf5.readthedocs.io/en/0.5.dev/

Documentation for 0.4 is still the default when going to

http://photon-hdf5.readthedocs.io/

[smXplorer]

Another measurement type we will soon need is smFRET-usPAX (2-colors). It could fit into the current us-ALEX measurement type (setting the donor period to be the full period), but the underlying data analysis is different than standard us-ALEX, therefore it is worth distinguishing.

[tritemio]

The combination the most common options of 1 or 2 lasers, 1-2 polarizations, 1-2 split channels, 1-2 spectral channels with and without TCSPC results in a big number of new measurement_type.

I think that would be much easier to implement a "generic" measurement type that will be specified by the fields in /setup and by fields in measurement_specs/detectors_specs.

See the current draft here: http://photon-hdf5.readthedocs.io/en/0.5.dev/whats_new_ver05.html

[lampo808]

I agree, a "generic" measurement type is a good idea, both to take into account the current measurement schemes and for possible new implementations.

[tritemio]

@lampo808, thanks for the feedback! Also @smXplorer agreed on the idea of a generic measurement type. I'll try to finish the specification document and the phconvert implementation with this change ASAP and then I'll ask for review/comments here.

[smXplorer]

I would suggest indicating changes from 0.4 in the 0.5 draft, so that potential reviewer (e.g. me) can focus on the new stuff to provide feedback.

[tritemio]

@smXplorer, please follow the last link I posted and send your comments.

[tritemio]

See http://photon-hdf5.readthedocs.io/en/0.5.dev/whats_new_ver05.html#the-generic-measurement-type for the description of the new "generic" measurement type.

[tritemio]

I moved the description of the "generic" measurement type to a new dedicated page:

http://photon-hdf5.readthedocs.io/en/0.5.dev/generic.html

[tritemio]

• [x] Add "generic" measurement type to the documentation
• [x] Add support for "generic" measurement type in phconvert