search

RNA-FRETools / MASH-FRET

MATLAB package to analyze single-molecule FRET data
https://rna-fretools.github.io/MASH-FRET/
GNU General Public License v3.0
8 stars 2 forks source link

New Video Project not calculating traces #114

Closed snguyen49 closed 9 months ago

snguyen49 commented 10 months ago

Hi,

I tried to import a video created from the simulation to try out the video project function. After inputting all of the spot coordinates and mapping the reference coordinates, I tried to calculate the traces. The system then showed an error and was not able to calculate the traces.

Screenshot 2024-01-10 142012 This is the error code.

image These were the parameters.

Many Thanks, Sydney Nguyen

mca-sh commented 10 months ago

Hello Sydney and thank you for reporting this bug. Could you please save the project to a .mash file and send it to me? I will try to reproduce the error and have a look at what went wrong.

Best, Mélodie

snguyen49 commented 10 months ago

Hi Mélodie,

I have attached the .mash file in this email.

Many Thanks, Sydney Nguyen

From: Mélodie Hadzic @.> Sent: Thursday, January 11, 2024 4:13 AM To: RNA-FRETools/MASH-FRET @.> Cc: Sydney Nguyen @.>; Author @.> Subject: Re: [RNA-FRETools/MASH-FRET] New Video Project not calculating traces (Issue #114)

Hello Sydney and thank you for reporting this bug. Could you please save the project to a .mash file and send it to me? I will try to reproduce the error and have a look at what went wrong.

Best, Mélodie

— Reply to this email directly, view it on GitHubhttps://github.com/RNA-FRETools/MASH-FRET/issues/114#issuecomment-1886684076, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BCCRYBC4RUEEXWFJKPQ4XG3YN6ULHAVCNFSM6AAAAABBVJ6YVSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBWGY4DIMBXGY. You are receiving this because you authored the thread.Message ID: @.***>

CAUTION: This email was sent from someone outside of the university. Do not click links or open attachments unless you recognize the sender and know the content is safe.

mca-sh commented 10 months ago

Oups Sorry, but attaching files to emails does not work here since we are mediated via Github. Could you please come to Github and send it from the issue you've posted (https://github.com/RNA-FRETools/MASH-FRET/issues/114)? Thank you.

snguyen49 commented 10 months ago

calculatingtraceserror.zip

snguyen49 commented 10 months ago

My bad. I am not good with this computer stuff.

From: Mélodie Hadzic @.> Sent: Thursday, January 11, 2024 1:44 PM To: RNA-FRETools/MASH-FRET @.> Cc: Sydney Nguyen @.>; Author @.> Subject: Re: [RNA-FRETools/MASH-FRET] New Video Project not calculating traces (Issue #114)

Oups Sorry, but attaching files to emails does not work here since we are mediated via Github. Could you please come to Github and send it from the issue you've posted (#114https://github.com/RNA-FRETools/MASH-FRET/issues/114)? Thank you.

— Reply to this email directly, view it on GitHubhttps://github.com/RNA-FRETools/MASH-FRET/issues/114#issuecomment-1887755315, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BCCRYBEES7FTMHZ4I4B6XLTYOAXILAVCNFSM6AAAAABBVJ6YVSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBXG42TKMZRGU. You are receiving this because you authored the thread.Message ID: @.***>

CAUTION: This email was sent from someone outside of the university. Do not click links or open attachments unless you recognize the sender and know the content is safe.

mca-sh commented 10 months ago

No worries that's a common mistake. Actually I would need also the associated video file (.sira I think). This might be too large for Github though..

mca-sh commented 10 months ago

Otherwise I will have to ask you many questions since I cant reproduce the error by doing the same steps as you did.

snguyen49 commented 10 months ago

I see. Is there any other way I can use to send it to you?

From: Mélodie Hadzic @.> Sent: Thursday, January 11, 2024 1:52 PM To: RNA-FRETools/MASH-FRET @.> Cc: Sydney Nguyen @.>; Author @.> Subject: Re: [RNA-FRETools/MASH-FRET] New Video Project not calculating traces (Issue #114)

Otherwise I will have to ask you many questions since I cant reproduce the error by doing the same steps as you did.

— Reply to this email directly, view it on GitHubhttps://github.com/RNA-FRETools/MASH-FRET/issues/114#issuecomment-1887767437, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BCCRYBAJ57VMOSKRT6HTVP3YOAYIDAVCNFSM6AAAAABBVJ6YVSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBXG43DONBTG4. You are receiving this because you authored the thread.Message ID: @.***>

CAUTION: This email was sent from someone outside of the university. Do not click links or open attachments unless you recognize the sender and know the content is safe.

mca-sh commented 10 months ago

Can you try to send it to my email address: melodiehadzic@protonmail.com Or, if it is still too big, you can try via this swiss website: https://www.grosfichiers.com/en/

snguyen49 commented 10 months ago

I just sent the file via the swiss website.

Thank you, Sydney

From: Mélodie Hadzic @.> Sent: Thursday, January 11, 2024 2:00 PM To: RNA-FRETools/MASH-FRET @.> Cc: Sydney Nguyen @.>; Author @.> Subject: Re: [RNA-FRETools/MASH-FRET] New Video Project not calculating traces (Issue #114)

Can you try to send it to my email address: @.**@.> Or, if it is still too big, you can try via this swiss website: https://www.grosfichiers.com/en/

— Reply to this email directly, view it on GitHubhttps://github.com/RNA-FRETools/MASH-FRET/issues/114#issuecomment-1887779939, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BCCRYBCIFY4WVKFTQT6NJXDYOAZFFAVCNFSM6AAAAABBVJ6YVSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBXG43TSOJTHE. You are receiving this because you authored the thread.Message ID: @.***>

CAUTION: This email was sent from someone outside of the university. Do not click links or open attachments unless you recognize the sender and know the content is safe.

mca-sh commented 10 months ago

Great thank you, I will have a look On Thursday, January 11th, 2024 at 20:07, snguyen49 @.***> wrote:

I just sent the file via the swiss website.

Thank you, Sydney

From: Mélodie Hadzic @.> Sent: Thursday, January 11, 2024 2:00 PM To: RNA-FRETools/MASH-FRET @.> Cc: Sydney Nguyen @.>; Author @.> Subject: Re: [RNA-FRETools/MASH-FRET] New Video Project not calculating traces (Issue #114)

Can you try to send it to my email address: @.**@.> Or, if it is still too big, you can try via this swiss website: https://www.grosfichiers.com/en/

— Reply to this email directly, view it on GitHubhttps://github.com/RNA-FRETools/MASH-FRET/issues/114#issuecomment-1887779939, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BCCRYBCIFY4WVKFTQT6NJXDYOAZFFAVCNFSM6AAAAABBVJ6YVSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBXG43TSOJTHE. You are receiving this because you authored the thread.Message ID: @.***>

CAUTION: This email was sent from someone outside of the university. Do not click links or open attachments unless you recognize the sender and know the content is safe.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were assigned.Message ID: @.***>

mca-sh commented 10 months ago

Hi Sidney, I had a look and can't find what went wrong. The video file you sent me might not the exact one with which you've created the video-based project since the average image changed drastically when I recalculated. Anyway, it does not matter much. You can try to reset the defaults interface parameters by:

  1. closing MASH
  2. deleting the file MASH-FRET/default_param.ini
  3. reopen MASH and re-create your video project

One suspicion I have is that, may be, the number of lasers was not well defined in the experiment settings. It should one laser in your case.

Please tell me if you still have this bug arising despite the procedure.

Best, Mélodie

mca-sh commented 10 months ago

I found a bug that might be related. Can you try with this version please: https://codeload.github.com/RNA-FRETools/MASH-FRET/zip/refs/heads/correct-doublon-coord

snguyen49 commented 10 months ago

Thank you. I will try it out and get back to you.

Sydney Nguyen

From: Mélodie Hadzic @.> Sent: Thursday, January 11, 2024 3:15 PM To: RNA-FRETools/MASH-FRET @.> Cc: Sydney Nguyen @.>; Author @.> Subject: Re: [RNA-FRETools/MASH-FRET] New Video Project not calculating traces (Issue #114)

I found a bug that might be related. Can you try with this version please: https://codeload.github.com/RNA-FRETools/MASH-FRET/zip/refs/heads/correct-doublon-coord

— Reply to this email directly, view it on GitHubhttps://github.com/RNA-FRETools/MASH-FRET/issues/114#issuecomment-1887895942, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BCCRYBDNH57B4BJB6HPLJITYOBB6ZAVCNFSM6AAAAABBVJ6YVSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBXHA4TKOJUGI. You are receiving this because you authored the thread.Message ID: @.***>

CAUTION: This email was sent from someone outside of the university. Do not click links or open attachments unless you recognize the sender and know the content is safe.

snguyen49 commented 10 months ago

Hi Melodie,

So far, it seems to be working fine. I will let you know if there is anything else that pops up pertaining to this issue.

Thank you so much for your help! Sydney Nguyen

Get Outlook for iOShttps://aka.ms/o0ukef

From: Mélodie Hadzic @.> Sent: Thursday, January 11, 2024 3:15:40 PM To: RNA-FRETools/MASH-FRET @.> Cc: Sydney Nguyen @.>; Author @.> Subject: Re: [RNA-FRETools/MASH-FRET] New Video Project not calculating traces (Issue #114)

I found a bug that might be related. Can you try with this version please: https://codeload.github.com/RNA-FRETools/MASH-FRET/zip/refs/heads/correct-doublon-coord

— Reply to this email directly, view it on GitHubhttps://github.com/RNA-FRETools/MASH-FRET/issues/114#issuecomment-1887895942, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BCCRYBDNH57B4BJB6HPLJITYOBB6ZAVCNFSM6AAAAABBVJ6YVSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBXHA4TKOJUGI. You are receiving this because you authored the thread.Message ID: @.***>

CAUTION: This email was sent from someone outside of the university. Do not click links or open attachments unless you recognize the sender and know the content is safe.

snguyen49 commented 10 months ago

Hey Melodie,

The calculated traces found using the import video project do not seem to find the same results as the original simulation project from which the video was obtained. [cid:0a7bdb89-3f38-4e4a-b313-c9ea4661d5d9] Here are the traces given from the simulation. I took the video used for this simulation and imported it into the video project which gave me the image below: [cid:c5978770-12a7-4718-a9fe-77bb59ca5806] This result does not look the same as the original simulated results, so I am confused if I had input any parameters wrong. I apologize for the constant questions, but since it pertains to my project it is rather urgent for me to get the data processing procedure ready as soon as possible. You might also get messages from my mentor, Gauri Tripathi, since she is also learning to use your program.

Many Thanks, Sydney Nguyen

From: Mélodie Hadzic @.> Sent: Thursday, January 11, 2024 3:15 PM To: RNA-FRETools/MASH-FRET @.> Cc: Sydney Nguyen @.>; Author @.> Subject: Re: [RNA-FRETools/MASH-FRET] New Video Project not calculating traces (Issue #114)

I found a bug that might be related. Can you try with this version please: https://codeload.github.com/RNA-FRETools/MASH-FRET/zip/refs/heads/correct-doublon-coord

— Reply to this email directly, view it on GitHubhttps://github.com/RNA-FRETools/MASH-FRET/issues/114#issuecomment-1887895942, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BCCRYBDNH57B4BJB6HPLJITYOBB6ZAVCNFSM6AAAAABBVJ6YVSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBXHA4TKOJUGI. You are receiving this because you authored the thread.Message ID: @.***>

CAUTION: This email was sent from someone outside of the university. Do not click links or open attachments unless you recognize the sender and know the content is safe.

mca-sh commented 9 months ago

Hi Sydney and thank you for your feedback,

I don't mind your questions, they also help me develop the software. I understand you are in a hurry to analyze your data and I 'll try my best.

I do not see the images you included in your message, probably because they are attached to your email. Could you please use Github directly (https://github.com/RNA-FRETools/MASH-FRET/issues/114)? Thank you Mélodie

mca-sh commented 9 months ago

I try to reproduce the problem that you might have by:

  1. simulating and exporting a .sira video file,
  2. creating a video project using this video file, and
  3. creating trajectories.

The difference you are observing between your simulated and video-extracted trajectories might come from the background and cross-talk corrections that are not configured for your data. These are an important part of trace processing and must be done every time you process new data.

Background correction

For the background correction, check that the correction is applied for each of the trajectory (module Trace processing > panel Background correction). I recommend to use the method <N median values> with option dynamic activated. It is more time consuming but more precise. Be sure the correction is applied (option Apply activated) for each trajectory in the list data, and use the same setting for each molecule by pressing All. If you want more information about background correction, please visit this section of the online doc: https://rna-fretools.github.io/MASH-FRET/trace-processing/components/panel-background-correction.html

Cross-talk correction

First, be sure of the cross-talk (bleedthrough and direct excitation) coefficients you used: they are specific to your experimental setup. In your case, they were defined in the simulation. Look for the donor bleedthrough coefficient set in the simulation (module Simulation > panel Molecules > field BtD) and report the exact same value to your trajectory processing (module Trace processing > panel Cross-talks > set emitter to donor > set bt into to acceptor > field bt). The cross-talk coefficients are common to all molecules. More information about cross-talk correction: https://rna-fretools.github.io/MASH-FRET/trace-processing/components/panel-crosstalk-corrections.html

I hope this fixes the problem, otherwise, I will need your images.

Mélodie

snguyen49 commented 9 months ago

Thank you, Melodie! I will try it out as soon as possible and get back to you with the results.

From: Mélodie Hadzic @.> Sent: Monday, January 15, 2024 6:16 AM To: RNA-FRETools/MASH-FRET @.> Cc: Sydney Nguyen @.>; Author @.> Subject: Re: [RNA-FRETools/MASH-FRET] New Video Project not calculating traces (Issue #114)

I try to reproduce the problem that you might have by:

  1. simulating and exporting a .sira video file,
  2. creating a video project using this video file, and
  3. creating trajectories.

The difference you are observing between your simulated and video-extracted trajectories might come from the background and cross-talk corrections that are not configured for your data. These are an important part of trace processing and must be done every time you process new data.

Background correction

For the background correction, check that the correction is applied for each of the trajectory (module Trace processing > panel Background correction). I recommend to use the method with option dynamic activated. It is more time consuming but more precise. Be sure the correction is applied (option Apply activated) for each trajectory in the list data, and use the same setting for each molecule by pressing All. If you want more information about background correction, please visit this section of the online doc: https://rna-fretools.github.io/MASH-FRET/trace-processing/components/panel-background-correction.html

Cross-talk correction

First, be sure of the cross-talk (bleedthrough and direct excitation) coefficients you used: they are specific to your experimental setup. In your case, they were defined in the simulation. Look for the donor bleedthrough coefficient set in the simulation (module Simulation > panel Molecules > field BtD) and report the exact same value to your trajectory processing (module Trace processing > panel Cross-talks > set emitter to donor > set bt into to acceptor > field bt). The cross-talk coefficients are common to all molecules.

I hope this fixes the problem, otherwise, I will need your images.

Mélodie

— Reply to this email directly, view it on GitHubhttps://github.com/RNA-FRETools/MASH-FRET/issues/114#issuecomment-1891958018, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BCCRYBDCEVQEVUOBEAMJMWLYOUFZNAVCNFSM6AAAAABBVJ6YVSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOJRHE2TQMBRHA. You are receiving this because you authored the thread.Message ID: @.***>

CAUTION: This email was sent from someone outside of the university. Do not click links or open attachments unless you recognize the sender and know the content is safe.

snguyen49 commented 9 months ago

Hey Melodie, I am currently using the most recently updated file you sent me but it is now reproducing the same error from the original issue that I had presented in this post. I am not sure what I am doing wrong, but here is a screenshot of the error code. image

mca-sh commented 9 months ago

Hi Sydney,

I see, so the problem was elsewhere..

Can you please detail the steps you went through? Are you working with the same project and video file you sent me?

snguyen49 commented 9 months ago

Hi Melodie,

I basically simulated a new system with the default parameters with 1000 frames and 100 molecules. Once I had acquired the simulation, I exported the corresponding avi and sira videos and coordinates. After that, I started a new video project and imported the newly simulated video. I used the spotfinder with Houghpeaks and mapped the reference coordinates. I also the spot coordinates were transformed and calculated for the transformation so I had all three green check marks as advised on the website.

mca-sh commented 9 months ago

Ok, you did exactly what we are supposed to do.

I have a question about the experiment settings: when you created your video-based project, how many alternating lasers did you set in the tab "lasers" (like below)?

image

snguyen49 commented 9 months ago

I set it as one. Should I have set it as 2?

mca-sh commented 9 months ago

No, you're right everything is correct. I don't understand where the problem is coming from, because when I repeat the procedure, everything works fine.

I have two questions:

  1. Did you use the avi or the sira file in the video-based project?
  2. May I ask you to press "Transform" and send me a screen shot of Video Processing after coordinates are transformed? May be no coordinates is left after transformation.
snguyen49 commented 9 months ago

I am currently using the .sira file instead of .avi. image

mca-sh commented 9 months ago

Ok thank you. So apparently there are coordinates left. And now, when you press "calculate traces" (after pressing transform) you get the exact same error you sent me?

snguyen49 commented 9 months ago

Yes. I tried transforming and calculating again, but it is generating the same error.

How do I go about deleting this file?

mca-sh commented 9 months ago

Ok, can you expand the "Intensity integration" panel so I can have a look at the parameters?

What may have work the last time is to delete the default_param.ini file. May be you can repeat this and see if it fixes the problem?

snguyen49 commented 9 months ago

image

mca-sh commented 9 months ago

You can try to reset the defaults interface parameters by:

1. closing MASH

2. deleting the file MASH-FRET/default_param.ini

3. reopen MASH and re-create your video project
mca-sh commented 9 months ago

So you did not delete this file the last time? The intensity integration parameters are fine.. I can't see where the bug is coming from..

snguyen49 commented 9 months ago

So you did not delete this file the last time? The intensity integration parameters are fine.. I can't see where the bug is coming from..

I am currently trying it out right now. Will see how it goes.

mca-sh commented 9 months ago

I will analyze your logs then, that's my last chance to udnerstand what is going on. Could you please give the file MASH-FRET/log/[XXXX-XX-XX].log with [XXXX-XX-XX] today's date?

snguyen49 commented 9 months ago

After deleting the default parameters file, it is now working.

mca-sh commented 9 months ago

Mmm.. I will have a look at what parameter can make such mess.

And 5 days ago, did you delete this file? Because this might have been why the problem was solved.

snguyen49 commented 9 months ago

I do not believe I had deleted the file 5 days ago.

mca-sh commented 9 months ago

Ok. Next time you get this error, could you send me the default_param.ini file so I can investigate where the conflict come from? Thanks!

snguyen49 commented 9 months ago

Unfortunately, I am still not acquiring the same trace results in my video project as found from the simulation. I had set the bleedthrough to 0.07 for the donor channel as given in the simulation. I am not sure as to what I may be doing wrong. image These are the traces from the video project. image These are the traces from the simulation. Thank you for the help. Sydney Nguyen

snguyen49 commented 9 months ago

Hey Melodie,

I have attached the original default parameters file that I deleted previously.

Many Thanks, Sydney Nguyen

From: Mélodie Hadzic @.> Sent: Tuesday, January 16, 2024 2:31 PM To: RNA-FRETools/MASH-FRET @.> Cc: Sydney Nguyen @.>; Author @.> Subject: Re: [RNA-FRETools/MASH-FRET] New Video Project not calculating traces (Issue #114)

Ok. Next time you get this error, could you send me the default_param.ini file so I can investigate where the conflict come from? Thanks!

— Reply to this email directly, view it on GitHubhttps://github.com/RNA-FRETools/MASH-FRET/issues/114#issuecomment-1894382933, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BCCRYBCVL5VFLCR2CIHOWP3YO3IQ7AVCNFSM6AAAAABBVJ6YVSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOJUGM4DEOJTGM. You are receiving this because you authored the thread.Message ID: @.***>

CAUTION: This email was sent from someone outside of the university. Do not click links or open attachments unless you recognize the sender and know the content is safe.

mca-sh commented 9 months ago

Thank you for the images. I can see two things:

  1. It seems from your screenshot of video processing, that too many dark spots were targetted (no molecule is present). and this is confirmed by your screenshot of trace processing where the integration zones (red and yellow squares) are empty. If you navigate through the molecule list, you will eventually fall on one real molecule. To prevent this "overfinding" of spots in the video, you need to adjust the parameters of the Houghpeaks function, especially the first one that is the low intensity threshold of a bright spot to be detected. To know what value to use, you need to check the color bar in the average image plot and select the value corresponding to the color of a single molecule in the average image.
  2. The second thing is that the background is not corrected, at least for the red signal (acceptor signal). You need to browse the data pop up and set the background parameters for each data in the list as they are data-specific. Please have a look at my previous message of yesterday that explains everything about background settings (https://github.com/RNA-FRETools/MASH-FRET/issues/114#issuecomment-1891958018).

Let's see if we obtain the same traces then.

mca-sh commented 9 months ago

Hey Melodie, I have attached the original default parameters file that I deleted previously. Many Thanks, Sydney Nguyen

I can't access it, you need to send it via Github sorry.

snguyen49 commented 9 months ago

default_param.zip Sorry about that. Here you go. And thank you, I will go over your tips and let you know if there are any other run-ins.

mca-sh commented 9 months ago

Great thank you!

snguyen49 commented 9 months ago

I DID IT! Thanks Melodie. T^T

Edit: I cannot seem to input the direct excitation coefficient in the trace processing tab under background correction for some reason. The option to select a laser only shows none. image I also got overexcited and thought I had gotten the same results but there is still a discrepancy with the video project results. I have done the background corrections as advised, but maybe I did them wrong. image Video project image Simulation

I apologize. I am bad with computers. T-T

Edit 2: Nevermind, I figured it out. Again, apologies and many thanks.

mca-sh commented 9 months ago

Hi Sydney, Great that it worked out :) Don't apology, your concerns are all fair.

Just in case you are still wondering: 1) to correct for acceptor direct excitation, you need a signal of the acceptor upon "red" laser excitation, and unfortunately, simulation is available only for one laser.. This is why the correction is not available in trace processing. 2) the intensity trajectories in the simulation tab are not background- and crosstalk-corrected, this is why you see such discrepancy.

I checked the content of the default_prm.ini file but nothing strange was in there.. I really can't figure out what happened.

I will close the issue now (which is very long ^^). You can open a new one anytime.

Best, Mélodie