Closed pr4deepr closed 2 years ago
@pr4deepr thanks for providing details!
The freeze issue has to do with the peculiarity of windows w.r.t. forking processes, I have to figure out which module(s) to protect with a if __name__ == '__main__' to solve this, if you know what I'm talking about you can try protecting
mesmerize\analysis\math\drfft_dtw.pywith an
if name == __main__
. I'm currently travelling so I could try in a few days.
I suspect that your other issue ('float' object has no attribute 'size'
) is because some of your peak curves are NaNs for some reason. If you open the peak editor GUI make sure that each peak is flanked by two bases on either side.
The DropNa
node will let you drop NaNs:
http://docs.mesmerizelab.org/en/master/user_guides/flowchart/nodes.html#dropna
I think that if you choose set axis as _pf_peak_curve
and how as any
it should remove the NaNs from the _pf_peak_curve
data. Play around with the settings because I don't remember exactly.
For the installation, yea pandas
, h5py
, numpy
and tensorflow
are in a bit of a compatibility mess at the moment w.r.t. verisons.
mamba
might help reduce the installation time, see:
https://github.com/kushalkolar/MESmerize/issues/53#issuecomment-845247138
Thanks heaps for the feedback.
I may try using module protection as you mentioned..
The dropna node is really useful, but I'll make sure my peaks are flanked by bases.. Yes, you are right there are NaNs..
I'll try the pf_peak_curve tip you mentioned. Hoping that helps.
Will give mamba a go. With the command I mentioned it installs reasonably quickly, but perhaps mamba is the long term solution!!
Happy Travels. Glad you can atleast travel, don't think we can do that anytime soon.. !! 😷
So, I went to mesmerize\analysis\ init.py and added this part
if __name__ == '__main__':
from .math import drfft_dtw
It seemed to fix the issue.
** One of my peaks didn't have a base. Is there an easy way to detect which cell/trace has it? In the console, I Saw this line:
\anaconda3\envs\mesmerize\lib\site-packages\mesmerize\analysis\compute_peak_features.py:106: UserWarning: Peak at curve index: <0> is not flanked by bases on both sides, ignoring
warn(f"Peak at curve index: <{p_ix}> is not flanked by bases on both sides, ignoring")
The dropna node suggestion worked well. **
I'd try protecting the drfft_dtw
module instead, if it works you can make
a PR :) . Changing imports in an __init__.py
like that is weird and could
cause other issues.
For the peak missing the bases, I think that the progress bar's index will denote the curve that it stopped at and the warning is the index location of the peak within the curve that is denoted by the progress bar.
Glad that DropNa worked for you.
On Wed, Jul 21, 2021 at 8:48 PM Pradeep Rajasekhar @.***> wrote:
So, I went to mesmerize\analysisinit.py and added this part
if name == 'main': from .math import drfft_dtw
It seemed to fix the issue.
** One of my peaks didn't have a base. Is there an easy way to detect which cell/trace has it? In the console, I Saw this line:
\anaconda3\envs\mesmerize\lib\site-packages\mesmerize\analysis\compute_peak_features.py:106: UserWarning: Peak at curve index: <0> is not flanked by bases on both sides, ignoring warn(f"Peak at curve index: <{p_ix}> is not flanked by bases on both sides, ignoring")
The dropna node suggestion worked well. **
— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/kushalkolar/MESmerize/issues/61#issuecomment-884588933, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACHXXRF5MLQZTRLJ7NMQNHTTY5TFXANCNFSM5AW5ILEA .
Actually, scrap that previous comment, that doesn't make sense..
So, Is this good practice? Wrap the drfft_dtw code in a main() function and then add this at the end of the module??
if __name__ == '__main__':
main()
I just took a look at that module and I think it'd be better placed in the mesmerize_manuscripts_repo than within mesmerize since it's doesn't exactly fit anywhere with mesmerize itself. I'll reorganize it in a few days and maybe make a new release, but for now you should be able to just safely remove that module's import from mesmerize.analysis.__init__
Thanks for that. Also, is there updated documentation or video available for KShapes with the new options? Thanks for all the tutorial videos you posted, they were extremely helpful! If it helps, I attended your I2K tutorial last year which is how I got onto using MESmerize..
@pr4deepr I haven't update the kshape docs with the gridsearch options. Here's an overview from an internal email:
I added a gridsearch feature to the kshape clustering GUI in mesmerize. It allows you to select a partition range, npart rng
, & a combination number, ncombs
.
npart rng
is a range of partition values for the “search space”.
In each iteration of the gridsearch, it will sort the data (by either peak width or amplitude, see the sortby
param), and randomly select cluster seeds from each partition.
ncombs
is the number of random cluster seed combinations to try for each partition value.
Note that the ncombs
is 10^, so the default value of 2 will do 100 combinations.
Unlike single kShape, the gridsearch is multithreaded so it will simultaneously perform npartitions
* ncombs
kshape-iterations, as per the number of threads that you’ve set in your system configuration.
When it’s done you’ll get a heatmap like this (but bigger). n_clusters
(i.e. npartitions
) are along the y-axis (labels are color coded), and seeded combinations are along the x-axis. The heatmap visualizes the inertia from each k-Shape model. The inertia is within cluster sum of squares, so the smaller the inertia value the tighter the clusters are. You can click on the squares in the heatmap to visualize the model in the rest of the kshape GUI.
Note: If you close the heatmap you’ll have to call this.kga_inertia_heatmap.show()
in the console to get it back, haven’t made a button yet.
You shouldn't necessarily pick the model with the lowest inertia, but it will help narrow down on a suitable model. Some models with very low inertia might have empty cluster(s) which skews the inertia value and these models should be avoided. => I think that current state of the heatmap is that it will indicate models that have empty clusters with a specific color so you can easily avoid them (I think it makes them white?).
You could google "gridsearch parameters" to learn about what a gridsearch is.
Glad to hear you found the tutorials and I2K workshop helpful! :)
Thanks for this detailed comment. I'll be testing it out and will post here or in gitter chat if anything is unclear!!
Closing due to inactivity.
Describe the bug I am using k-Shape Clustering to determine if I can find clusters in my traces based on the shapes of the peak. I get the following error when I click Start.
I followed your video to perform Peak detection and got to the point where I got peak features. I can successfully view different peak parameters using the BeeSwarm Plot. When I go back to the flowchart and connect K-shape to the Peak Detect node, and try the clustering with default options I get the error above. This happens for any peak parameter I select in the data column.
On a related note for K-shape, when I try _pf_peak_curve in the data column, I get this error:
Operating System & specs (CPU, RAM etc.). Please complete the following information:
Details about your Mesmerize install
Thanks for developing and supporting Mesmerize.
Cheers Pradeep