search

immunogenomics / harmony

Fast, sensitive and accurate integration of single-cell data with Harmony
https://portals.broadinstitute.org/harmony/
Other
515 stars 98 forks source link

did not converge in 25 iterations #177

Open ljy-sys opened 1 year ago

ljy-sys commented 1 year ago

Hello, Thank you very much for the development of this tool. However, when I was analyzing the data, I came across this warning message. Although someone asked me a question, I did not see a clear reply.

cds.harmony<-RunHarmony(cds,group.by.vars = "Sample",max.iter.harmony = 100) Harmony 1/100 0% 10 20 30 40 50 60 70 80 90 100% [----|----|----|----|----|----|----|----|----|----| **| Harmony 2/100 0% 10 20 30 40 50 60 70 80 90 100% [----|----|----|----|----|----|----|----|----|----| **| Harmony 3/100 0% 10 20 30 40 50 60 70 80 90 100% [----|----|----|----|----|----|----|----|----|----| **| Harmony 4/100 0% 10 20 30 40 50 60 70 80 90 100% [----|----|----|----|----|----|----|----|----|----| **| Harmony 5/100 0% 10 20 30 40 50 60 70 80 90 100% [----|----|----|----|----|----|----|----|----|----| **| Harmony 6/100 0% 10 20 30 40 50 60 70 80 90 100% [----|----|----|----|----|----|----|----|----|----| **| Harmony 7/100 0% 10 20 30 40 50 60 70 80 90 100% [----|----|----|----|----|----|----|----|----|----| **| Harmony converged after 7 iterations Warning: Invalid name supplied, making object name syntactically valid. New object name is Seurat..ProjectDim.RNA.harmony; see ?make.names for more details on syntax validity Warning messages: 1: did not converge in 25 iterations 2: Quick-TRANSfer stage steps exceeded maximum (= 7601850) 3: Quick-TRANSfer stage steps exceeded maximum (= 7601850) 4: Quick-TRANSfer stage steps exceeded maximum (= 7601850)

ljy-sys commented 1 year ago

One more question, does RunHarmony involve random algorithms? Why are the results different between the two runs?

lauren-fish commented 1 year ago

I get the same warning message despite harmony converging after ~8 iterations...not sure what this means and having trouble finding out.

hdm-23 commented 10 months ago

I'm seeing these same issues: number of iterations required for convergence changing between runs; invalid name supplied; quick-TRANSfer stage steps exceeded maximum... RunHarmony() also is ignoring the max.iter.harmony call and continues to only run 10 iterations. This occurs with both max.iter.harmony and max.iter options.

pati-ni commented 10 months ago

Hi @hdm-23

We have changed the API of harmony in harmony v1. Now to adjust the harmony iterations, these parameters are max_iter which controls the # of iterations and early_stop=FALSE

pati-ni commented 10 months ago

@ljy-sys @hdm-23 @lauren-fish

These wanings are emitted by the cluster centroid initialization, which is done by a hard k-means cluster. It is just the initial seeding of the cluster centroids before harmony starts running. Initialization of k-means centers for the hard k-means is random. while I doubt that these warnings affect the algorithm in some substantial way, there have been some research on the issue that suggests it may influence the general convergence, but I expect it doesn't affect most datasets since we use several clusters to capture the dataset..

We are considering changing this part of the algorithm with something that is asymptotically more robust but we have not started testing alternatives.

For now you can ignore these warnings and raise an issue if you think something is wrong with the results.

@ljy-sys besides the random seeding of kmeans cells are updated in a random order inside harmony. I would say that this should not affect the outcome.