search

jolars / slopecd

4 stars 2 forks source link

Adding pgd_freq argument #27

Closed Klopfe closed 2 years ago

Klopfe commented 2 years ago

This PR adds an argument for the hybrid_solver: pgd_freq. This argument controls the frequency of PGD updates in the hybrid solvers. PGD updates are performed every pgd_freq iterations.

As the first experiment shows, for any pgd_freq > 2, there are no large differences.

pgd_freq = 1 means that it is a PGD solver.

Klopfe commented 2 years ago
Capture d’écran 2022-05-30 à 09 31 46
JonasWallin commented 2 years ago

Could it be an idea to run the CG until either a cluster joins, a cluster switch places, or until CG converges. If we assume that the clusters do not join or cluster, the problem is a just a weighted lasso, in which case duality gap for this subproblem is easy?

Klopfe commented 2 years ago

I am not sure I completely understand what you would like to try?

jolars commented 2 years ago

I am not sure I completely understand what you would like to try?

This is basically what I'm trying here: https://github.com/QB3/slopecd/pull/22

jolars commented 2 years ago

Looks good, @Klopfe. But do you think you could just upload a plot without pgd_freq=1?

Also it might be worth considering a setup where there are a lot of clusters, so high q and high correlation.

Klopfe commented 2 years ago

Looks good, @Klopfe. But do you think you could just upload a plot without pgd_freq=1?

Also it might be worth considering a setup where there are a lot of clusters, so high q and high correlation.

Yes, we have to try multiple settings, indeed. I can do that. This was just to initiate the PR with the possibility to run some tests. I think it's important to show that the choice of this parameter does not drastically affects the performance.

Klopfe commented 2 years ago

I am not sure I completely understand what you would like to try?

This is basically what I'm trying here: #22

I see. Adaptively choose whether we do pgd or cd. Seems interesting. We can continue this discussion on #22 then. I'll look at what you've done there @jolars.

Klopfe commented 2 years ago
Capture d’écran 2022-05-30 à 10 11 52

Thanks! If we don't use an adaptive setting as in #22, then I think we should include a plot like this to, as you say, show that the parameter doesn't make much difference.

By the way, pgd_Freq=2 looks that way just because of numba compiling, right?

Yes, I have updated the cached version...

Klopfe commented 2 years ago

with cluster_updates = True (which was not the case in the first figure) and on suboptimality.

jolars commented 2 years ago

Thanks! If we don't use an adaptive setting as in #22, then I think we should include a plot like this to, as you say, show that the parameter doesn't make much difference.

By the way, pgd_Freq=2 looks that way just because of numba compiling, right?