Add CP_OPT

[ntjohnson1]

Goal: Align implementation with GCP_OPT while retaining the cp_opt top level interface as faithfully as possible.

Components:

• [x] cp_opt.m
• [x] ktensor/fg.m
• [x] tt_opt_lbfgsb.m
• [ ] tt_opt_adam.m
• [ ] cp_opt_doc.m

Won't do unless needed/requested as follow up:

• cp_opt_legacy.m
• tt_opt_lbfgs.m, Poblano isn't available in python, scipy has a standard BFGS solve but not LBFGS. With the other implementations adding a new optimizer shouldn't be too much of a burden
• tt_opt_compact.m The solver from this paper is publicly available but uses pytorch. That's a pretty heavy include for our project (also it doesn't look like they publish the solver to pypi)
• tt_opt_fmincon.m/ tt_opt_fminunc.m Both of these seem to use matlab optimization tools that run a variety of heuristics before dispatching to a specific solver. We have an example of lbfgsb from scipy so if there are specific solvers desired we could add those. But would be hard to come up with an apples to apples comparison.

[dmdunla]

I think that scipy’s L-BFGS-B (via minimize) without constraints reduces to L-BFGS. Will that work to support the tt_opt_lbfgs.m use case you called out for cp_opt?

---

From: Nick @.> Sent: Saturday, November 23, 2024 10:15 AM To: sandialabs/pyttb @.> Cc: Subscribed @.***> Subject: [EXTERNAL] [sandialabs/pyttb] Add CP_OPT (Issue #351)

Goal: Align implementation with GCP_OPT while retaining the cp_opt top level interface as faithfully as possible.

Components: [ ] cp_opt.m [ ] ktensor/fg.m [ ] tt_opt_lbfgsb.m [ ] tt_opt_fmincon.m [ ] tt_opt_fminunc.m [ ] tt_opt_compact.m [ ] tt_opt_adam.m

Won't do unless needed/requested as follow up:

• cp_opt_legacy.m
• tt_opt_lbfgs.m, Poblano isn't available in python, scipy has a standard BFGS solve but not LBFGS. With the other implementations adding a new optimizer shouldn't be too much of a burden

— Reply to this email directly, view it on GitHubhttps://github.com/sandialabs/pyttb/issues/351, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AHIY2IVG4U3YLTJ27INH45L2CCZ4LAVCNFSM6AAAAABSLIQZ4KVHI2DSMVQWIX3LMV43ASLTON2WKOZSGY4DMMRYGM4TMNA. You are receiving this because you are subscribed to this thread.Message ID: @.***>

[ntjohnson1]

I can take a look at the end (or potentially do it as a follow up).

[ntjohnson1]

I timed out for this afternoon but here is my development branch for this: https://github.com/ntjohnson1/pyttb/tree/nick/cp_opt

I have most of cp_opt, function/gradient handles for the specific cp opt problem, and lbfgsb implemented. Still need to improve doc strings, and tests to confirm things are functionally correct (right now I have a simple test to confirm when the answer is provided for init things word). I basically followed the same internal structure for the solvers in gcp_opt, however the cp_opt components operate on actual tensors rather than just blocks of data so I had to slightly tweak some interfaces. No rush, but feel free to share if you have any thoughts on the layout/alignment with gcp_opt.

I also updated the description for some of the other optimizers python had used. My preference would be to implement a subset of the optimizers if the primary motivation is the textbook rather than real world usage.