Dynamically change the dimensions of unknowns without re-compiling?

[shengyenlo]

Is it possible changing the dimensions of unknowns dynamically without re-compiling (calling Opt_ProblemPlan(.) )? Thanks.

[Mx7f]

Not in the current version of Opt. We are working on a larger-scale revision to how Opt handle scheduling units of work that would likely subsume this concern. Out of curiosity, what is your use case; some sort of remeshing? And how urgent is this ask?

[shengyenlo]

OK, many thanks. And Yes, I'm working on the nonrigid surface tracking, fusion and remeshing now. I'd just like to compare the performance between different solvers.

[eshafeeqe]

I think I have a similar problem, I am working on non-rigid tracking and reconstruction, so after each iteration, my surface gets updated, and unknowns need to be updated. Currently, I am calling these functions to delete the problem and start new problem on each iteration

Opt_PlanFree(m_optimizerState, m_plan); Opt_ProblemDelete(m_optimizerState, m_problem);

However, after 36 iterations, I am getting following error

nUnknowns = 32304 nResiduals = table: 0x7ccd7d10 nnz = table: 0x7ccd96a0 //////////// ITERATION1 (OPT) /////////////// 0 terra (JIT) 0x00007fc01446c004 $opt.SetSolverParameter + 4 /home/aslab/shafeeq-project/new_codes/C_plus_plus/my_codes/NICP_testing/src/cmake-build-debug/Kintinuous(_ZN12ImageWarping5solveEv+0x19e) [0x804d2e]

Any workarounds for this problem?

[eshafeeqe]

Sometimes I am getting this error message also

PANIC: unprotected error in call to Lua API (not enough memory)

[Mx7f]

That sounds like a memory leak to me! I'll work on tracking that down this weekend. As for a workaround, you could define some maximum problem size and have dummy unknowns that pad out the space. Though, if you don't need to fix it before Monday I should hopefully have the leak tracked down by then.

[eshafeeqe]

I am not in that hurry, one more thing, I have your previous version of the library (got by contacting niessner). In that, I can able to run 100-110 iterations before crashing.

[Mx7f]

That's consistent with the memory leak theory; I should have a fix by tomorrow for you to try.

[eshafeeqe]

OK thank you so much

[Mx7f]

I've fixed leaking of cuda buffers. There are likely still other memory leaks, but you might be able to get more iterations. I'll work on this more next week.

[eshafeeqe]

I tried the new version and got the crash after 36 iterations (same as before) with following error message.

PANIC: unprotected error in call to Lua API (not enough memory)

[eshafeeqe]

I think the error is not because of memory leaks, similar error reported in other github projects

https://github.com/OpenNMT/OpenNMT/issues/26

[Mx7f]

The error is we are hitting the (very low) LuaJIT memory limit; I am working on lowering our Lua memory use.

[Mx7f]

Should be helped greatly by #73; give it a try if you can.

[eshafeeqe]

Hello @Mx7f

Thanks for the update, I tried the new library, and now I am only doing Opt_PlanFree, since my problem is same only the dimension is different. I can able to run the code 520+ iterations before crashing. But the crashing problem still exists.

[Mx7f]

This is definitely a wart on the current version of Opt that I will work on, but you can workaround this by always using the maximum problem size (could have a small set of problem sizes you choose from) and padding the dimensions with dummy unknowns with a signal value that you detect in the energy and set the energy to 0 when it involves said dummy unknowns.

[dgrzech]

@Mx7f has there been any progress on this? i'm running into the same problem after only about 20 iterations of the solver. is it likely that the luajit memory limit will be bypassed?