Question: hiding away parts of terms

[jodavies]

I have a series of modules which operate on parts of terms, which many terms have in common. So, of course, it helps a lot to do something like

Bracket x,y,z;
.sort
Collect f;
* expensive modules involving x,y,z
Identify f(x?) = x;

However, the intermediate .sorts are still expensive, because the size of the expression is still large.

It is much faster to do something like

Bracket x,y,z;
.sort
Collect f;
ArgToExtraSymbol f;
* expensive modules involving x,y,z
Identify f(x?) = x;
FromPolynomial;

since the size of the expression becomes much much smaller than in the first case.

The problem here is that storing the extrasymbols becomes tricky as the expressions get larger. I have many crashes due to ==> No memory while allocating compiler buffer-4 and, if I print the size of the failed allocation, it can be >60GB.

Is there a better way to do this? These extrasymbols ideally could all be stored on the disk, if I could somehow get the function arguments into individual local expressions which I then hide, or similar.

Any ideas? Thanks, Josh.

[magv]

So, are there any updates on this? I've also found that hiding the temporary unneeded parts of terms into the extra symbols via brackets and topolynomial and/or argtoextrasymbol significantly improves the runtime of e.g. color.h, but No memory while allocating compiler buffer-4 error breaks this technique for non-trivial calculations. Can one overcome this somehow?