Speeding up the insertion of reduction tables using tablebases

[vsht]

Hi,

I'm sure that people have already run into similar issues in the past, so hopefully there should be a solution to my problem.

Upon doing an IBP reduction I end up with quite complicated tables that I want to insert using FORM. To speed things up as much as possible, I first generate fill statements, then create tablebases out of them and finally apply those to my expression.

In the case at hand there's only one single integral but a very complicated reduction rule.

S lsclEp, u0b;
CF topology5534, lsclNum, lsclDen, lsclEpHelpFlag;

L exp = topology5534(1,1,1,1,1,1,1,1,1,1,1,-1)*lsclEp;

.sort 

TableBase "tablebaseExpanded0.tbl" open;
TableBase "tablebaseExpanded0.tbl" enter;

 .sort: TableBase;

id topology5534(?a) = tabIBPtopology5534(?a);

 testuse tabIBPtopology5534;

.sort: TestUse;

* print;

apply;

.sort: Apply;

.end

The corresponding tablebase and the fill statements used to generate it are attached.

tablebase.zip

Here's the code I use for generating tablebases

off statistics;

S lsclEp, u0b;
CF topology5534, lsclNum, lsclDen, lsclEpHelpFlag;

#include fillStatementsExpanded0.frm #lsclCurrentTopologyDefinitions

CF 
#include fillStatementsExpanded0.frm #lsclTopologyNames
;

Table,sparse,zerofill,strict,tabIBPtopology5534(`LSCLCURRENTNPROPS');

#include fillStatementsExpanded0.frm #lsclFillStatements
.sort 

TableBase "tablebaseExpanded0-new.tbl" create;
TableBase "tablebaseExpanded0-new.tbl" addto tabIBPtopology5534;

.sort

.end

My problem is that .sort after apply; literally takes ages (currently running on a cluster for over 6 days!) and requires a lot of RAM.

So any trick to speed up this insertion would be highly appreciated.

Cheers, Vlad

[jodavies]

It will help if you can prepare your Fill statements without any factorization. Brackets are not expanded in the Fill statement's RHS, so those are being expanded only when the tablebase is applied.

[vsht]

But I thought that using factorization I kind of reduce the complexity of the expression: without lsclNum and lsclDen the expression would become even larger so even more terms will be generated. Or am I missing something here?

[jodavies]

You are right, but your formatting doesn't quite do what you want. You have structures like: lsclDen(3 + u0b - u0b^2)*(1242 - 7038*lsclNum(u0b) + 19239*lsclNum(u0b^2) - 23772*lsclNum(u0b^3) + ... )*topology5534(1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 0).

Form will be much happier if you can make a numerator function like: lsclDen(3 + u0b - u0b^2)*lsclNum(1242 - 7038*u0b + 19239*u0b^2 - 23772*u0b^3 + ... )*topology5534(1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 0).

The attached fill statement should apply basically instantly. fillStatementsExpanded0-single-jd.txt.gz

This is generated with the following after removing all of the existing lsclNum and lsclDen functions (but maybe it doesn't work more generally for your problems)

lsclNum[x_Integer] := x;
lsclDen[x_Integer] := 1/x;
coeffSimplify[expr_] := Module[
   {tmp, num, den},
   tmp = Together[expr];
   den = Times @@ (lsclDen[#[[1]]]^#[[2]]& /@ FactorList[Denominator[tmp]]);
   num = Times @@ (lsclNum[#[[1]]]^#[[2]]& /@ FactorList[Numerator[tmp]]);
   Return[num*den];
];

test = Collect[test, {_topology5534,lsclEp,_lsclEpHelpFlag}, coeffSimplify];

In general formatting tables so they can be applied quickly in form is a bit non-trivial, I also post-process IBP tables in a somewhat similar way to make tablebases, but this is itself a computationally intensive step...

[vsht]

Ok, now I see your point. Indeed, the way I generate the fill statements is probably quite inefficient and creates too much extra work for FORM. I'll try to rework it accordingly.