search

vermaseren / form

The FORM project for symbolic manipulation of very big expressions
GNU General Public License v3.0
1.14k stars 136 forks source link

(1)Output should fit inside a single term #177

Open jodavies opened 7 years ago

jodavies commented 7 years ago

Hello,

I am using polyratfun for some coefficient simplification, and I hit the error

(1)Output should fit inside a single term. Increase MaxTermSize?

This of course is not a problem, but it would be helpful if the error message also printed the size of the output. Then, the user can adjust the MaxTermSize accordingly (or give up, if this size would be very very large...).

There are several routes to this message (via the TooLarge label, sort.c line 930) so it is not clear to me precisely how one would implement this.

Is this a trivial addition?

Thanks, Josh.

jodavies commented 7 years ago

Along these lines... do you have any tips for dealing with such large polyratfuns? I have expressions that don't fit into a MaxTermSize of 20M. TFORM allocates an enormous amount of memory (>1.5TB) but its actual resident set is not so bad, more like 50GB.

Is there any hope at these sizes, or do I need to re-think things? What are the largest polyratfuns you have dealt with?

Thanks, Josh.

vermaseren commented 7 years ago

Hi Josh,

When things get this big, you better rethink a bit. Many buffers have sizes that are coupled to MaxTermSize. The easiest is that you need more workspace, but that is still rather benign but it helps in that case if your modules do not get into recursions that are very deep. The more problematic thing is that the sort buffers are automatically expanded, because the way they are set up, they can be divided into many subbuffers (for TFORM) and each needs to contain at least 2*MaxtermSize. This includes the large buffer because it is used during sorting. If you make maxtermsize too big, your computer may not have enough virtual memory for the allocations. Personally I have not gone beyond 10-50 Mbytes and that only once or twice on something that was otherwise relatively simple and hence did not need excessive amounts of space otherwise. And in that case it helps a lot when you just use FORM and not TFORM.

Printing the size of the term that makes the system crash, may or may not be useful. There may be even bigger ones coming and the crash is immediate. What I mean is that if you have (in order) terms of 10M,20M,30M,40M,50M coming up and 10M is already too big, it would print that you need more than 10M and have no idea about the remaining part, because it just bails out.

I do not (yet?) have a ready made solution for your problem. It is something I struggle with as well.

Jos

On 26 apr. 2017, at 09:24, Josh Davies notifications@github.com wrote:

Along these lines... do you have any tips for dealing with such large polyratfuns? I have expressions that don't fit into a MaxTermSize of 20M. TFORM allocates an enormous amount of memory (>1.5TB) but its actual resident set is not so bad, more like 50GB.

Is there any hope at these sizes, or do I need to re-think things? What are the largest polyratfuns you have dealt with?

Thanks, Josh.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/vermaseren/form/issues/177#issuecomment-297264731, or mute the thread https://github.com/notifications/unsubscribe-auth/AFLxEgGzHEb1FGZ37yFe-QMPTpWQeY5Jks5rzvE7gaJpZM4MsonT.