I have find a new bug in your code. Heap Overflow

[lyciumlee]

` YAFU Version 2.08 Built with GCC 9 Using GMP-ECM 7.0.5-dev, Powered by GMP 6.2.1 Detected Intel Xeon Processor (Cascadelake) Detected L1 = 32768 bytes, L2 = 16777216 bytes, CL = 64 bytes Using 1 random witness for Rabin-Miller PRP checks Cached 664579 primes; max prime is 9999991 Parsed yafu.ini from /home/lll/yafu-2.0-with-avx512-and-sse4.1-main/yafu

=============================================================== ======= Welcome to YAFU (Yet Another Factoring Utility) ======= ======= bbuhrow@gmail.com ======= ======= Type help at any time, or quit to quit =======

factor(29993503600164729015956324786706888442519396064191560750084140480290745819535179693348191507149250430395035552797283845777601934626280373409219187420866944308214314519319036917673538094015208260669572499228580056077786705598288026374871729383433345507107962917127650202187058774356364876840623349671218708163) fac: check tune params contained invalid parameter(s), ignoring tune info. fac: factoring 29993503600164729015956324786706888442519396064191560750084140480290745819535179693348191507149250430395035552797283845777601934626280373409219187420866944308214314519319036917673538094015208260669572499228580056077786705598288026374871729383433345507107962917127650202187058774356364876840623349671218708163 fac: using pretesting plan: normal fac: check tune params contained invalid parameter(s), ignoring tune info. fac: no tune info: using qs/gnfs crossover of 100 digits fac: no tune info: using qs/snfs crossover of 75 digits div: primes less than 10000 fmt: 10000000 iterations rho: x^2 + 3, starting 500 iterations on C308 rho: x^2 + 2, starting 500 iterations on C308 rho: x^2 + 1, starting 500 iterations on C308 fac: check tune params contained invalid parameter(s), ignoring tune info. nfs: searching for brent special forms... nfs: searching for homogeneous cunningham special forms... nfs: searching for XYYXF special forms... nfs: searching for direct special forms... nfs: snfs form detection took 0.053620 seconds nfs: couldn't find special form pm1: starting B1 = 150K, B2 = gmp-ecm default on C308 pm1: Process took 0.1911 seconds. fac: check tune params contained invalid parameter(s), ignoring tune info. fac: setting target pretesting digits to 94.77 fac: estimated sum of completed work is t0.00 fac: work done at B1=2000: 0 curves, max work = 30 curves fac: 30 more curves at B1=2000 needed to get to t94.77 ecm: 512/64 curves on C308 @ B1=2000, B2=100B1 ecm: process took 1.0121 seconds. fac: check tune params contained invalid parameter(s), ignoring tune info. fac: setting target pretesting digits to 94.77 t15: 17.07 t20: 0.61 fac: estimated sum of completed work is t18.03 fac: work done at B1=11000: 0 curves, max work = 74 curves fac: 74 more curves at B1=11000 needed to get to t94.77 ecm: 512/128 curves on C308 @ B1=11000, B2=100B1 ecm: process took 1.6222 seconds. fac: check tune params contained invalid parameter(s), ignoring tune info. fac: setting target pretesting digits to 94.77 t15: 59.73 t20: 7.53 t25: 0.34 t30: 0.01 fac: estimated sum of completed work is t21.71 fac: work done at B1=50000: 0 curves, max work = 214 curves fac: 214 more curves at B1=50000 needed to get to t94.77 ecm: 512/256 curves on C308 @ B1=50000, B2=100B1 ecm: process took 5.6094 seconds. fac: check tune params contained invalid parameter(s), ignoring tune info. pm1: starting B1 = 3750K, B2 = gmp-ecm default on C308 pm1: Process took 3.6892 seconds. fac: check tune params contained invalid parameter(s), ignoring tune info. fac: setting target pretesting digits to 94.77 t15: 132.88 t20: 31.91 t25: 2.73 t30: 0.17 fac: estimated sum of completed work is t25.83 fac: work done at B1=250000: 0 curves, max work = 430 curves fac: 430 more curves at B1=250000 needed to get to t94.77 ecm: 512/448 curves on C308 @ B1=250000, B2=100B1 ecm: process took 22.3597 seconds. fac: check tune params contained invalid parameter(s), ignoring tune info. pm1: starting B1 = 15M, B2 = gmp-ecm default on C308 pm1: Process took 13.3670 seconds. fac: check tune params contained invalid parameter(s), ignoring tune info. fac: setting target pretesting digits to 94.77 t15: 235.28 t20: 95.91 t25: 12.97 t30: 1.36 t35: 0.11 fac: estimated sum of completed work is t30.56 fac: work done at B1=1000000: 0 curves, max work = 904 curves fac: 904 more curves at B1=1000000 needed to get to t94.77 ecm: 1024/960 curves on C308 @ B1=1000000, B2=100B1 ecm: process took 155.5060 seconds. fac: check tune params contained invalid parameter(s), ignoring tune info. fac: setting target pretesting digits to 94.77 t15: 576.61 t20: 300.71 t25: 64.17 t30: 10.03 t35: 1.24 t40: 0.13 t45: 0.01 fac: estimated sum of completed work is t35.63 fac: work done at B1=3000000: 0 curves, max work = 2350 curves fac: 2350 more curves at B1=3000000 needed to get to t94.77 ecm: 2560/2368 curves on C308 @ B1=3000000, B2=100B1 ecm: process took 1120.9277 seconds. fac: check tune params contained invalid parameter(s), ignoring tune info. fac: setting target pretesting digits to 94.77 t15: 1856.61 t20: 1154.04 t25: 296.90 t30: 57.44 t35: 9.19 t40: 1.22 t45: 0.14 t50: 0.01 fac: estimated sum of completed work is t40.69 fac: work done at B1=11000000: 0 curves, max work = 4480 curves fac: 4480 more curves at B1=11000000 needed to get to t94.77 ecm: 4608/4480 curves on C308 @ B1=11000000, B2=100*B1 ecm: process took 7135.9423 seconds. corrupted size vs. prev_size Aborted (core dumped)

` It seem that there are some heap overflow. It seems that code has not check data len before out them on heap.

[lyciumlee]

another crash infos. >> factor(22111930331381204542945530852361998548318442407872218298358617096708819116917760050316820810420025909178795848107273937553163317177369208499520284574181398183015911140668391445813446367863710044212006290553153052918732733827420444142923043854261762712224865735741111776384712426729456003107915641158324427237) fac: check tune params contained invalid parameter(s), ignoring tune info. fac: factoring 22111930331381204542945530852361998548318442407872218298358617096708819116917760050316820810420025909178795848107273937553163317177369208499520284574181398183015911140668391445813446367863710044212006290553153052918732733827420444142923043854261762712224865735741111776384712426729456003107915641158324427237 fac: using pretesting plan: normal fac: check tune params contained invalid parameter(s), ignoring tune info. fac: no tune info: using qs/gnfs crossover of 100 digits fac: no tune info: using qs/snfs crossover of 75 digits div: primes less than 10000 fmt: 10000000 iterations rho: x^2 + 3, starting 500 iterations on C308 rho: x^2 + 2, starting 500 iterations on C308 rho: x^2 + 1, starting 500 iterations on C308 fac: check tune params contained invalid parameter(s), ignoring tune info. nfs: searching for brent special forms... nfs: searching for homogeneous cunningham special forms... nfs: searching for XYYXF special forms... nfs: searching for direct special forms... nfs: snfs form detection took 0.053647 seconds nfs: couldn't find special form pm1: starting B1 = 150K, B2 = gmp-ecm default on C308 pm1: Process took 0.2033 seconds. fac: check tune params contained invalid parameter(s), ignoring tune info. fac: setting target pretesting digits to 94.77 fac: estimated sum of completed work is t0.00 fac: work done at B1=2000: 0 curves, max work = 30 curves fac: 30 more curves at B1=2000 needed to get to t94.77 ecm: 512/64 curves on C308 @ B1=2000, B2=100*B1 ecm: process took 0.8397 seconds. fac: check tune params contained invalid parameter(s), ignoring tune info. fac: setting target pretesting digits to 94.77 t15: 17.07 t20: 0.61 fac: estimated sum of completed work is t18.03 fac: work done at B1=11000: 0 curves, max work = 74 curves fac: 74 more curves at B1=11000 needed to get to t94.77 ecm: 512/128 curves on C308 @ B1=11000, B2=100*B1 ecm: process took 1.4716 seconds. fac: check tune params contained invalid parameter(s), ignoring tune info. fac: setting target pretesting digits to 94.77 t15: 59.73 t20: 7.53 t25: 0.34 t30: 0.01 fac: estimated sum of completed work is t21.71 fac: work done at B1=50000: 0 curves, max work = 214 curves fac: 214 more curves at B1=50000 needed to get to t94.77 ecm: 512/256 curves on C308 @ B1=50000, B2=100*B1 ecm: process took 5.1241 seconds. fac: check tune params contained invalid parameter(s), ignoring tune info. pm1: starting B1 = 3750K, B2 = gmp-ecm default on C308 pm1: Process took 3.8899 seconds. fac: check tune params contained invalid parameter(s), ignoring tune info. fac: setting target pretesting digits to 94.77 t15: 132.88 t20: 31.91 t25: 2.73 t30: 0.17 fac: estimated sum of completed work is t25.83 fac: work done at B1=250000: 0 curves, max work = 430 curves fac: 430 more curves at B1=250000 needed to get to t94.77 ecm: 512/448 curves on C308 @ B1=250000, B2=100*B1 ecm: process took 20.5225 seconds. fac: check tune params contained invalid parameter(s), ignoring tune info. pm1: starting B1 = 15M, B2 = gmp-ecm default on C308 pm1: Process took 13.3410 seconds. fac: check tune params contained invalid parameter(s), ignoring tune info. fac: setting target pretesting digits to 94.77 t15: 235.28 t20: 95.91 t25: 12.97 t30: 1.36 t35: 0.11 fac: estimated sum of completed work is t30.56 fac: work done at B1=1000000: 0 curves, max work = 904 curves fac: 904 more curves at B1=1000000 needed to get to t94.77 ecm: 1024/960 curves on C308 @ B1=1000000, B2=100*B1 ecm: process took 154.3178 seconds. fac: check tune params contained invalid parameter(s), ignoring tune info. fac: setting target pretesting digits to 94.77 t15: 576.61 t20: 300.71 t25: 64.17 t30: 10.03 t35: 1.24 t40: 0.13 t45: 0.01 fac: estimated sum of completed work is t35.63 fac: work done at B1=3000000: 0 curves, max work = 2350 curves fac: 2350 more curves at B1=3000000 needed to get to t94.77 ecm: 2560/2368 curves on C308 @ B1=3000000, B2=100*B1 ecm: process took 1157.2819 seconds. fac: check tune params contained invalid parameter(s), ignoring tune info. fac: setting target pretesting digits to 94.77 t15: 1856.61 t20: 1154.04 t25: 296.90 t30: 57.44 t35: 9.19 t40: 1.22 t45: 0.14 t50: 0.01 fac: estimated sum of completed work is t40.69 fac: work done at B1=11000000: 0 curves, max work = 4480 curves fac: 4480 more curves at B1=11000000 needed to get to t94.77 ecm: 4608/4480 curves on C308 @ B1=11000000, B2=100*B1 ecm: process took 7176.1531 seconds. munmap_chunk(): invalid pointer Aborted (core dumped)

[bbuhrow]

So far I have been unable to reproduce this problem. I've used both gcc-11.1 and icc 19.1 on RHEL 7.9. I have also used 64 threads as you appear to be doing here. I will keep investigating

[lyciumlee]

Thank you, I will use the gcc 11 to compile it. Multithreading programming is a sophisticated process. It is hard to reproduce the problem.

[alexhiggins732]

Interestingly I see: fac: check tune params contained invalid parameter(s), ignoring tune info. fac: setting target pretesting digits to 94.77

What is your tune info?

I also wonder if pretesting up to 94 bits is causing an issue. I see in your traces it crashes at pretesting is above 40 trying to move to 45. I looked at the doc file and cannot find, but I think I remember reading about one of the routines having a limit of between 45 and 50.

[alexhiggins732]

When I run v 2.11.0, with 16 threads, I don't get an exception. I also don't get a factor. Not sure why fac ends prematurely here.

2.0.6 doesn't seem to respect the threads argument and ECM is only ran on a single core.

C:\Users\alexh\Downloads\yafu\yafu-2.11.0>yafu-x64-2.11.0 -threads 16

YAFU Version 2.11
Built with Microsoft Visual Studio 1934 and Intel Compiler 2021
Using GMP-ECM 7.0.6-dev, Powered by MPIR 3.0.0
Detected AMD Ryzen 7 2700X Eight-Core Processor
Detected L1 = 32768 bytes, L2 = 16777216 bytes, CL = 64 bytes
CPU features enabled: SSE41 AVX2 BMI2
Using 1 random witness for Rabin-Miller PRP checks
Cached 664579 primes; max prime is 9999991
Parsed yafu.ini from C:\Users\alexh\Downloads\yafu\yafu-2.11.0

===============================================================
======= Welcome to YAFU (Yet Another Factoring Utility) =======
=======             bbuhrow@gmail.com                   =======
=======     Type help at any time, or quit to quit      =======
===============================================================

>> factor(22111930331381204542945530852361998548318442407872218298358617096708819116917760050316820810420025909178795848107273937553163317177369208499520284574181398183015911140668391445813446367863710044212006290553153052918732733827420444142923043854261762712224865735741111776384712426729456003107915641158324427237)

fac: factoring 22111930331381204542945530852361998548318442407872218298358617096708819116917760050316820810420025909178795848107273937553163317177369208499520284574181398183015911140668391445813446367863710044212006290553153052918732733827420444142923043854261762712224865735741111776384712426729456003107915641158324427237
fac: using pretesting plan: normal
fac: custom pretesting limit is: 35
fac: no tune info: using qs/gnfs crossover of 95 digits
fac: no tune info: using qs/snfs crossover of 95 digits
div: primes less than 10000
fmt: 1000000 iterations
rho: x^2 + 3, starting 1000 iterations on C308
rho: x^2 + 2, starting 1000 iterations on C308
rho: x^2 + 1, starting 1000 iterations on C308
nfs: searching for brent special forms...
nfs: searching for homogeneous cunningham special forms...
nfs: searching for XYYXF special forms...
nfs: searching for direct special forms...
nfs: snfs form detection took 0.054000 seconds
nfs: couldn't find special form
pm1: starting B1 = 150K, B2 = gmp-ecm default on C308
ecm: 30/30 curves on C308, B1=2k, B2=gmp-ecm default
ecm: 74/74 curves on C308, B1=11k, B2=gmp-ecm default
ecm: 214/214 curves on C308, B1=50k, B2=gmp-ecm default, ETA: 0 sec
pm1: starting B1 = 3750K, B2 = gmp-ecm default on C308
ecm: 430/430 curves on C308, B1=250k, B2=gmp-ecm default, ETA: 0 sec
pm1: starting B1 = 15M, B2 = gmp-ecm default on C308
ecm: 824/824 curves on C308, B1=1M, B2=gmp-ecm default, ETA: 0 sec
Total factoring time = 943.9406 seconds

***factors found***

C308 = 22111930331381204542945530852361998548318442407872218298358617096708819116917760050316820810420025909178795848107273937553163317177369208499520284574181398183015911140668391445813446367863710044212006290553153052918732733827420444142923043854261762712224865735741111776384712426729456003107915641158324427237

ans = 22111930331381204542945530852361998548318442407872218298358617096708819116917760050316820810420025909178795848107273937553163317177369208499520284574181398183015911140668391445813446367863710044212006290553153052918732733827420444142923043854261762712224865735741111776384712426729456003107915641158324427237

>>

[alexhiggins732]

I think the issue not getting a factor might be yafu.ini. Going to re-rerun with the pretest commented out.

ggnfs_dir=ggnfs\ %work=65 pretest=35 % <-- Dope!

[alexhiggins732]

Nope. Wasn't the ini. Same result with no factor with:

ggnfs_dir=ggnfs\ %work=65 %pretest=35