Handling of large files

[andreas-kupries]

tcllib - migrate check - the troublesome files are two large generated pieces, one is not even installed, comes from research into what implementation to use:

# Module           | File                                | Size [byte] | Check time
# ----------------- ------------------------------------- ------------- ------------
# modules/fumagic  | filetypes.tcl                       | 1,320,164   | ~10 minutes
# modules/textutil | build/alternates/i.map/wcswidth.tcl | 4,492,049   | kill -SIGTERM
# ----------------- ------------------------------------- ------------- ------------

Exclude these two and the check runs in 80 seconds, about, here.

Tx @aku. Good to know though I am not sure much can be done about large files. I will take a look in any case.

Well, we now know that such are a hazard for checking. Fixing ... only if nagelfar can be made faster, I suspect.

A possible amendment in migrate: check file size, warn when > 500K, and skip file. Maybe an option to force scanning despite size, knowing that this will take quite bit longer.

Looking at tcllib, the largest files are:

[...]
-rw-r--r-- 1 aku aku  147014 Jan 16 09:06 ./modules/page/parse_lemon.tcl
-rw-r--r-- 1 aku aku  265465 Jan 16 09:06 ./modules/practcl/practcl.tcl
-rw-r--r-- 1 aku aku 1320164 Jan 16 09:06 ./modules/fumagic/filetypes.tcl
-rw-r--r-- 1 aku aku 4492049 Jan 16 09:06 ./modules/textutil/build/alternates/i.map/wcswidth.tcl

And the 3rd-largest, practcl.tcl is still ok, speed-wise.

[aku]

@andreas-kupries probably you tagged a wrong person )

[andreas-kupries]

My apologies. Had not realized that the @ would tag somebody when I pasted things into the description. The aku is usually a reference to myself.

[apnadkarni]

Added controllable --sizelimit.