voxel_vote produces broken voted file for ultra-high res unsigned-int label files

gdevenyi commented 2 years ago

I'm putting in files that are properly aligned:

With the following properties:

$ mincinfo reg2_ABI_to_Dorr_2_10_1_NL.mnc
file: reg2_ABI_to_Dorr_2_10_1_NL.mnc
image: unsigned long 0 to 614454277
image dimensions: zspace yspace xspace
    dimension name         length         step        start
    --------------         ------         ----        -----
    zspace                    241         0.04         -4.2
    yspace                    478         0.04        -8.19
    xspace                    315         0.04        -6.27
$ $ print_all_labels reg2_ABI_to_Dorr_2_10_1_NL.mnc
Label: 1 1684
Label: 2 1459
Label: 6 19685
Label: 7 18351
Label: 9 3818
Label: 10 31582
Label: 12 1381
Label: 15 2152
Label: 17 14202
Label: 19 4331
Label: 20 30299
Label: 23 8509
Label: 26 5838
Label: 27 2115
Label: 28 15478
Label: 30 451
Label: 33 19907
Label: 35 573
Label: 36 2961
Label: 38 1671
Label: 41 4284
Label: 42 1945
Label: 50 1846
Label: 52 16301
Label: 54 597
Label: 56 57048
Label: 58 482
Label: 59 709
Label: 62 2631
Label: 63 997
Label: 64 2992
Label: 66 288
Label: 67 1302
Label: 68 3079
Label: 72 1561
Label: 74 2922
Label: 75 332
Label: 78 8460
Label: 81 38454
Label: 83 16515
Label: 84 7335
Label: 88 6782
Label: 91 13073
Label: 93 348
Label: 96 9634
Label: 97 9473
Label: 98 3974
Label: 100 172
Label: 101 17566
Label: 102 966
Label: 105 1962
Label: 106 165
Label: 108 18013
Label: 113 16956
Label: 114 4811
Label: 115 171
Label: 117 3189
Label: 118 2008
Label: 120 15531
Label: 121 619
Label: 122 4059
Label: 123 2940
Label: 125 8316
Label: 126 835
Label: 128 61826
Label: 129 12900
Label: 131 11482
Label: 132 415
Label: 133 1479
Label: 136 62363
Label: 139 15366
Label: 140 3367
Label: 143 505
Label: 145 6410
Label: 146 35897
Label: 147 102
Label: 148 2524
Label: 149 4285
Label: 153 6696
Label: 155 15994
Label: 156 2308
Label: 158 504
Label: 159 73170
Label: 162 2630
Label: 163 18467
Label: 164 1
Label: 169 3296
Label: 171 10506
Label: 173 2416
Label: 177 902
Label: 178 10047
Label: 180 6661
Label: 181 2919
Label: 186 2899
Label: 187 9711
Label: 188 1429
Label: 189 563
Label: 190 4749
Label: 194 28474
Label: 196 1184
Label: 197 810
Label: 198 1092
Label: 201 34751
Label: 202 38211
Label: 203 1372
Label: 204 3175
Label: 206 5309
Label: 207 407
Label: 209 4477
Label: 210 1624
Label: 211 15586
Label: 214 10638
Label: 215 10782
Label: 217 4699
Label: 218 17986
Label: 222 1898
Label: 223 2014
Label: 225 12292
Label: 226 10912
Label: 229 6113
Label: 230 782
Label: 231 698
Label: 233 3915
Label: 234 5657
Label: 237 127
Label: 238 2100
Label: 243 270
Label: 246 1336
Label: 249 966
Label: 250 7212
Label: 251 3426
Label: 252 5944
Label: 255 6806
Label: 257 2296
Label: 258 21430
Label: 260 1355
Label: 262 24192
Label: 263 1111
Label: 266 6023
Label: 268 1002
Label: 269 3643
Label: 271 366
Label: 272 1574
Label: 274 28574
Label: 279 435
Label: 280 96
Label: 281 2438
Label: 286 749
Label: 287 92
Label: 288 10840
Label: 289 13939
Label: 292 352
Label: 296 13328
Label: 298 5783
Label: 301 2694
Label: 303 7189
Label: 304 7167
Label: 305 1994
Label: 307 15294
Label: 310 7081
Label: 311 7213
Label: 313 85402
Label: 314 2843
Label: 318 89
Label: 320 21607
Label: 321 444
Label: 325 3650
Label: 326 6893
Label: 327 11411
Label: 328 18166
Label: 330 391
Label: 332 45
Label: 333 893
Label: 334 7165
Label: 335 675
Label: 336 305
Label: 338 390
Label: 342 47408
Label: 344 8608
Label: 347 971
Label: 349 330
Label: 350 585
Label: 351 16533
Label: 354 87846
Label: 355 204
Label: 356 130
Label: 358 1139
Label: 362 6997
Label: 363 19590
Label: 364 1797
Label: 366 2246
Label: 368 231
Label: 372 849
Label: 374 1983
Label: 377 2047
Label: 380 195
Label: 381 22621
Label: 382 225737
Label: 390 525
Label: 393 126
Label: 397 565
Label: 401 1229
Label: 403 27837
Label: 412 9804
Label: 413 2659
Label: 414 449
Label: 421 2732
Label: 422 1639
Label: 423 12700
Label: 427 11178
Label: 428 115
Label: 429 36586
Label: 430 14969
Label: 433 5069
Label: 434 9787
Label: 436 4156
Label: 437 43899
Label: 440 6278
Label: 441 301
Label: 442 13877
Label: 443 12809
Label: 445 18591
Label: 448 7210
Label: 449 1543
Label: 450 6695
Label: 451 5178
Label: 452 577
Label: 456 215
Label: 460 155
Label: 461 419
Label: 463 119847
Label: 466 46735
Label: 469 410
Label: 470 1944
Label: 477 44096
Label: 478 6482
Label: 482 4032
Label: 483 3917
Label: 484 16316
Label: 488 691
Label: 501 2662
Label: 502 41081
Label: 506 147
Label: 507 297357
Label: 510 701
Label: 512 9531
Label: 515 2835
Label: 520 4155
Label: 523 8498
Label: 525 3251
Label: 526 30462
Label: 527 1885
Label: 530 1010
Label: 531 777
Label: 534 4513
Label: 538 1580
Label: 540 8444
Label: 542 20493
Label: 543 21095
Label: 544 2994
Label: 549 13438
Label: 551 1701
Label: 553 1286
Label: 556 4213
Label: 558 6066
Label: 559 7786
Label: 564 5091
Label: 565 6905
Label: 566 15296
Label: 573 2604
Label: 574 11419
Label: 575 2071
Label: 576 155
Label: 577 12408
Label: 579 19369
Label: 580 1391
Label: 581 5027
Label: 582 7057
Label: 583 7108
Label: 587 1245
Label: 588 13481
Label: 590 25897
Label: 591 1453
Label: 593 16644
Label: 595 1584
Label: 596 18167
Label: 597 24327
Label: 598 527
Label: 599 1248
Label: 600 3710
Label: 601 938
Label: 603 25915
Label: 604 1721
Label: 605 19035
Label: 608 3530
Label: 609 721
Label: 610 18572
Label: 611 596
Label: 612 6540
Label: 613 3552
Label: 614 9080
Label: 616 2857
Label: 620 9181
Label: 621 7251
Label: 622 975
Label: 625 9052
Label: 628 3072
Label: 629 7232
Label: 630 20429
Label: 632 22704
Label: 633 92
Label: 634 2958
Label: 638 5874
Label: 639 13582
Label: 642 1303
Label: 643 2090
Label: 648 53846
Label: 649 187
Label: 651 19053
Label: 653 521
Label: 654 14860
Label: 655 16205
Label: 656 37032
Label: 657 31905
Label: 658 5527
Label: 661 16647
Label: 662 743
Label: 663 13906
Label: 664 14752
Label: 665 12094
Label: 667 2232
Label: 670 6216
Label: 671 9154
Label: 672 333393
Label: 673 563
Label: 675 2015
Label: 678 1590
Label: 679 11135
Label: 680 92
Label: 681 1161
Label: 685 9574
Label: 687 32151
Label: 689 1107
Label: 690 1260
Label: 692 2316
Label: 693 3516
Label: 694 13499
Label: 696 1441
Label: 697 8393
Label: 698 73511
Label: 699 65
Label: 702 7931
Label: 703 5297
Label: 704 4592
Label: 706 718
Label: 707 7728
Label: 711 6472
Label: 718 17092
Label: 721 17994
Label: 725 1058
Label: 727 12923
Label: 728 99227
Label: 729 6133
Label: 732 359
Label: 733 20889
Label: 735 2750
Label: 741 1100
Label: 743 9852
Label: 744 496
Label: 749 4657
Label: 750 4176
Label: 753 449
Label: 754 49968
Label: 755 3946
Label: 757 220
Label: 759 1161
Label: 763 39
Label: 765 866
Label: 767 90564
Label: 771 53809
Label: 772 20322
Label: 773 2565
Label: 774 12489
Label: 778 28150
Label: 780 9882
Label: 781 233
Label: 783 9191
Label: 784 1213
Label: 786 1310
Label: 788 17672
Label: 791 2719
Label: 794 18284
Label: 795 33686
Label: 797 27952
Label: 798 829
Label: 800 9530
Label: 802 4015
Label: 803 14889
Label: 804 2380
Label: 805 3240
Label: 806 27313
Label: 810 8468
Label: 811 3932
Label: 812 507
Label: 814 12413
Label: 816 1313
Label: 819 2248
Label: 820 9447
Label: 821 28618
Label: 827 5367
Label: 828 13293
Label: 830 2691
Label: 831 926
Label: 832 82
Label: 834 13562
Label: 836 8108
Label: 838 21526
Label: 839 1814
Label: 841 1860
Label: 842 23351
Label: 843 14186
Label: 844 37810
Label: 846 4608
Label: 847 9503
Label: 848 1218
Label: 849 646
Label: 850 1056
Label: 851 13415
Label: 852 34761
Label: 854 29303
Label: 857 5044
Label: 859 644
Label: 862 25007
Label: 863 5435
Label: 866 3000
Label: 867 14448
Label: 869 706
Label: 872 2227
Label: 873 11919
Label: 874 298
Label: 878 9579
Label: 880 1245
Label: 882 2330
Label: 884 5422
Label: 888 9849
Label: 889 6825
Label: 893 2418
Label: 897 6686
Label: 898 9106
Label: 900 7038
Label: 902 5766
Label: 903 2716
Label: 905 2838
Label: 906 9304
Label: 907 963
Label: 908 2323
Label: 910 2337
Label: 911 109
Label: 912 871
Label: 914 85
Label: 916 8879
Label: 919 11945
Label: 924 13409
Label: 927 549
Label: 929 477
Label: 930 3133
Label: 931 19946
Label: 935 14382
Label: 936 43974
Label: 939 335
Label: 940 25840
Label: 943 69753
Label: 944 17530
Label: 945 7789
Label: 946 7866
Label: 949 250
Label: 950 29473
Label: 951 24504
Label: 952 20824
Label: 954 7234
Label: 955 11309
Label: 956 14107
Label: 957 45272
Label: 959 2766
Label: 961 178779
Label: 962 66402
Label: 963 332
Label: 964 872
Label: 965 10409
Label: 966 13265
Label: 968 31112
Label: 969 11535
Label: 970 6799
Label: 971 19457
Label: 973 4700
Label: 974 24575
Label: 975 197
Label: 976 8229
Label: 977 5557
Label: 978 14753
Label: 980 1542
Label: 981 12365
Label: 982 2873
Label: 984 33546
Label: 986 15944
Label: 988 6848
Label: 989 6987
Label: 990 1953
Label: 996 7226
Label: 997 69137
Label: 998 5579
Label: 1004 2624
Label: 1005 684
Label: 1006 3257
Label: 1007 76975
Label: 1009 22797
Label: 1010 3157
Label: 1015 35103
Label: 1016 46609
Label: 1020 9277
Label: 1021 40732
Label: 1022 12727
Label: 1023 9908
Label: 1025 76018
Label: 1026 871
Label: 1029 2900
Label: 1030 4820
Label: 1031 5119
Label: 1033 36022
Label: 1035 19162
Label: 1037 41078
Label: 1038 11795
Label: 1039 610
Label: 1041 66810
Label: 1043 10759
Label: 1044 644
Label: 1045 1291
Label: 1046 2275
Label: 1047 32567
Label: 1048 55553
Label: 1049 18554
Label: 1052 12599
Label: 1054 2492
Label: 1056 77513
Label: 1058 9904
Label: 1060 113
Label: 1061 883
Label: 1062 1513
Label: 1064 57073
Label: 1066 2940
Label: 1069 928
Label: 1070 17107
Label: 1072 2895
Label: 1074 1795
Label: 1077 1047
Label: 1079 3088
Label: 1081 182
Label: 1084 13082
Label: 1085 838
Label: 1086 3093
Label: 1088 4368
Label: 1089 8139
Label: 1090 43303
Label: 1091 89438
Label: 1092 2598
Label: 1093 38463
Label: 1094 7003
Label: 1096 1915
Label: 1097 37801
Label: 1098 32657
Label: 1101 25887
Label: 1102 15361
Label: 1104 1814
Label: 1105 3030
Label: 1106 11687
Label: 1107 46356
Label: 1108 21108
Label: 1109 1542
Label: 1111 7572
Label: 1113 797
Label: 1114 2122
Label: 1116 422
Label: 1120 276
Label: 1121 30199
Label: 1123 7816
Label: 1125 16242
Label: 1126 182
Label: 1127 11948
Label: 1128 9689
Label: 1139 797
Label: 10671 325
Label: 10703 82515
Label: 10704 6198
Label: 182305693 2506
Label: 182305697 8081
Label: 182305701 3929
Label: 182305705 3306
Label: 182305709 1930
Label: 182305713 266
Label: 312782550 3939
Label: 312782554 5535
Label: 312782558 2800
Label: 312782562 8051
Label: 312782566 4012
Label: 312782570 501
Label: 312782578 1279
Label: 312782582 2134
Label: 312782586 906
Label: 312782590 1516
Label: 312782594 758
Label: 312782598 199
Label: 312782604 2424
Label: 312782608 4303
Label: 312782612 3633
Label: 312782616 4704
Label: 312782620 1139
Label: 312782624 253
Label: 312782632 4294
Label: 312782636 5786
Label: 312782640 578
Label: 312782644 4856
Label: 312782648 2417
Label: 312782652 558
Label: 484682470 22830
Label: 484682508 10003
Label: 484682512 12311
Label: 484682516 68028
Label: 484682520 22800
Label: 484682524 6131
Label: 484682528 460
Label: 496345664 4832
Label: 496345668 8903
Label: 496345672 1815
Label: 526157192 7436
Label: 526157196 2176
Label: 526322264 5
Label: 527696977 44
Label: 549009203 474
Label: 549009211 354
Label: 549009215 5314
Label: 549009219 277
Label: 549009223 1085
Label: 549009227 415
Label: 560581551 2847
Label: 560581559 453
Label: 560581563 1430
Label: 563807435 4014
Label: 563807439 484
Label: 576073699 315
Label: 576073704 3365
Label: 589508447 2922
Label: 589508451 1166
Label: 589508455 1117
Label: 599626923 207
Label: 599626927 306
Label: 606826647 3802
Label: 606826651 1483
Label: 606826655 738
Label: 606826659 686
Label: 606826663 182
Label: 607344830 449
Label: 607344834 3221
Label: 607344838 2454
Label: 607344842 513
Label: 607344846 1905
Label: 607344850 1238
Label: 607344854 261
Label: 607344858 1362
Label: 607344862 640
Label: 614454277 516

I've checked and the values are properly integers, with no slice/volume rescaling.

voxel_vote (after adjusting https://github.com/Mouse-Imaging-Centre/minc-stuffs/blob/master/python/voxel_vote#L32 to uint) produces:

 $ Display ../../Dorr-orig/DSURQE_40micron.mnc -label voxelvote.mnc 
Input ../../Dorr-orig/DSURQE_40micron.mnc
Objects input.
Input voxelvote.mnc
Error: tried to realloc invalid number of elements, 0.
Error: tried to realloc invalid number of elements, 0.
Segmentation fault

A crash on Display, and a broken (wrong location) and "grid-overlayed" output in register

bcdarwin commented 2 years ago

Hmm... a couple questions:

Is ImageMath 3 <out.mnc> MajorityVoting <voting_files> similarly broken?
Did you also alter line 34 in the code?
Can you send me a subset of these files? (Does it happen with only two files with these label sets?)

gdevenyi commented 2 years ago

Is ImageMath 3 MajorityVoting similarly broken?

No, it works with both ANTs, and SimpleITK (which is how I did it in the end, on a big high-mem machine)

Did you also alter line 34 in the code?

Yes. Good question.

Can you send me a subset of these files? (Does it happen with only two files with these label sets?)

I can send you everything, its not that many :) Of course I also want to contribute them back to you, since its my optimized Allen-on-Dorr labelset I made.

https://we.tl/t-NVgbDSakZX

bcdarwin commented 2 years ago

We have found another issue with voxel_vote producing broken labels. Strangely, it appears to have been introduced by pyminc changes between versions 0.52 and 0.54. I intend to investigate further shortly (more for the sake of pyminc than voxel_vote).

bcdarwin commented 2 years ago

According to git bisect the voxel_vote issue (unsure if related) on our data was introduced in https://github.com/Mouse-Imaging-Centre/pyminc/commit/5487a477c6036daa80537ffef4ab578852ec9683.

bcdarwin commented 2 years ago

If you are still interested in this issue and have a chance, could you check if this issue is still present with Pyminc 0.56?

Mouse-Imaging-Centre / minc-stuffs

voxel_vote produces broken voted file for ultra-high res unsigned-int label files #49