stat.py performance issues

[zihuaihuai]

permuted the previous release. large number of computation amount persons. for loops 2 heavy -> np matrix calc eiggn func should work line 219

[jordandekraker]

This should be testable using the data form OSF Storage (for example, 2-3 random maps in /OSF Storage /HippoMaps-initializationMaps /Dataset-MICs) after loading them and stacking on axis=1

[jordandekraker]

import nibabel as nib
import numpy as np
import hippomaps as hm

map1 = nib.load("YOUR MAP").darrays[0].data
map2 = nib.load("YOUR MAP2").darrays[0].data
map_set = np.stack((map1.reshape(-1,1),map2.reshape(-1,1)), axis=1)

hm.stats.contextualize2D(map_set)

[zihuaihuai]

import nibabel as nib
import numpy as np
import hippomaps as hm

map1 = nib.load("YOUR MAP").darrays[0].data
map2 = nib.load("YOUR MAP2").darrays[0].data
map_set = np.stack((map1.reshape(-1,1),map2.reshape(-1,1)), axis=1)

hm.stats.contextualize2D(map_set)

In this branch there is still bug when i tried to run the test code here is the bug info

 hm.stats.contextualize2D(map_set)
  File "/Users/enningyang/CodeProj/hippomaps/hippomaps/stats.py", line 251, in contextualize2D
    taskMapsresamp[:,t],_,_ = hippomaps.utils.density_interp(hippomaps.config.get_label_from_nV(taskMaps.shape[0]),'0p5mm',taskMaps[:,t], label='hipp')
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/enningyang/CodeProj/hippomaps/hippomaps/utils.py", line 229, in density_interp
    raise ValueError("results: indensity must be one of %r." % VALID_STATUS)
ValueError: results: indensity must be one of {'unfoldiso', '0p5mm', '1mm', '2mm'}.

To replicate the bug, just simply runpython hippomaps/test_stats.py

[jordandekraker]

sorry about that, there was still a big in my quickfix. It should be working new with 995008c, please pull that to your dev branch. Thanks!

[zihuaihuai]

after merging it now i have this new bug:

Traceback (most recent call last):
  File "/Users/enningyang/CodeProj/hippomaps/hippomaps/test_stats.py", line 11, in <module>
    hm.stats.contextualize2D(map_set)
  File "/Users/enningyang/CodeProj/hippomaps/hippomaps/stats.py", line 251, in contextualize2D
    taskMapsresamp[:,t],_,_ = hippomaps.utils.density_interp(hippomaps.config.get_label_from_nV(taskMaps.shape[0])[1],'0p5mm',taskMaps[:,t], label='hipp')
    ~~~~~~~~~~~~~~^^^^^
ValueError: could not broadcast input array from shape (7262,1) into shape (7262,)

[jordandekraker]

ahh sorry I'm so lazy about testing. Luckily this should provide a tiny speed boost while also fixing that bug: a859a0e

[jordandekraker]

missed another quick thing! 427a33e

[zihuaihuai]

here is a new one

  File "/Users/enningyang/CodeProj/hippomaps/hippomaps/test_stats.py", line 11, in <module>
    hm.stats.contextualize2D(map_set)
  File "/Users/enningyang/CodeProj/hippomaps/hippomaps/stats.py", line 258, in contextualize2D
    eigen = SurfaceEigenstrapping(surface=f"{resourcesdir}/canonical_surfs/tpl-avg_space-canonical_den-0p5mm_label-hipp_midthickness.surf.gii", 
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/enningyang/opt/anaconda3/envs/hippomap/lib/python3.12/site-packages/eigenstrapping/base.py", line 419, in __init__
    self.evals, self.emodes = calc_surface_eigenmodes(self.surface_file, self.medial_wall, save_cut=True, num_modes=self.num_modes, use_cholmod=self.cholmod)
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/enningyang/opt/anaconda3/envs/hippomap/lib/python3.12/site-packages/eigenstrapping/geometry.py", line 795, in calc_surface_eigenmodes
    surface_cut = _surface_mask(surface_orig, mask)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/enningyang/opt/anaconda3/envs/hippomap/lib/python3.12/site-packages/eigenstrapping/geometry.py", line 845, in _surface_mask
    return _surface_selection(surf, mask, low=1, upp=1, use_cell=use_cell)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/enningyang/opt/anaconda3/envs/hippomap/lib/python3.12/site-packages/eigenstrapping/geometry.py", line 884, in _surface_selection
    raise ValueError('Arrays has more than one dimension.')
ValueError: Arrays has more than one dimension.

i would suggest jump into this branch and do some testing lol

[jordandekraker]

My PC is on the fritz, someone seems to be using all my cores for a remote job :angry: So i haven't tested this but,

Pretty sure this is the issue though - the input had an extra dimension 1b19465

[zihuaihuai]

cool it is running！