mantidproject / mantidimaging

Graphical toolkit for neutron imaging.
https://mantidproject.github.io/mantidimaging
GNU General Public License v3.0
10 stars 6 forks source link

Changed the polyfit to compute function #2223

Open ashmeigh opened 3 weeks ago

ashmeigh commented 3 weeks ago

Issue

Closes #2051

Description

This pull request refactors the polyfit_correlation module to transition from the partial function style to the compute_function approach. This change improves code clarity, maintainability, and performance.

Changes Refactored find_center to use the new compute_function method. Introduced compute_correlation_error to encapsulate core logic for parallel execution. Allocated shared memory for arrays to enhance performance. Updated code to handle shared memory arrays and ensure compatibility.

Testing

Verifiy the refactored find_center function by running unit tests. Ensure that compute_correlation_error correctly computes correlation errors in parallel. Test shared memory allocation and access in a multiprocessing context.

Acceptance Criteria

Ensure all unit tests pass. Verify that the find_center function performs as expected with different image inputs. Check that the compute_function handles shared memory correctly without errors.

samtygier-stfc commented 2 weeks ago

I get the hang when I run locally with python -m pytest -vs mantidimaging/core/rotation/test/polyfit_correlation_test.py -k test_find_center

Also some error

mantidimaging/core/rotation/test/polyfit_correlation_test.py::PolyfitCorrelationTest::test_find_center Exception ignored in: <function SharedArray.__del__ at 0x7760ac343910>
Traceback (most recent call last):
  File "/home/sam/git/mantidimaging/mantidimaging/core/parallel/utility.py", line 155, in __del__
    if self.has_shared_memory:
  File "/home/sam/git/mantidimaging/mantidimaging/core/parallel/utility.py", line 166, in has_shared_memory
    return self._shared_memory is not None
AttributeError: 'SharedArray' object has no attribute '_shared_memory'
Exception ignored in: <function SharedArray.__del__ at 0x711fd2343910>
MikeSullivan7 commented 1 week ago

I get the hang when I run locally with python -m pytest -vs mantidimaging/core/rotation/test/polyfit_correlation_test.py -k test_find_center

Also some error

mantidimaging/core/rotation/test/polyfit_correlation_test.py::PolyfitCorrelationTest::test_find_center Exception ignored in: <function SharedArray.__del__ at 0x7760ac343910>
Traceback (most recent call last):
  File "/home/sam/git/mantidimaging/mantidimaging/core/parallel/utility.py", line 155, in __del__
    if self.has_shared_memory:
  File "/home/sam/git/mantidimaging/mantidimaging/core/parallel/utility.py", line 166, in has_shared_memory
    return self._shared_memory is not None
AttributeError: 'SharedArray' object has no attribute '_shared_memory'
Exception ignored in: <function SharedArray.__del__ at 0x711fd2343910>

This seems to be a Linux specific issue, as I dont get this on Windows. Makes sense with it being related to the Shared Memory.

samtygier-stfc commented 5 days ago

I think the problem here is that SharedArray objects are being passed to the compute function as params where they need to be in using arrays. params should only contain simple types.

This is needed because SharedArrays need to be converted to SharedArrayProxys before being passed to the other processes (see _check_shared_mem_and_get_data). If this does not happen then the other process thinks it owns the SharedArray and therefore tries to deleted it when it goes out of scope.