Open ashmeigh opened 3 weeks ago
I get the hang when I run locally with
python -m pytest -vs mantidimaging/core/rotation/test/polyfit_correlation_test.py -k test_find_center
Also some error
mantidimaging/core/rotation/test/polyfit_correlation_test.py::PolyfitCorrelationTest::test_find_center Exception ignored in: <function SharedArray.__del__ at 0x7760ac343910>
Traceback (most recent call last):
File "/home/sam/git/mantidimaging/mantidimaging/core/parallel/utility.py", line 155, in __del__
if self.has_shared_memory:
File "/home/sam/git/mantidimaging/mantidimaging/core/parallel/utility.py", line 166, in has_shared_memory
return self._shared_memory is not None
AttributeError: 'SharedArray' object has no attribute '_shared_memory'
Exception ignored in: <function SharedArray.__del__ at 0x711fd2343910>
I get the hang when I run locally with
python -m pytest -vs mantidimaging/core/rotation/test/polyfit_correlation_test.py -k test_find_center
Also some error
mantidimaging/core/rotation/test/polyfit_correlation_test.py::PolyfitCorrelationTest::test_find_center Exception ignored in: <function SharedArray.__del__ at 0x7760ac343910> Traceback (most recent call last): File "/home/sam/git/mantidimaging/mantidimaging/core/parallel/utility.py", line 155, in __del__ if self.has_shared_memory: File "/home/sam/git/mantidimaging/mantidimaging/core/parallel/utility.py", line 166, in has_shared_memory return self._shared_memory is not None AttributeError: 'SharedArray' object has no attribute '_shared_memory' Exception ignored in: <function SharedArray.__del__ at 0x711fd2343910>
This seems to be a Linux specific issue, as I dont get this on Windows. Makes sense with it being related to the Shared Memory.
I think the problem here is that SharedArray
objects are being passed to the compute function as params
where they need to be in using arrays
. params
should only contain simple types.
This is needed because SharedArray
s need to be converted to SharedArrayProxy
s before being passed to the other processes (see _check_shared_mem_and_get_data
). If this does not happen then the other process thinks it owns the SharedArray
and therefore tries to deleted it when it goes out of scope.
Issue
Closes #2051
Description
This pull request refactors the polyfit_correlation module to transition from the partial function style to the compute_function approach. This change improves code clarity, maintainability, and performance.
Changes Refactored find_center to use the new compute_function method. Introduced compute_correlation_error to encapsulate core logic for parallel execution. Allocated shared memory for arrays to enhance performance. Updated code to handle shared memory arrays and ensure compatibility.
Testing
Verifiy the refactored find_center function by running unit tests. Ensure that compute_correlation_error correctly computes correlation errors in parallel. Test shared memory allocation and access in a multiprocessing context.
Acceptance Criteria
Ensure all unit tests pass. Verify that the find_center function performs as expected with different image inputs. Check that the compute_function handles shared memory correctly without errors.