bug: qt3scan - optimize function doesn't handle all exceptions

[gadamc]

While there is a try/catch around the optimization function, it only catches nidaq errors. We should change this to handle all errors (since the piezo stage controller may also throw an error).

[gadamc]

In particular, it seems that if a scipy optimization fails with an error, the function does not return and all the buttons remain greyed out

[gadamc]

Suggested changes

https://github.com/qt3uw/qt3-utils/blob/main/src/qt3utils/datagenerators/piezoscanner.py#L177-L190

change to:

        optimal_position = axis_vals[np.argmax(count_rates)]
        coeff = None
        params = [np.max(count_rates), optimal_position, 1.0, np.min(count_rates)]
        bounds = (len(params)*tuple((0,)), len(params)*tuple((np.inf,)))
        try:
            coeff, var_matrix = scipy.optimize.curve_fit(gauss, axis_vals, count_rates, p0=params, bounds=bounds)
            optimal_position = coeff[1]
            # ensure that the optimal position is within the scan range
            optimal_position = np.max([min_val, optimal_position])
            optimal_position = np.min([max_val, optimal_position])
        except (ValueError, RuntimeError, OptimizeWarning) as e:
            logger.warning(e) #this sends warning to logger
            raise(e)   # optionally, we could throw the error again after logging, and require that the caller handles the Exception
        finally:
            return count_rates, axis_vals, optimal_position, coeff

https://github.com/qt3uw/qt3-utils/blob/main/src/applications/piezoscan.py#L489-L507

change to

        try:
            data, axis_vals, opt_pos, coeff = self.counter_scanner.optimize_position(axis,
                                                                           central,
                                                                           range,
                                                                           step_size)

        except nidaqmx.errors.DaqError as e:
            logger.error(e)
            logger.error('NIDAQ Error. Check for other applications using resources. If not, you may need to restart the application.')

        except (ValueError, RuntimeError, OptimizeWarning) as e:
            logger.error(e)

        finally: 
            self.optimized_position[axis] = opt_pos
            self.counter_scanner.stage_controller.go_to_position(**{axis:opt_pos})
            self.view.show_optimization_plot(f'Optimize {axis}',
                                             central,
                                             self.optimized_position[axis],
                                             axis_vals,
                                             data,
                                             coeff)
            self.view.sidepanel.update_go_to_position(**{axis:self.optimized_position[axis]})

            self.view.sidepanel.startButton['state'] = 'normal'
            self.view.sidepanel.stopButton['state'] = 'normal'
            self.view.sidepanel.go_to_z_button['state'] = 'normal'
            self.view.sidepanel.gotoButton['state'] = 'normal'
            self.view.sidepanel.saveScanButton['state'] = 'normal'
            self.view.sidepanel.popOutScanButton['state'] = 'normal'

            self.view.sidepanel.optimize_x_button['state'] = 'normal'
            self.view.sidepanel.optimize_y_button['state'] = 'normal'
            self.view.sidepanel.optimize_z_button['state'] = 'normal'

Also, could use from tkinter import messagebox to display error in window instead of on console. (Thanks to @cmordi for discovering that tool in tkinter..)

        except nidaqmx.errors.DaqError as e:
            error_msg = f"{e}\n NIDAQ Error. Check for other applications using resources. If not, you may need to restart the application."
            messagebox.showerror("Error", error_msg)

        except Exception as e:
            messagebox.showerror("Error", e)

With the solution above, the piezo actuator will be moved to a new optimal position even if the scipy optimize function throws. The new optimal position will be the position with the maximum count rates. This, of course, could be changed. If it is desired to remain in the original position when scipy fails, then perhaps

change

https://github.com/qt3uw/qt3-utils/blob/main/src/qt3utils/datagenerators/piezoscanner.py#L162-L190

to

    def optimize_position(self, axis, center_position, width = 2, step_size = 0.25, return_to_initial_on_failure = True):
        """  (docstring omitted for brevity)
        """
        min_val = center_position - width
        max_val = center_position + width
        if self.stage_controller:
            min_val = np.max([min_val, self.stage_controller.minimum_allowed_position])
            max_val = np.min([max_val, self.stage_controller.maximum_allowed_position])
        else:
            min_val = np.max([min_val, 0.0])
            max_val = np.min([max_val, 80.0])

        initial_position = self.stage_controller.get_current_position()
        initial_position = {'x': initial_position[0], 'y': initial_position[1], 'z': initial_position[2]}

        self.start()
        raw_counts = self.scan_axis(axis, min_val, max_val, step_size)
        self.stop()
        axis_vals = np.arange(min_val, max_val, step_size)
        count_rates = [self.sample_count_rate(count) for count in raw_counts]

        optimal_position = axis_vals[np.argmax(count_rates)]
        coeff = None
        params = [np.max(count_rates), optimal_position, 1.0, np.min(count_rates)]
        bounds = (len(params)*tuple((0,)), len(params)*tuple((np.inf,)))
        try:
            coeff, var_matrix = scipy.optimize.curve_fit(gauss, axis_vals, count_rates, p0=params, bounds=bounds)
            optimal_position = coeff[1]
            # ensure that the optimal position is within the scan range
            optimal_position = np.max([min_val, optimal_position])
            optimal_position = np.min([max_val, optimal_position])
        except (ValueError, RuntimeError, OptimizeWarning) as e:
            if return_to_initial_on_failure:
                optimal_position = initial_position[axis]
            logger.warning(e) #this sends warning to logger
            raise(e)   # optionally, we could throw the error again after logging, and require that the caller handles the Exception
        finally:
            return count_rates, axis_vals, optimal_position, coeff

and then call

return_to_initial = True  # or False?  could be configured as a GUI check box to control behavior

data, axis_vals, opt_pos, coeff = self.counter_scanner.optimize_position(axis,
                                                                           central,
                                                                           range,
                                                                           step_size, 
                                                                           return_to_initial)