Manual Testing Muon - Githubissues

koshchii commented 4 months ago

You have been assigned manual testing. The hope is to catch as many problems with the code before release, so it would be great if you can take some time to give a serious test to your assigned area. Thank you!!

The general guide to manual testing:

The tests must be performed on the installer versions of the final release candidate. Not on local compiled code.
Serious errors involving loss of functionality, crashes etc. should be raised as issues with the current release as a milestone and an email sent to the project manager immediately.
Minor and cosmetic issues should be raised as issues against the forthcoming releases.
First try things that should work, then try to break Mantid, e.g. entering invalid values, unexpected characters etc.
Don't spend more than a few hours on the testing as fatigue will kick in.
If you find errors in the documentation, please correct them.
Comment against this ticket the OS environment you are testing against.
Disable Usage reporting before you start testing
Close the this issue once you are done.
Time how long this manual test takes for you to do and leave a comment about it in this issue.

Specific Notes:

See instructions at: https://developer.mantidproject.org/Testing/MuonAnalysis_test_guides/index.html

jclarkeSTFC commented 4 months ago

Not sure if this counts as a bug, you might want it to behave like this if you're entering multiple values, but in the ALC peak fitting bit, if you plot a guess, then change the parameters of the fit function, to get the guess to update you need to remove the guess and add it again.

jclarkeSTFC commented 4 months ago

In the MUSR test, the Add Group option only appears when right clicking in some columns of the table.

GuiMacielPereira commented 4 months ago

Clicking Load Current Run gives and error: Cannot find file: //MUSR/data/autosave.run

jclarkeSTFC commented 4 months ago

In the Transverse Field Asymmetry Test, near the end, I changed the Fitting Type to TF Asymmetry and got the below exception.

Traceback (most recent call last):
  File "C:\MantidNightlyInstall\bin\lib\site-packages\mantidqtinterfaces\Muon\GUI\Common\fitting_widgets\tf_asymmetry_fitting\tf_asymmetry_fitting_presenter.py", line 80, in handle_tf_asymmetry_mode_changed
    self.fitting_mode_changed_notifier.notify_subscribers()
  File "C:\MantidNightlyInstall\bin\lib\site-packages\mantidqt\utils\observer_pattern.py", line 101, in notify_subscribers
    Observable.notify_subscribers(self, *args)
  File "C:\MantidNightlyInstall\bin\lib\site-packages\mantidqt\utils\observer_pattern.py", line 56, in notify_subscribers
    self.thread_safe_update_call(arg, **kwargs)
  File "C:\MantidNightlyInstall\bin\lib\site-packages\mantidqt\utils\qt\qappthreadcall.py", line 69, in __call__
    return self._callable(*args)
  File "C:\MantidNightlyInstall\bin\lib\site-packages\mantidqt\utils\observer_pattern.py", line 60, in _notify_subscribers_impl
    observer.update(self, arg, **kwargs)
  File "C:\MantidNightlyInstall\bin\lib\site-packages\mantidqt\utils\observer_pattern.py", line 84, in update
    self.callback()
  File "C:\MantidNightlyInstall\bin\lib\site-packages\mantidqtinterfaces\Muon\GUI\Common\seq_fitting_tab_widget\seq_fitting_tab_presenter.py", line 81, in handle_selected_workspaces_changed
    self.handle_fit_function_updated()
  File "C:\MantidNightlyInstall\bin\lib\site-packages\mantidqtinterfaces\Muon\GUI\Common\seq_fitting_tab_widget\seq_fitting_tab_presenter.py", line 54, in handle_fit_function_updated
    parameter_values = self._get_fit_function_parameter_values_from_fitting_model()
  File "C:\MantidNightlyInstall\bin\lib\site-packages\mantidqtinterfaces\Muon\GUI\Common\seq_fitting_tab_widget\seq_fitting_tab_presenter.py", line 60, in _get_fit_function_parameter_values_from_fitting_model
    parameter_values = [
  File "C:\MantidNightlyInstall\bin\lib\site-packages\mantidqtinterfaces\Muon\GUI\Common\seq_fitting_tab_widget\seq_fitting_tab_presenter.py", line 61, in <listcomp>
    self.model.get_all_fit_function_parameter_values_for(fit_function)
  File "C:\MantidNightlyInstall\bin\lib\site-packages\mantidqtinterfaces\Muon\GUI\Common\fitting_widgets\tf_asymmetry_fitting\tf_asymmetry_fitting_model.py", line 703, in get_all_fit_function_parameter_values_for
    return self._get_all_fit_function_parameter_values_for_tf_simultaneous_function(fit_function)
  File "C:\MantidNightlyInstall\bin\lib\site-packages\mantidqtinterfaces\Muon\GUI\Common\fitting_widgets\tf_asymmetry_fitting\tf_asymmetry_fitting_model.py", line 722, in _get_all_fit_function_parameter_values_for_tf_simultaneous_function
    normal_domain_function = self._get_normal_fit_function_from(tf_domain_function)
  File "C:\MantidNightlyInstall\bin\lib\site-packages\mantidqtinterfaces\Muon\GUI\Common\fitting_widgets\tf_asymmetry_fitting\tf_asymmetry_fitting_model.py", line 306, in _get_normal_fit_function_from
    return tf_asymmetry_function.getFunction(0).getFunction(1).getFunction(1)
RuntimeError: Function ExpDecayMuon doesn't have children.

I was able to continue afterwards

Edit: See #37392

jclarkeSTFC commented 4 months ago

EMU page, near the end of the Ionic Diffusion Test, clicking External Plot produced this:

jclarkeSTFC commented 4 months ago

EMU page, in the Ionic Diffusion Test section, it says to click Set all to log, but the values I get are not 0, 5, and 10.5, they're more like 0, 20.49, 0. I don't know if this is a problem with the test instructions or the calculation.

Edit: I couldn't reproduce this in 6.9.1 or the nightly

jclarkeSTFC commented 4 months ago

EMU page again, when I tick the Exclude Range option and try to run the fit I get this error:

Edit: I can't reproduce this

jclarkeSTFC commented 4 months ago

EMU page again, I couldn't find the "Model Fitting" tab. I also couldn't see a Results table combo box anywhere on the other tabs either.

robertapplin commented 4 months ago

EMU page again, I couldn't find the "Model Fitting" tab. I also couldn't see a Results table combo box anywhere on the other tabs either.

You need to follow the instruction at the top of this test to enable "Model Fitting" https://developer.mantidproject.org/Testing/MuonAnalysis_test_guides/Muon_EMU.html#superconducting-copper-test

jclarkeSTFC commented 4 months ago

FDA analysis page, FFT Test, when the padding is set to zero the instructions say that "it will have lots of straight lines", but the plot looked pretty similar to when the padding is set to 50.

Edit: See #37393

jclarkeSTFC commented 4 months ago

HIFI page, HIFI Transverse Field Simultaneous Fitting, the section at the end says that chi-squared should be reduced to <10, but for me it was 31.65

jclarkeSTFC commented 4 months ago

HIFI page, HIFI Transverse Field Simultaneous Fitting, the tiled plots at the end look like this:

But apparently they should look like this:

The plot in the top left is quite different.

Edit: See #37394

jclarkeSTFC commented 4 months ago

HIFI page, MultiPeriod Data, the sequential fit fails for a few of the runs, I'm not sure if that's supposed to happen or not:

jclarkeSTFC commented 4 months ago

I couldn't find the data to run the PSI test

robertapplin commented 4 months ago

It seems the HIFI instructions give better results if the fit for 134028 starts with a value of 0.01 for frequency rather than 0. I can change the instructions to mention this

I think we can ignore the "Failed" in the sequential table as the fits produced are still quite good. It just needs more than 500 iterations to converge. I'll add to the instruction to mention this

jclarkeSTFC commented 4 months ago

I've now made separate issues for all the items I found that require it.

mantidproject / mantid

Manual Testing Muon #37326

Specific Notes: