Manual Testing Engineering Diffraction

sf1919 commented 1 year ago

You have been assigned manual testing. The hope is to catch as many problems with the code before release, so it would be great if you can take some time to give a serious test to your assigned area. Thank you!!

The general guide to manual testing:

The tests must be performed on the installer versions of the final release candidate. Not on local compiled code.
Serious errors involving loss of functionality, crashes etc. should be raised as issues with the current release as a milestone and an email sent to the project manager immediately.
Minor and cosmetic issues should be raised as issues against the forthcoming releases.
First try things that should work, then try to break Mantid, e.g. entering invalid values, unexpected characters etc.
Don't spend more than a few hours on the testing as fatigue will kick in.
If you find errors in the documentation, please correct them.
Comment against this ticket the OS environment you are testing against.
Close the this issue once you are done.

Specific Notes:

https://developer.mantidproject.org/Testing/EngineeringDiffraction/EngineeringDiffractionTestGuide.html

DanielMurphy22 commented 1 year ago

https://github.com/mantidproject/mantid/issues/35052 There are no GSASII tab instructions.

DanielMurphy22 commented 1 year ago

https://github.com/mantidproject/mantid/issues/35053 A little irritating that while typing in a run number box the file searching hasn\t begun, which means the Focus or calibration won't run as the file search hasn't completed. I think by force of habit I click in a blank part of the interface to start the file searching. Not an urgent bug but not very user friendly.

DanielMurphy22 commented 1 year ago

https://github.com/mantidproject/mantid/issues/35053 It was also found that retyping in the run numbers in the sample and vanadium inputs on the focus tab give odd asterisk behaviour (shown when it shouldn't and not shown when it should sometimes). Try changing a run number and see that the asterisk doesn't appear.

DanielMurphy22 commented 1 year ago

https://github.com/mantidproject/mantid/issues/35056 separate to the interface, we found a hard crash where Workbench closes when running the load dialog. If you type in 305738 (doesn't matter what instrument in settings) and click run and then close in the algorithm dialog. This crashes workbench pretty reliably on Windows and Ubuntu.

DanielMurphy22 commented 1 year ago

https://github.com/mantidproject/mantid/issues/35052 Test 2 fix indent of point 2. Test 3, points 8+9 say to repeat steps after telling tester to change region of interest. Test 7 evaluate function kept not succeeding for us and I think this section of the instruction should be a little clearer what is expected. Generally the instructions are descriptive but a little dense for testers unfamiliar with the interface so maybe making clear what are instructions and extra info would be valuable. Test 8 is particularly hard to follow and we should descibe the relevant logs to select.

DanielMurphy22 commented 1 year ago

https://github.com/mantidproject/mantid/issues/35054 If focused data already loaded into the fitting tab, selecting more sample logs to load in the settings doesn't get applied (even after loading the focused data again) until you have closed and reopened the interface.

Steps:

Open EngDiff interface
In the fitting tab, load a focused file and check what sample logs are output in the ADS
In the EngDiff settings tick another sample log to be output.
Click apply and ok
In the fitting tab, reload the focused data
The new sample log you asked to be output was not!
Close the interface, and reopen
Go to the fitting tab and load focused data
The new sample log is now output!

gui-co commented 1 year ago

https://github.com/mantidproject/mantid/issues/35055 In the "Calibration" tab, if a random file (I tested with a text file) is provided in the "Load Existing Calibration" path and "Load" is pressed we get a mantid crying dialog.

gui-co commented 1 year ago

In the browse dialog of the focused data of the "Fitting" tab: the file type option is not working properly. If "All File(s)" is selected, only nxs files are displayed.

DanielMurphy22 commented 1 year ago

separate to the interface, we found a hard crash where Workbench closes when running the load dialog. If you type in 305738 (doesn't matter what instrument in settings) and click run and then close in the algorithm dialog. This crashes workbench pretty reliably on Windows and Ubuntu.

@robertapplin @MohamedAlmaki Can either of you reproduce this crash? I'm struggling on Ubuntu at the moment. I wonder if this was just an archive problem at the time?

DanielMurphy22 commented 1 year ago

In the browse dialog of the focused data of the "Fitting" tab: the file type option is not working properly. If "All File(s)" is selected, only nxs files are displayed.

@gui-co I'm not able to reproduce this problem. Are there other files in that folder? Are you on Ubuntu 18.04 or another OS? Screenshot 2023-01-20 at 16 22 29

robertapplin commented 1 year ago

@robertapplin @MohamedAlmaki Can either of you reproduce this crash? I'm struggling on Ubuntu at the moment. I wonder if this was just an archive problem at the time?

Its not as easy to reproduce as previously, but I've managed to still get it. I tried:

Mantid v6.5 on Windows:

I entered 305738, clicked Run and then close.
The algorithm dialog closed.
I waited for about a minute and did nothing - Mantid then crashed

Mantid nightly on Ubuntu 22:

I entered 305738, clicked Run and then close.
Mantid crashed immediately.

In both cases I had to make sure the instrument was ENGINX and that the Manage User directories search directories was empty, and then restart mantid when they have been emptied.

Seeing as I can reproduce, I'd be happy to pick up the issue? https://github.com/mantidproject/mantid/issues/35056

DanielMurphy22 commented 1 year ago

Thanks for this @robertapplin Please raise a separate issue for this. Can you check if the instrument must be set to Enginx?? If this is the only instrument we get the crash for then we can be more targeted in our approach!

MohamedAlmaki commented 1 year ago

@DanielMurphy22 I was able to reproduce it on my side. The crash happens after I press close and wait for around 30 seconds. The crash happens with different kinds of instruments, not just Enginx. I tried 305738 with ENGINS, 82301 with ALF, and 74044 with SANS and they all crashed mantid.

gui-co commented 1 year ago

In the browse dialog of the focused data of the "Fitting" tab: the file type option is not working properly. If "All File(s)" is selected, only nxs files are displayed.

@gui-co I'm not able to reproduce this problem. Are there other files in that folder? Are you on Ubuntu 18.04 or another OS?

I looked at this today. It seems that the filename has to contain "TOF" to be displayed in the file browser.

DanielMurphy22 commented 1 year ago

In the browse dialog of the focused data of the "Fitting" tab: the file type option is not working properly. If "All File(s)" is selected, only nxs files are displayed.

@gui-co I'm not able to reproduce this problem. Are there other files in that folder? Are you on Ubuntu 18.04 or another OS?

I looked at this today. It seems that the filename has to contain "TOF" to be displayed in the file browser.

Ah yes! That is intended! TOF is the default "Browse Filter" set just below the filefinder box. This can be set to show only TOF, dSpacing or all files. I'll update the testing instructions to make this clearer!

mantidproject / mantid

Manual Testing Engineering Diffraction #34949

Specific Notes: