The tests were aimed to capture pain points for beginners. Also, the process itself should create a usability test culture in Inkscape community.
Requirements
Environment
Where the tests should run? The tests should all run in the last release, 1.0.1.
Preparation: The icon set and the theme should be the default one.
Instructions to get this interface: Open Edit > Preferences (or Ctrl + K), go to Interface > Theme. Choose Adwaita for Theme and hicolor for icons.
User profile
Users with no previous relevant experience with Inkcape. However, it would be nice to have a balance with users experienced with other drawing software (i.e. Adobe Illustrator)
Test design
Tasks
Change the application theme.
Change the background color.
Draw the following image:
Draw the following image:
Export the drawing as a PNG image.
Scenarios
I haven't developed scenarios for my tests. However, this is far from ideal, since it could create artificial results, away from a natural workflow in the software.
Results
Task 1
5 of 5 participants succeeded.
The problem of this task was to find where the option was, changing it was easy.
What went well?
After getting to the right menu, changing the theme was easy and they happy that it didn't require a restart.
What were the challenges?
People clearly were expecting to have something at View menu. If the list of Interface's submenus on the Preference window appeared automatically, probably it would be easier to spot the Theme submenu.
Task 2
4 of 5 participants succeeded.
Similarly as in the task above, wasn't very intuitive to find where the option was, but after getting there, it was easy to complete the task.
What went well?
When people opened the right window, it was easy to find the right option there.
What were the challenges?
People went back to Preferences > Interface, which is a nice place to have this option to be honest. The expected behavior for some subjects was that only the background color would change, while the canvas would remain white (I think this can be seen in Adobe software). When the background color gets darker, the border (in default color) blends with it and it's harder to distinguish between the two areas.
Task 3
5 of 5 participants succeeded.
This test should show how is using the software itself, since it's a drawing exercise. It seems that the tools are in the right place, but using them for something slight more advanced seems hard.
What went well?
The "draw square" tool itself was easy to find and the more experienced users could easily use well know keyboard shortcuts to make it symmetric.
What were the challenges?
The double Fill/Stroke sections on top-right and bottom-left need some rework in order to be clear what they are for, since they do different things and the user needs to memorize what each one does. As for the border, the initial size should be either 0 or some value that the person can see it without zooming. We also need to figure out a way to get the Width option more visible appealing than the Dashes one, since this is what people click first for changing the size. Alpha component vs Object Opacity is also trick to get in the first trial.
Task 4
4 of 5 participants succeeded.
Again, the tool for drawing was quite easy, but the more complex step (applying the linear gradient as fill) was a big step. The icon set also played an important role in this tests, since some themes don't display the pentagon icon behind the star icon on the "Draw stars and polygons" tools.
What went well?
Using the "Draw polygon" tool and selection the number of vertices was trivial.
What were the challenges?
For the top menu (where one can change the number of corners) there might be a way to make it more visible. The pain point here is clearly the gradient menu, since the Add button doesn't add something on the selected image and the action that Edit does is not very showy. The fact that when you click on some of the points of the gradient (the small circle/square) the selection on Fill moves from Gradient to Flat color can cause some confusion as well.
Task 5
3 of 5 participants succeeded.
The success of this task was directly related to how many menus the subject had open before trying to Export the PNG.
What went well?
The menu is good to use, the buttons and its purposes are clear. If there are no others menus opened on the right side, it will be noticed for sure.
What were the challenges?
Seeing the menu. If the right side is already polluted, is very likely that it will remain unnoticed, and the user will think that nothing happened.
See this issue on Inkscape repository
Objective
The tests were aimed to capture pain points for beginners. Also, the process itself should create a usability test culture in Inkscape community.
Requirements
Environment
User profile
Users with no previous relevant experience with Inkcape. However, it would be nice to have a balance with users experienced with other drawing software (i.e. Adobe Illustrator)
Test design
Tasks
Scenarios
I haven't developed scenarios for my tests. However, this is far from ideal, since it could create artificial results, away from a natural workflow in the software.
Results
Task 1
5 of 5 participants succeeded.
The problem of this task was to find where the option was, changing it was easy.
What went well?
After getting to the right menu, changing the theme was easy and they happy that it didn't require a restart.
What were the challenges?
People clearly were expecting to have something at View menu. If the list of Interface's submenus on the Preference window appeared automatically, probably it would be easier to spot the Theme submenu.
Task 2
4 of 5 participants succeeded.
Similarly as in the task above, wasn't very intuitive to find where the option was, but after getting there, it was easy to complete the task.
What went well?
When people opened the right window, it was easy to find the right option there.
What were the challenges?
People went back to Preferences > Interface, which is a nice place to have this option to be honest. The expected behavior for some subjects was that only the background color would change, while the canvas would remain white (I think this can be seen in Adobe software). When the background color gets darker, the border (in default color) blends with it and it's harder to distinguish between the two areas.
Task 3
5 of 5 participants succeeded.
This test should show how is using the software itself, since it's a drawing exercise. It seems that the tools are in the right place, but using them for something slight more advanced seems hard.
What went well?
The "draw square" tool itself was easy to find and the more experienced users could easily use well know keyboard shortcuts to make it symmetric.
What were the challenges?
The double Fill/Stroke sections on top-right and bottom-left need some rework in order to be clear what they are for, since they do different things and the user needs to memorize what each one does. As for the border, the initial size should be either 0 or some value that the person can see it without zooming. We also need to figure out a way to get the Width option more visible appealing than the Dashes one, since this is what people click first for changing the size. Alpha component vs Object Opacity is also trick to get in the first trial.
Task 4
4 of 5 participants succeeded.
Again, the tool for drawing was quite easy, but the more complex step (applying the linear gradient as fill) was a big step. The icon set also played an important role in this tests, since some themes don't display the pentagon icon behind the star icon on the "Draw stars and polygons" tools.
What went well?
Using the "Draw polygon" tool and selection the number of vertices was trivial.
What were the challenges?
For the top menu (where one can change the number of corners) there might be a way to make it more visible. The pain point here is clearly the gradient menu, since the Add button doesn't add something on the selected image and the action that Edit does is not very showy. The fact that when you click on some of the points of the gradient (the small circle/square) the selection on Fill moves from Gradient to Flat color can cause some confusion as well.
Task 5
3 of 5 participants succeeded.
The success of this task was directly related to how many menus the subject had open before trying to Export the PNG.
What went well?
The menu is good to use, the buttons and its purposes are clear. If there are no others menus opened on the right side, it will be noticed for sure.
What were the challenges?
Seeing the menu. If the right side is already polluted, is very likely that it will remain unnoticed, and the user will think that nothing happened.