Heuristic evaluation - Auto-complete

[rubinahaddad]

_Note: This interface is dependant on the core device or platform components (ex. keyboards, dropdown rendering etc). This component may require additional testing on platforms such as iOS, Android, Blackberry._

Please perform a heuristic review of the following component: Auto-complete example.

Performing a heuristic evaluation

1. Post a comment on this issue indicating that you will perform a heuristic evaluation of the component. If you see that at least three people have already indicated that they are performing a heuristic evaluation, consider reviewing another component.
2. Use the heuristic template to guide your evaluation.
3. Make sure that you review all responsive states of the component.
4. Once you have completed your evaluation, post the results as a comment on this issue as follows:
   • For each heuristic, create a heading containing the letter of the section, a period and the number of the heuristic (e.g., ## A.1).
   • Below each heading, provide the results of your evaluation of that heuristic. Make sure you specify which responsive state each result applies to. If a result applies to all responsive states, specify that it applies to all responsive states.

[rubinahaddad]

A. Please indicate which devices were tested

Desktop: IE 9

B. Provide status feedback

1. No – When you keyboard into the auto-complete suggestion drop-down list the text input box should change based on what value you are on. This is so the user knows the current status.
2. Yes – When you click into the text input box or start typing, selections start to appear.
3. No – the user’s keyboard selection state should reflect in the text inputbox. The visual cues are also misleading – there is a vertical scroll bar in the auto-complete list, but when you click on it with your mouse the box disappears. There should either never be a scroll bar or the user must be able to use it with a mouse/touch screen.
4. Yes - When the user starts typing or clicks on the text input box they will be able to choose from the list.

C. Match real world conventions

1. Yes – As you start typing you see a selection list which is a common expectation
2. No – There is usually no vertical scroll bar in other auto-completes – users don’t like too many choices at once. Is it better to have a shorter list than a scrollable list? The text in the text input box jumps when you click in the field. When you start typing, and arrow down to a selection, the enter key and tab key should both select your selection (only enter does this)[Note: This should not be a problem if B.1 is implemented]

See http://search.gc.ca for a good example of how it should work. Note: in http://search.gc.ca if you mouse select an option it automatically goes to the search results page. Should this happen? The user might want to add on to the query or change something before searching.

D. Apply standards and ensure consistency

1. No – there is not usually a vertical scroll bar
2. Yes
3. N/A
4. Yes

E. Help users recognize, prevent and recover from errors

1. No – Sometimes when you select an option and then start typing, it positions you at the beginning of the text when it should be at the end of the text therefore you have to erase what you inputted, and re-type it.
2. N/A
3. N/A

F. Design for human interaction

1.a. Yes 1.b. No – See B.1, B.3, C.2

2. Yes
3. N/A

G. Keep it simple

1. No – It displays the full list of selectable items even before the user has started typing. The user has too many choices to choose from initially, it should only display a list when the user has started typing.
2. Yes – simple, but not intuitive
3. N/A

[pcwsmith]

I will review.

[Robin-Lang]

I'll work on this one.

[pcwsmith]

A. Devices, browsers, etc.

IPad, 4th gen., iOS 6.1.3: Safari, Chrome v. 31. Screen resolution 2048 x 1536 px. Reviewed in both landscape and portrait orientation.

Blackberry Curve 9360, OS 7.1, default browser. Screen resolution 480 x 360 px. Landscape orientation only.

B. Provide status feedback

1. Always lets users know about its current status? Yes, it changes when interacted with.
2. Gives immediate, easy to understand feedback? No - the text input box initially behaves like a dropdown picklist. On iPad, when I first tap in the input box, I don't get a cursor. Rather I get a dropdown picklist of predefined options. Only after choosing one of the options am I able to get a cursor and type. This is the case with both Chrome and Safari. (See screenshot below.) Slightly better on the Blackberry - while the text input field still becomes a dropdown when I first place my cursor in it, at least the cursor is there and I can start typing without having to pick an item first.
3. Adequate visual cues to easily complete the component's tasks? No - looks like a text input box, but initially behaves like a dropdown.
4. When hidden, something that allows it to be brought in or out of view? No. while tapping or typing brings up the options, but the picklist runs off the bottom edge of the screen, and I am unable to scroll through the picklist.

C. Match real world conventions

1. Meets common expectations and is easily recognizable? No - as explained above. I am expecting something along the lines of when I am using Google for searching - that I start typing and then receive a list of options to complete my query.
2. Results of user actions are what would be commonly expected? No - even when I am able to start typing, the options that I get in response are confusing. This is because the options presented contain what I have typed rather than start with what I have typed. (See screenshot below.)

D. Apply standards and ensure consistency

1. Industry formatting standards used? No - as explained above. It's a stealth dropdown masquerading as a text input field.
2. Elements used: logical and consistent? No - as explained above.
3. Use of language: consistent, clearly understandable, relevant, concise, current and to the point? Yes, however this is dependent on two pieces of content: the label for the input box, and the list of items that can be picked. Both are clear in the example, but care needs to be taken for real-world implementations of this component.
4. Consistency across screen sizes? Yes

E. Help users recognize, prevent and recover from errors

1. Help to prevent user error? No - due to confusing interactions.
2. Error message simple and informative? N/A
3. Information to fix or correct errors? N/A

F. design for human interaction

1. When using the component: a. Is the layout intuitive? Yes, the initial layout of the component is intuitive. b.are the interactions intuitive? No - as explained above.
2. Is the layout designed to accommodate various input methods and screen sizes? No - in the example, the picklist runs off the bottom edge of the screen, and I am unable to scroll to the end of the picklist.
3. Option to return to task in later session? N/A

G. Keep it simple

1. Only display the information needed at any given point? No - the component displays options before I've even started typing. Should only display relevant choices based on what I type.
2. Overall, are user interactions kept as simple as possible? Yes
3. Does the component use plain language? Yes - see caveat in D.3 above.

[pcwsmith]

Screenshot to support B.2 - on iPad, when the user taps to select the input box, a dropdown list appears instead of a cursor:

[pcwsmith]

Screenshot for C.2 - when the user starts to type, the options presented contain the characters typed rather than starting with what's been typed.

[pcwsmith]

Just read @rubinahaddad's review and agree that the search.gc.ca example is a good starting point.

[rubinahaddad]

@Robin-Lang are you able to complete this by Friday?

[Robin-Lang]

@rubinahaddad yes, will be completed by Friday.

[Robin-Lang]

A. Please indicate which devices, browsers, and operating systems were tested (Desktop/Mobile/Tablet): Desktop: Mac OS 10.9.1, Firefox v26.0, Safari v7.0.1 Mobile: Galaxy Nexus running Android 4.2.1 with Chrome v31.0.1650.59 and Firefox v26.0.1

B. Provide status feedback 1. Does the component always let users know about its current status?

Yes & No. In both desktop browsers and Firefox mobile the vertical text entry cursor appears in the input field (and keyboard appears in FF mobile) and the field changes outline colour to demonstrate that the user may type into the field.

The component is buggy on the Chrome mobile browser in that more often that not, the keyboard does NOT appear at all when the input field is clicked. When keyboard does appear, entering letters does NOT change the state of the dropdown menu at all. I cannot replicate this bug consistently enough to speculate about its source and was not able to evaluate this component in the Chrome mobile browsers completely because of this issue.

In the desktop Safari and mobile Chrome browsers the dropdown menu remains on the screen even when an area outside the dropdown or the input field is clicked. This suggests, incorrectly, that the user is still acting upon the dropdown menu. The dropdown menu disappears when user clicks on other area of screen in both desktop and mobile FF browser.

In all browsers except desktop FF the dropdown menu appears immediately upon clicking into the text input field rather than waiting for the user to enter a letter and begin suggestions from there. Note: this does not happen in desktop Safari when the user navigates to the input field using tab key.

2. Does the component give immediate, easy to understand feedback?

Yes, with the exception of the mobile Chrome bug, the input field displays when it has been clicked on and items on the dropdown menu become highlighted when hovered on or navigated to via arrow keys.

3. Will the user have enough visual cues to easily complete the component’s tasks?

Yes. The text input field has the appropriate visual cues for users to know it is an input field, the dropdown menu is scrollable and menu items highlight when chosen then fill the input field when chosen.

4. If some part of a component is hidden, can users see something that allows them to bring it in and out of view?

Yes & No. In desktop Safari the dropdown menu either displays all relevant items or, if there are too many to display, the list is truncated and only the top half of the lowest item in the menu is visible, inviting the user to scroll down to see the rest of the item and possibly more below it. In desktop FF the dropdown menu remains the same size regardless of number of items, but a side scrollbar is briefly presented to indicate to the user there are more items than displayed. Unfortunately this brief flash of a scrollbar is too brief and users may miss it. This scroll bar appears long enough in mobile FF for users to see it, however when the dropdown menu is too long the upper-most items are hidden behind the URL bar and there is no way to scroll up to them until the list is shortened by entering a letter that calls a shorter list of matching items in dropdown menu.

C. Match real world conventions 1. Do parts of the component match what you see, hear (if any) and touch (if any) meet common expectations and are they easily recognizable?

Yes & No. While the text input field and dropdown menu match convention, in all browsers when the dropdown menu is activated its upper edge occludes the bottom edge of the input field, looking as though it floats above the input field. This is irregular.

2. When users perform actions within the component, are the results they get what they would commonly expect?

No. 1) Entering a letter into an autocomplete text field does not usually return all possible entries containing that letter, but usually just those who start with the entered letter. 2) It is expected that the tab button can bring a user from the text input field into the dropdown menu in desktop browsers, but this is not the case. Instead the user is brought out of the text input field to the ‘view code’ link below. 3) In desktop Safari, after using the backspace key to clear a highlighted and selected menu item, the user then choses to enter another letter, the relevant items appear in the dropdown menu but the user is no longer able to use the arrow key to highlight and select items and must use the cursor.

D. Apply standards and ensure consistency 1. Have industry formatting standards been used consistently throughout?

No. See above, dropdown menu occludes text input field and scroll bar flashes too quickly in desktop FF browser.

2. Is there a logical and consistent usage of elements throughout the component?

Yes & No. Though each browser uses slight variations on displaying hidden dropdown items the user is able to see that more items exist. However, the desktop and mobile FF browsers both remember search history, so when the dropdown menu is activated (pressing down arrow in desktop, clicking the text field in mobile) the first few items are always those previously searched for. This may be an unavoidable inconsistency, I believe this is just standard FF function.

3. Is there a consistent usage of language that is clearly understandable, relevant, concise, current and to the point throughout the component?

N/A.

4. Is the experience consistent across screen sizes?

Yes, the component does not require considerable width, the experience is consistent.

E. Help users recognize, prevent and recover from errors 1. Does the component help to prevent users from making errors whenever possible?

No. When a city name is entered incorrectly there is no ‘did you mean…?’ suggestion in the dropdown menu. Only correctly entered city names will activate menu items.

2. When there is an error, is it simple and informative?

No, entry errors provide no feedback to user.

3. Does it provide information to fix or correct the error?

No. See above regarding ‘did you mean…’ suggestion, incorrectly entered items reduce the dropdown menu size to 0 entries.

F. Design for human interaction 1. When using the component: a. Is the layout intuitive?

Yes, the dropdown menu appears below (desktop) or above (mobile) the text input field as expected.

2. Is the layout designed so that various input methods (touch, keyboard, mouse, gesture) and screen sizes can easily interact with things like input fields, sliders and buttons?

Yes, see last comment. Any previously mentioned interaction issues are not a product of the component’s layout.

3. If the task can't be completed in one session, does the user have an option to save and return to it at a later time?

N/A

G. Keep it simple 1. Does the component display only the information needed by the user at any given point in time?

No. In all browsers but desktop FF the dropdown menu appears as soon as the text input field is clicked. User does not need to see all menu items before they request to with down arrow press.

2. Overall, are user interactions kept as simple as possible?

Yes. Interacting with this component is kept simple enough apart from the issue raised in last comment.

3. Does the component use plain language?

N/A

[rubinahaddad]

Thanks @Robin-Lang, @pcwsmith ! 3 HEs complete so closing this.

[ballantr]

Notes on Bugs and Browser Differences

Devices used

Device	OS	Browsers
Laptop (24 inch Monitor)	Windows 8.1	IE 11.0.3
Laptop (24 inch Monitor)	Windows 8.1	FF 27.0.1
Laptop (24 inch Monitor)	Windows 8.1	Chrome 33.0.1750.146 m
Nexus 7	Android 4.3	Chrome 33.0.1750.136
Nexus 7	Android 4.3	FF 28
iPhone 4	iOS 7	Safari

Desktop

Chrome

A down-pointing arrow supports opening up the whole unfiltered list.

A filtered list pops down once you start typing.
Selection works as expected using:

• Click select;
• [arrow key selection] + [tab]; and
• [arrow key selection] + [Enter]

IE

The full list pops down before you type anything. Also, once you start typing an ‘X’ can be used to clear what has been entered.

It filters correctly as you proceed. Selection works as expected using:

• Click select; and
• [arrow key selection] + [Enter]

This common selector does not work

• [arrow key selection] + [tab]

FF

There is not clear way to open up the whole unfiltered list.

A filtered list pops down once you start typing. The results include an extra (invalid) item

Selection works as expected using:

• Click select;
• [arrow key selection] + [tab]; and
• [arrow key selection] + [Enter]

Tablet

FF

The full list pops UP (usually, but sometimes down) before you type anything. You scroll the list by dragging inside the list. If you drag outside the list the list closes, then reopens when you stop dragging.

It is not possible to scroll to the top item if the list extends above the top of space below the tabs (the top item remains hidden)

The filtered list includes invalid items

Selection works as expected using touch-select

Once you select you get a one-line drop down that remains till you click away.

Chrome

A filtered list pops UP once you start typing. You scroll this list by dragging in the list.

Touch-select works as expected.

Once you have selected, the drop up disappears as expected.

Mobile

Safari

When you touch the field, in rapid succession it

• opens the full list below the field; then
• scrolls the field to the top &
• closes the full list
• displays the typing pad.

A filtered list pops down once you start typing – partly under the keypad if the list is long.

The list of options includes invalid options.

Touch-select works as expected.

You cannot scroll the list, but you can scroll the page (and the list) dragging outside the list (in the narrow space available.

You cannot change your mind about selecting something. Clicking on the background does not cause the list to go away

[ballantr]

Updating the above...

I now see that the filter works as a "Starts with.." filter in some instances (Chrome and IE) and a "Contains..." filter in others (FF & Safari)! So I was wrong to say the lists include "invalid" items. I should have said "unexpected and inconsistent".

[ballantr]

Autocomplete Summary

The widget was tested over the last several months using the browsers and devices below:

Devices and Browsers Used Previously

Device	OS	Browsers
Desktop		IE 8
iPad 4 Screen resolution 2048 x 1536 px. Reviewed in both landscape and portrait orientation	iOS 6.1.3	Safari
iPad 4 Screen resolution 2048 x 1536 px. Reviewed in both landscape and portrait orientation	iOS 6.1.3	Chrome 31
Blackberry Curve 9360, Screen resolution 480 x 360 px. Landscape orientation only	OS 7.1,	default browser.
Desktop	Mac OS 10.9.1	Firefox v26.0
Desktop	Mac OS 10.9.1	Safari v7.0.1
Galaxy Nexus	Android 4.2.1	Chrome v31.0.1650.59
Galaxy Nexus	Android 4.2.1	Firefox v26.0.1

Many issues were discovered in the above browsers, and changes appear to have been made to the widget since. As a result the widget was briefly re-reviewed today using the following:

Devices and Browsers Used Today

Device	OS	Browsers
Laptop (24 inch Monitor)	Windows 8.1	IE 11.0.3
Laptop (24 inch Monitor)	Windows 8.1	FF 27.0.1
Laptop (24 inch Monitor)	Windows 8.1	Chrome 33.0.1750.146 m
Nexus 7	Android 4.3	Chrome 33.0.1750.136
Nexus 7	Android 4.3	FF 28
iPhone 4	iOS 7	Safari

Notes:

Given that the initial reviews are now out of date, this “Summary” is really just a selection of points that still seem to apply, plus a summary of my observations in a quick review today. It is not clear which of the original bugs reported remain valid. Only the most obvious and problematic are reported here.

Bugs

1. The filter works as a "Starts with.." filter in some instances (Chrome and IE) and a "Contains..." filter in others (FF & Safari)! It should work as a “Starts with...” in every instance. (Maximum priority for fixing.)
2. It is not possible to scroll to the top item if the list extends above the top of space below the tabs in FF on the Nexus Tablet (the top item remains hidden). (Medium priority - not because a lot of folks will struggle with it, but because there is a possibility of serious error if a user concludes that an item is not in a list because of this.)
3. pcwsmith reports “On iPad, when I first tap in the input box, I don't get a cursor. Rather I get a dropdown picklist of predefined options. (Low to medium priority for fixing, in my opinion.)
4. On the iPad/Safari, when the dropdown menu is activated its upper edge occludes the bottom edge of the input field, looking as though it floats above the input field. Robin-Lang found this on other browsers as well. (Low priority)
5. You cannot change your mind about selecting something on the iPhone/Safari. Clicking on the background does not cause the list to go away until you make a selection. (Low priority)

Opportunities for improving

The High and Medium Bug priorities supersede the below. The below are not barriers to using the widget.

1) There is no highlighting of the found letters in the filtered list. This is a convenience that makes these tools more obvious, and so more usable. (Low to Medium priority) This is particularly important if you decide to go with a “includes” search strategy (Medium to High priority)

2) One reviewer correctly observed that the control did not include ‘did you mean…?’ capabilities (that predict even in the case of spelling errors). These capabilities are now becoming common in the search tools we all use. (Low priority for fixing)

[rubinahaddad]

@pjackson28 Because this is a polyfill, what can we actually change and improve?

[pjackson28]

@rubinahaddad Nothing in WET as the only browser that uses this polyfill is IE8. Any issues with what the HTML5 spec describes for datalist would need to be filed with the W3C. Any issues with a browser's specific implementation would need to be filed with the relevant browser vendor.

Note that it is expected that there will be differences in the autocomplete from browser to browser just like with any other native UI component.

[pjackson28]

@rubinahaddad It may be possible to override some aspects of native support with CSS, but I would expect that what we could do would be very limited.