Add help text and (?) icon popups or both to the search pages

[jon-freed]

Problem: Inexperienced users do not know what to put into each search field.

1. The current Advanced Query Techniques page is a separate page, so users have to open it separately to be able to reference it.
2. Search fields that have a mostly defined list of valid values, like "Nature of Suit", are not identified as such.
3. Search fields with values that usually follow a specific format, like "Docket Number", do not have any information to help the user provide values in that format.

Proposed solution: The CL search pages should have some level of help text for each search field.

1. Phase 1: Create a single source of truth for the search fields' short and long help text. While doing so, consider:
   1. The Advanced Query Techniques page's field descriptions, which come from the cl / simple_pages / templates / includes / available_fields.html code (via the cl / simple_pages / templates / advanced_search.html code).
   2. The search field "help text" embedded within cl / search / model.py code. (That text does not appear to be used or displayed anywhere. For example, the text does not match what is in the Advanced Query Techniques page's field descriptions.)
2. Phase 2: Expose the search field's help text in the UI.
   1. Each search field should have basic help text adjacent to it or within an adjacent (?) icon popup or both.

Anybody have an example they want the CL search page's help text to resemble?

Resources:

• Web Content Accessibility Guidelines (WCAG) tutorial regarding forms
• UX StackExchange question about input form help text

Notes:

1. Search form code includes what is in cl / search / forms.py

[mlissner]

This is a great idea, and probably long overdue. A couple other thoughts:

• Date fields are actually super flexible wrt what you can put in them, but discoverability of this is is pretty terrible. For example, the placeholder says to do: YYYY-MM-DD, but it'll take just about any format. You can even do fun things like YYYY or YYYY-MM, and it'll figure that out.

• Date fields need little calendar drop down things. I didn't do them back in the day because browsers were promising they'd have a native solution soon. I think they still don't, so I guess it's time to do a JS thing if we want to.

• We've got some examples of the (?) technique here and there in the UI. Check out a visualization page for example. I believe these are touch, mouse, and accessibility friendly.

• We'll want to implement this both on the search homepages and on the search results pages.

As to how to avoid duplicating the search help across the Advanced Page and this, you might give a read over the Django templates docs, but I think there are two ways to approach this:

1. We store the strings in a python variable of some kind and then pass that to the various pages. Something mapping fields to help, like:

   search_help {
     'attorney': 'Some useful attorney tips.',
   }

   Then in the template you can just do:

   {{ search_help.attorney }}

2. Another way to do this would be to use django template includes. But I don't love this because we'd have to create dozens of includes, one per field, and that'll get awful.

3. This might be possible to add to the form, maybe as part of the label? Not sure. That'd take some tinkering and could get ugly. In any case, it'd be hard to get this value on the advanced search page when you wanted it (probably).

Anyway, this would be a nice improvement to search. Yep.

[jon-freed]

The current consensus (after some Slack chat) seems to be that it would be nice if the field-level help text could be added to the existing model fields in the file https://github.com/freelawproject/courtlistener/blob/master/cl/search/models.py.

There seems to be consensus that there are different levels of information for the fields, e.g. developer level information (which is currently in those model fields' "help_text" attribute) and user level information (of the sort that might appear in (?) icon popups and on the search "Advanced Techniques" web page).

We can't just add another attribute to each field, e.g. "user_help", because Field __init__ won't accept such additions. (It says "unexpected keyword".)

So, if we want to keep each field's information together in that model.py file, then it appears there are at least two options:

1. Do custom fields. See the HandField example and its additional "description" attribute at https://docs.djangoproject.com/en/2.0/howto/custom-model-fields/. However, that seems to get really ugly really fast because of how it can affect migrations (as is described about halfway down that page).

2. Store the multiple levels of help information within the existing "help_text" attribute. Then, when displaying the help text, just show and hide what is appropriate. The help_text could be html with different span elements for each kind of help. Using html in "help_text" is already contemplated within the Django help. See https://docs.djangoproject.com/en/2.0/ref/models/fields/#django.db.models.Field.help_text

I'm favoring the second option, but I wanted to put this out here before I start moving text around. As an example, here is what the model field code currently looks like for docket number:

    docket_number = fields.CharNullField(
        help_text="The docket numbers of a case, can be consolidated and "
                  "quite long",
        max_length=5000,  # was 50, 100, 300, 1000
        blank=True,
        null=True,
        db_index=True,
    )

And here is what it could look like under the second option:

    docket_number = fields.CharNullField(
        help_text='<span helptype="dev">The docket numbers of a case, can be consolidated and quite long</span>'
                  '<span helptype="user">The docket number for a case.</span>',
        max_length=5000,  # was 50, 100, 300, 1000
        blank=True,
        null=True,
        db_index=True,
    )

Thoughts?

[mlissner]

I agree, the first option just isn't worth it.

The second option is problematic too though, because different libraries rely on the help_text being a string not a list. The biggest example that comes to mind is the Django Rest Framework, which uses the help_text field to generate HTTP OPTIONS responses. You can see an example of this if you go here, then click the big button at the top that says "OPTIONS":

https://www.courtlistener.com/api/rest/v3/courts/

I haven't looked at the Django REST Framework, but I bet that it'd do bad things if we tweaked this field to make it a list.

Soooo....I wonder what other options we have. It may just be that duplicating the explanations is the best option. Or we could just make a dict of these descriptions and pass it to each template that we want to use. Not a great option, and it adds complexity, but I'm not sure what else we've got.

[jon-freed]

Three more options:

1. Set up a data structure at the top of the model.py file that contains all of the fields and all of the different types of help text for each field. Then, for each model field's help_text attribute, just reference the appropriate help_text in that data structure at the top. There is a precedent for this. Currently, the model.py file has a SOURCES tuple that is then referenced within OpinionCluster.source.

2. The same as 1, except that for readability, we don't have a data structure at the top of the model.py file, and we instead have a dictionary for each model field immediately above each model field in the code. There is somewhat of a precedent for this as well. See the Docket class, its SOURCE_CHOICES attribute (line 75), and the reference within source (see line 88) to those SOURCE_CHOICES.

Here, for option 2, is an example of what we could do for the help texts. The first model field source currently looks like this:

    source = models.SmallIntegerField(
        help_text="contains the source of the Docket.",
        choices=SOURCE_CHOICES,
    )

For option 2, we could change it to something like this:

    source_help = {
        "dev_help":"contains the source of the Docket.",
        "search_advanced_techniques_help":"",
        "search_popup_help":"to be determined",
    }
    source = models.SmallIntegerField(
        help_text=source_help["dev_help"],
        choices=SOURCE_CHOICES,
    )

We would also change the Advanced Query Techniques page so that it references those "_help" attributes. Similarly, we can add (?) icon popups on the search page to reference those.

3. Kind of a combination between option 1 and 2. Set up one top-level data dictionary in models.py, and then the code that we insert above each field just adds to it. Something like this:

flp_data_dictionary = { "Docket":{}, "DocketEntry":{},

etc

}

class Docket(models.Model):

snip

source_help = flp_data_dictionary["Docket"]["source"] = {
    "dev_help":"contains the source of the Docket.",
    "search_advanced_techniques_help":"",
    "search_popup_help":"to be determined",
}
source = models.SmallIntegerField(
    help_text=source_help["dev_help"],
    choices=SOURCE_CHOICES,
)

Thoughts?

[mlissner]

1. Seems workable to me, though I'd probably want that dict outside of the models.

2. Seems like overkill and like it'd bloat the already huge models.py files.

3. Seems even more complex and like we're trying too hard. I'd loathe coming back to that a year after implementing it.

So...should we go for 1?

[jon-freed]

Per Slack convo, I will proceed with option 1 as Mike described. Will create a dictionary in /cl/search/constants.py in a variable named search_help_texts. Will update the models.py file's help_text attributes and the "Advanced Query Techniques" page to use that dictionary.

*edit: Corrected file and variable names per Mike's comment below

[mlissner]

Er, that should be cl/search/constants.py with a variable named search_help_texts

[jon-freed]

Mike, option 1's effects may not be so great. Please consider the following and let me know if you want to continue with option 1 or try option 2 or 3.

1. Look at models.py, line 919, the Court model.
2. See that the model has a JURISDICTIONS attribute. (line 937)
3. See that JURISDICTIONS is referenced in the "jurisdiction" field's help_text (line 1045) and choices (line 1047).
4. See that for option 1, if we want to pull the help_text out of models.py and put it in a constants.py file, we would need to either (a) import models into constants to be able to access JURISDICTIONS and import constants into models to be able to access the help text (and obviously, that's circular so it will necessitate some refactoring and precise from/import statements), OR (b) move JURISDICTIONS from models to constants and then update the reference to it in cl\people_db\management\commands\cl_import_judges for the argument being added at line 59
5. See that there are similar problems with other constants currently in models.py. E.g. the opinion cluster DOCUMENT_STATUSES.

So, do you want me to continue with option 1?

• If yes, then should I move the constants like JURISDICTIONS and DOCUMENT_STATUSES to the constants.py file or not?
• And if no (so do not continue with option 1), then should I proceed with option 2 or 3, and if yes, which one? If no, then what?