Design a new table in Wagtail to display the 2020 congressional pre-primary reports info

[kathycarothers]

Summary

Looking to create a table in Wagtail that is user friendly and easy to update the 2020 reporting information. The table will be used on the child pages to display reporting information under Dates and deadlines.

Proposed Behavior / Feature

• [x] Looking to create a dropdown for an individual to pick a state to view reporting information.
• [x] Looking for an option to display all 50 states and territories, so the regulated community can also see the big picture.
• [x] Decrease possibility of human error by creating a table that is easy to input data and display information clearly using Wagtail.
• [x] Choose tool we'll use to populate data.

Current Situation

Current situation is we are using straight html code in the template. The current table for pre-election report information is huge. The increased ability for human error is a factor, and it is very time consuming to update.

Use Case

The content team/Info division would benefit from this added feature along with the regulated community.

Misc

Here is a link to how the data was displayed in the 2018 election https://transition.fec.gov/info/charts_primary_dates_2018.shtml

A draft template is in Wagtail under " 2020 Dates and deadlines." It is a child page under this heading, which will display the lengthy html code when a chart is inserted as html code.

Future Work

Other tables to be created in #3017

[rfultz]

Not to commit or over-promise, I have something in mind that I used for WordPress sites, Advanced Custom Fields. There's an element they call a repeater (example video) that contains other fields. I'd like to offer Content a series of rows of fields, like maybe they'd choose a state and table across the various required date fields. We'll need to think about notes, links, etc., though.

Doing some brief research, it looks like a StreamField might come close to offering the same functionality as an ACF repeater. I'd definitely like to research and test it out.

[johnnyporkchops]

Wagtail StructBlock/ListBlock allows similar functionality. We are using this in a few places on the site now. I did some preliminary tests of using this to make a "table builder". One issue with it is how to make it look intuitive in the admin interface. There's also the possibility of adding JS to the frontend interaction to enhance the experience

[JonellaCulmer]

Here's the start of a table wishlist: https://docs.google.com/document/d/1AlYC-ihoMkda0tpYM3Hhew7_bD_nV7yvvuITeCsKeXo/edit#

@dorothyyeager Did you have another list?

[dorothyyeager]

I have a bigger Wagtail wish list, but this is the only table item on it. Looks like @johnnyporkchops commented on it...

Tables: Would be great if we had the ability to merge cells in a table or make text that’s not a header bold. A little more ability to customize (but not the custom table or manually coding the table). Might be possible with handsontable API -JC

[JonellaCulmer]

@kathycarothers Was there a published version of this table anywhere that I can see (on .gov, not transition), or is it in draft in wagtail?

[kathycarothers]

@JonellaCulmer I have draft in wagtail at https://fec-prod-proxy.app.cloud.gov/admin/pages/10428/view_draft/ and here is the chart for 2018 https://transition.fec.gov/info/charts_primary_dates_2018.shtml

[JonellaCulmer]

@rfultz Below is a mock-up of what I have so far and something that can kick-off a discussion about capabilities, including those on the back-end where this information is entered.

Some functionality I've been pondering: https://docs.google.com/document/d/1XfB4LeZgRQlXXV12k5AxS4c87PQDNFv7QLfQl8EnSEE/edit

@dorothyyeager @kathycarothers Please let me know if there's anything else that should be added to the list above, or what you think at this very preliminary stage.

[dorothyyeager]

I have some text and word style edits to suggest, but I don't think we're at that point yet? The only other comment I have is I'd love to get away from footnotes (in general, I'd love to do this on everything). I'm not sure how to do that with this chart though. Maybe something akin to a tool tip.

From trying to squeeze similar charts into the Record, one thing that always is tricky is that the 48-Hour notices are due during a date range while all the other columns have a single date.

Another item to consider is that Info will need the ability to add a row. Sometimes party committees within a state decide to have a convention triggering these reports at the last minute, and we scramble to get the info up.

[AmyKort]

Wow, I think this looks super great! Thanks, everybody! I totally agree with @dorothyyeager about the footnotes, but I'm not sure what to replace them with . . . Ideas?

[JonellaCulmer]

Great point, @dorothyyeager I can add that to the list of things to think about. But that seems like it might involve a lot of conversation, right?

[rfultz]

I'm not sure about the alternating row colors. The even-odd coloring is simple, but we'll probably be hiding and removing rows, not adding or deleting them so if every other row were hidden, the visible rows would be the same color.

Would it help clarify the relations between rows if a state's row is one color and its related rows are the same color? So a state would be grey and its runoff row would be, too? I'm not sure how closely related the rows are supposed to be for your Connecticut example.

[johnnyporkchops]

@rfultz, for the Wagtail implementation aspect of this, in addition to building it in Wagtail templates, there is be possibility of extending the API that the current default- and custom-table blocks uses. As Dorothy can attest too, they the current tableblocks are limited in what they offer the user. I found the API a bit daunting, but maybe we should take another look at in in case it offers a more versatile solution. https://handsontable.com/examples?manual-resize&context-menu&filters&dropdown-menu&headers&trim-rows&collapsible-columns&multi-column-sorting&fixed

[kathycarothers]

@rfultz @johnnyporkchops @JonellaCulmer In terms of row coloring - A state could be all one color, but there does need to be alternating color for each different state. Does this make any sense at all? FYI I am out tomorrow.

[JonellaCulmer]

For most states it's not a problem that they all be one color. I'm concerned about the states with 5, 6, 7 elections. And the alternating colors will really help in those cases. I think accessibility and readability needs should be the guide here.

[kathycarothers]

Going back to the footnote discussion. Information in the footnotes is very important what if we added another/extra column and labeled it Notes or Information? Just an idea. Would that be possible? @JonellaCulmer @AmyKort @dorothyyeager @rfultz @johnnyporkchops

[AmyKort]

That would work for me @kathycarothers. You make a really good point about how important those footnotes are.

[dorothyyeager]

I like that idea @kathycarothers - it is important information and stakeholders won't want to get rid of it, but footnotes aren't very user-friendly and anchors aren't really doable in Wagtail, so making a column for them would eliminate a click and make the information more visible.

[rfultz]

We could do notes, disclaimers, etc. but if we're making this content dynamic, footnotes and endnotes are going to be difficult. It would be so much easier to put the fine print with the entries, like if we included it near the bottom of (but still inside) the relevant row/item rather than separated elsewhere on the page.

[JonellaCulmer]

@kathycarothers @AmyKort @dorothyyeager Some of the text in the footnotes are very long. We can certainly discuss condensing some of the language, but at present, those long blurbs would make for a very long and skinny column. Below is a mock-up of what it could look like if we incorporate the footnotes into the table.

Unfortunately, I'm not sure how to handle the footnotes that explain what each of the headers mean. We may still need to rely on footnotes for that, or find another way to explain what each column means without all the elaboration.

[JonellaCulmer]

What it looks like with a new column:

[dorothyyeager]

Ai yi yi. I like your first mock-up @JonellaCulmer.

For the header text, I'd want to see what the footnotes say, but maybe that could be introductory text.

[rfultz]

We have other tools available to us, too. Tooltips (accessibility challenge), light boxes, abbreviated/teaser text that expands on click…

[dorothyyeager]

Let us know what might work best! I just think we need to find a way to incorporate the important information that is not a footnote.

Also we'll need to eliminate references to the Secretary of the Senate in said note when the time comes to draft these tables' wording. That's so 2018. ;)

[kathycarothers]

These are all great ideas. I think we are going down the right path to incorporate this important information into the chart. I am bugging out at noon today, but would like to continue this discussion back on Tuesday. I am off on Monday. I do like the mock-up with the footnotes right under the filing information that Jonella just put together. It's like a merged row.

[JonellaCulmer]

Using the above designs as a starting point for discussion we've compiled and prioritized the work relating to building the new pre-primary table and a new template for that table. Below is the document again which lays out the work: https://docs.google.com/document/d/1XfB4LeZgRQlXXV12k5AxS4c87PQDNFv7QLfQl8EnSEE/edit

[rfultz]

If building the admin interface inside Wagtail proves problematic for our target launch date, an interim solution could be to set up a spreadsheet with the data with a place in the spreadsheet where the page/site editors could copy the proper html out of the spreadsheet and paste that into Wagtail. The spreadsheet would incorporate its own data into the format of the html of the table as it is now. Then we could continue working on the more advanced table functionality without holding up the table publication.

[johnnyporkchops]

It looks like this spreadsheet option might work. It would get a little tricky with including the footnote rows as it was decided to put them under each row they apply to instead of in the last column. https://github.com/fecgov/fec-cms/issues/3016#issuecomment-510931494. If we can figure out those issues, then I think it's up to the content team wether they would like to use either the spreadsheet or a WSYWIG html editor to maintain/copy-paste the html.

I have the JS interaction to show/hide rows and am working on an interaction decided today to show/hide footnote rows as well. The team decided against tooltips due to the importance of the language and the typically large word-count.

To rule out the Wagtail solution, I did create a working table generator block in Wagtail, seen here: https://fec-feature-cms.app.cloud.gov/admin/pages/10046/edit/ The table-generator block creates a table that is specific to the Pre-Primary table. The team agrees that this would ultimately more tedious to maintain than other solutions. However, this full-page template that was created, and the general structure will likely be a good starting point for an Org Chart Wagtail template, which is not as complex and large as the reporting dates tables and might work in Wagtail.

[dorothyyeager]

We looked at a demo of @rfultz 's spreadsheet and also discussed the use of the WSYWIG editor. After discussing it with @kathycarothers and @johnnyporkchops and David Garr (who's the person who inputs the data), we'd like to investigate using the WSYWIG editor. I've sent an email to Info management and @patphongs to ask about getting a Creative Cloud license longterm. In the short term, @johnnyporkchops is going to send us a list of possible online and open source editors to check out. We'll have to make sure whatever we use is OK security-wise.

[johnnyporkchops]

Here is a list of desktop and online WYSIWYG html editors and a quick evaluation of each. There are hundreds of web editing tools with live preview, but with most, you cannot edit on the live preview. All in this list have that feature. I am including several because I have only tried them on Mac and if we find that they behave differently in Windows then we have more to choose from. Favorites are: Desktop: Dreamweaver (paid), Amaya (Free- this one has a good WYSIWYG table editor) Online: html5-editor.net/ (Free)

Desktop Editors

App	Cost	Platforms	Notes
Dreamweaver	Paid	WIn/Mac	This one is my pick. Has the most stable/robust interface and editing tools. As mentioned in the past we want to limit the editors' use of panels and buttons that add code to the html.
Amaya ("W3C's Web Editor")	Free	WIn/Mac	. Originally, I picked this one, but after using it to edit tables, I found the interface a little sticky and cumbersome. Has some nice table-editing options like merge-cells, add row, add column. Also allows user to add classes to the Table rows via a chooser menu which might be useful for our US State classes (But as mentioned, we really want limit editors' use of panel options that add code :-) . The interface is a bit Old School but in a no frills way
KompoZer	Free	WIn/Mac	Old-school looking interface that is a bit klunky
Pinegrow	Paid-but cheap lifetime one time usage fee	WIn/Mac	Works well, nice modern interface
BlueGriffon	Free	WIn/Mac	Works OK, old-school interface
KodeWeave	Free	Win/Mac	modern interface - Has panels for html, js, css, liveview that were a little hard to manage if I just wanted to see html and liveview but I am sure its just a matter of configuring your workspace

---

Free Online Editors

App	Cost	Platforms	Notes
https://html5-editor.net/	Free	online	simple interface- works with table the size of ours, Has subscript/superscript options on WSYWG side
https://html-online.com/editor/	Free	online	Works with table of our size-has subscript/superscript options on WSYWG side
https://htmleditor.online/	Free	online	Toggles between code/live view -- no side by side view, but works well, no sub/superscript option

[kathycarothers]

Just taking a peep at each recommendation - Amaya and Pinegrow look good. Pinegrow seems to be modern and I like the way they presented their info on their website. But you all are the experts how these items work.

[dorothyyeager]

So David Garr and I just took a look. Of the desktop ones, if @johnnyporkchops recommends Dreamweaver, that seems like the best option. We've not heard back on our request to get it.

Of the online editors, we liked https://html5-editor.net/ best. Not as concerned about subscript/superscript options - they're more of a nice to have than a must. That particular editor seemed to be the simplest to work with. Do we need to check in with anyone to see if it's OK to use that one?

[dorothyyeager]

We are switching back to using the spreadsheet to do the data entry, in order to minimize user error.

We just had a meeting after @patphongs @rfultz and @johnnyporkchops took a look at Info Div's master 2020 dates spreadsheet. Pat's going to work on automating the html code for it while John continues working on the template side.

cc: @kathycarothers

[patphongs]

@dorothyyeager @johnnyporkchops @rfultz I've been able to develop the spreadsheet to match the current Election Dates spreadsheet as much as possible. There were additional columns I needed to add to accommodate for additional fields like state classes, footnotes, urls, etc. Let's talk about that sometime this week and see if this is something the information division would be ok with. HTML proof of concept can be found here: Wagtail page :lock:

I'll share the spreadsheet with you all after that meeting since I would like to explain how it's currently working before sharing it.

UPDATE: There is one to-do left, in which case we need to find out the best way to accommodate for states that are still TBD for their Election Dates. Because that value is needed for all the other dates, excel is not able to calculate the HTML for those rows. That may need to be manually populated.

[johnnyporkchops]

@JonellaCulmer , @djgarr , @kathycarothers Here are two slightly different treatments of the footnote rows for the Reporting dates tables. 1) One gives user ability to open all footnotes for the row. 2) The second, only opens one at a time and removes the bottom border on the cell to create a "faux" tabbed panel experience.

Curious which you think is better? After looking more closely at an example table I have noticed that there are rarely ever more than one footnote per row, but maybe others have more...or maybe I am overthinking the whole footnote interaction.

All footnotes for a row

One footnote at a time per row

[djgarr]

Sorry, didn't mean to close the issue

[djgarr]

@johnnyporkchops @JonellaCulmer @kathycarothers I like option one that shows all of them. There aren't too many rows that'll have more than one footnote to them, but I do like the option to see them all at once if possible. But that's just my two cents

[rfultz]

@johnnyporkchops how will it work for the smallest devices? Will the footnote appear under the single date or under the last date? Also, are any of the footnotes so long that expanding them will overflow a phone screen? What would showing every note together do to phone users? I'd like to keep the content and its related footnote on the screen at the same time if we could.

[djgarr]

@johnnyporkchops I sent the options to Liz to take a look at. Will report back once I hear from her

[djgarr]

@johnnyporkchops Liz likes the first option too

[kathycarothers]

@johnnyporkchops First option too for our users.

[rfultz]

@johnnyporkchops If we go with the first option and, for the smallest widths, if all of the footnotes are after all of the dates (Election Date, Close of books, Mailing deadline, Filing deadline, 48-hour notices, footnote 1, footnote 2, footnote 3, etc), could we at least scroll to the footnote when it's tapped/toggled on? I can easily see opening footnote 1 and it stretching off the bottom of the screen, tapping footnote 2 and nothing seems to happen because it opened after #1 well off the bottom of the screen.

[johnnyporkchops]

@rfultz The footnotes currently open under the main row in smaller screens (the same as they do desktop size) @jonella and I were thinking about this issue you mention as well --user not seeing the footnote opening underneath either the main row or another footnote. I considered scrolling to footnote upon opening it (in small screens) but was trying to avoid jumping around the screen, but maybe that's the way to do it. I will give it a try.

[JonellaCulmer]

@rfultz As @johnnyporkchops mentioned, I have the same concern about the scrolling. I'd like to minimize the amount of scrolling the user does to find what they just clicked on, or the confusion that might occur when they click on something and not see it at all and assume that something went wrong.

Below are some mockups of what it can look like on mobile if we incorporate each footnote to go along with each date/item with a footnote. Each one will open just below the applicable date. Hopefully, this will have a shorter learning curve, because if the user can see one footnote open below the relevant date, they'll know to scroll down to the next one when they do it again later on.

As for desktop, I think this implementation matches most closely, John's second example for populating the table with footnotes. Each one opens up like a drawer and makes it clear which one pertains to which date, similar to how it will go on mobile. Instead of implementing one behavior for mobile and one for desktop, these two seem to be the closest in terms of the behavior a user will experience.

Thoughts? cc: @kathycarothers @djgarr

Also, need feedback from the content team on whether or not we want italicized footnotes. The current implementation on other pages is italics. But these can be long paragraphs, not sure what folks think about that.

[dorothyyeager]

I think we have been inconsistent over the years about italicizing footnotes in Guides and the Record (not sure about prior notices). We've tended to do it. That said, we aren't doing it due to GPO style requirements, because GPO style is basically that the footnote should follow the text's style.

So I actually think that in a situation like pre-election charts where we are required to use footnotes, we should stop italicizing them (and we should take out the italics on Dates and Deadlines page too). @AmyKort what do you think?

[patphongs]

This will move into 10.2. There's some finishing touches we need for the PR.