v4 graph search wireframes

[noahmanger]

Ok, so in light of the FEC user / stakeholder meetings the other week, I've got a slightly new take on the graph search I want to share. The idea here is that in addition to being able to construct a graph search or search for a specific entity, the user can also easy pull up all records for, say, candidates, and then slice it down with filters. I heard in those meetings that users want the ability to easily pull up a data set and filter down, so this responds to this.

Also these new views represent a flow that has the search results on the same search page, instead of a separate results page.

There's a couple sub-options, but both start with a basic search box and big buttons that "view all" of a particular data set.

Option A keeps the buttons in place, showing relevant filters for that data set that can be expanded.

Option B would have the page scroll down, and in so doing, the buttons and search collapse into a single persistent nav bar at the top. Everything else stays the same. (Personally, I think I like this idea most).

[dwillis]

Is there a finite set of filters, or will possibly all fields be filterable?

[noahmanger]

Note on the above screens: there shouldn't be anything typed in the search field.

Also, here's a take at how a "disambiguation" style view would work if you execute a search for a name.

1. You search for a name

2. It pulls up records, categorized by type, collapsed

3. You can expand and then click through to the full record view

I recognize there's some inconsistencies between these two sets of screens. For instance, the candidate records are presented in more of a summary view, instead of a more condensed table view.

[noahmanger]

@dwillis -- undetermined as of yet. Curious what you think. In theory we could filter by any field, but my working assumption is that we don't want to overwhelm the users of this web-based tool with filters for all potential fields. I'd like to do some research on what fields are most useful and then perhaps limit to those, and then if the user wants to download the data and filter offline they can.

But this assumption is in turn based on the assumption that typical power users will be doing heavy filtering of their own offline—but who knows, maybe a slick web app is better?

[dwillis]

@noahmanger I think it's important not to overwhelm folks, so I think having a finite set is best. My list would be: date, name, city/state, amount and maybe employer/occupation.

[noahmanger]

@dwillis noted! What about for candidates and committees?

For candidates, I imagine what I have above: name, party, state, district, office sought and year.

For committees I imagine having nested sets of filters so that if you set a filter for committee type that brings up filters unique to that type. Potential filters in addition to type could include name, candidate supported (if relevant to that type), and maybe a checkbox for lobbyist affiliated?

[dwillis]

@noahmanger For candidates it would be state, office, district and status (incumbent/challenger/open). For committees I'd go state, designation, type and interest group category. lobbyist-affiliated is a tough one, but there are basic proxies for that.

[noahmanger]

@dwillis cool. Re: lobbyist-affiliated, I meant what the FEC calls Lobbyist/Registrant Committee Statements of Organization http://fec.gov/data/Lobbyist.do?format=html

Does the FEC have interest group category data?

[dwillis]

Ah, right. There are plenty of committees that are definitely "lobbyist-affiliated" without meeting that specific criteria, but it's a useful thing to note.

[theresaanna]

WRT the view all screens, option B immediately struck me as preferable. I imagine that there will be an awful lot of data on that screen and perhaps we will end up with more filters/options/toggles/sorts than we can foresee now. I think making the best use of screen real estate possible is a good bet.

I'm not sure option A would scale very well if/when we discover new ways to look at the available data. Option B speaks of the power and breadth of what's available in a glance.

[theresaanna]

@noahmanger For the disambiguation flow, screen # 3: do you think that the previews of the data under candidate, committee and contributor records would be necessary if the site felt smooth and snappy? If the cost of going into a new "page" (whether or not it reloads) felt minimal.

I envision myself using this and expanding the section just to immediately click through to a full(er) page of results, especially if I'm looking at contributors or even committees where there are probably more results. It seems like the difference between the three verticals is fairly self-explanatory.

[noahmanger]

@theresaanna agree with all of the above. I think for the disambiguation let's start with a more condensed view like what you see in the "view all" results screens. And if we hear from people that they want more summary data, we can look into it at that point.

[theresaanna]

Semi-random thought: I'm using the current data exploration offerings on the FEC website and find myself wishing that the office and/or active through year were the most prominent columns for candidates.

I searched for "obama" here: http://www.fec.gov/finance/disclosure/candcmte_info.shtml

[adamclayman]

Would it be possible to design a view of and filter for "In-state" vs. "Out-of-state" funds?

It would be interesting to see which Members of Congress are leaning most heavily on corporate and individual contributions from outside of their Congressional District. Based on my own experience reading this campaign data, such spending tends to be related to graft that attaches to their Committee Appointments. Having a simple way to filter out-of-state from in-state would help show any major fractures in spending that are "influence buys" at the committee level.

[saizai]

One more basic suggestion than UI:

Other than the relative handful of people who know what "Form 3X" or "AOR" mean, most people probably don't know or care about the difference between multiple entities which are basically raising money to elect the same human in different ways or times.

Therefore, I'd suggest collapsing all of them into a single entry — including both from different races and IEOCs — and have the display for that human (e.g. Obama) show their full history (including "non-coordinated" money) in the same place.

Practically speaking, what people want to know is a) who's supporting this person, and b) how much money is going into that from where

Someone's history of financial support is going to be mostly just as relevant to understanding their possible biases (or alternatively framed, their support base) as their current-cycle official entity.

It would also help to mitigate the ongoing issue of various forms of 'dark money' if it's all shown in one place (clearly labeled, of course).

Then within that entry, you could subselect for some section of history (e.g. something like Google's combo stock market time-range selector / amount-raised graph), or for some type of money (official committee(s), party contributions, single-candidate supportive IEOCs, anti-this-person's-opponents IEOCs, etc etc).

[seanherron]

@adamclayman Would having the ability to filter contributors by state satisfy that need? It's fairly easy to introduce something like that as a filter (eg. imagine something like /contributors?state=AZ+CA+RI+TX)

[seanherron]

@saizai We should be careful of is how we represent contributions and their relationship to a candidate. I'm not sure if it's accurate to commingle non-coordinated money and campaign contributions and represent them as having equal value or benefit to the candidate. I think there is value on a "candidate page" of showing explicitly the primary and authorized committees on their own, as they represent the "official" record of the campaign.

I'm also wondering how we could best support things like aggregate contribution amounts in a singular view like that. In particular, say we have just a feed of all "transactions" related to a candidate - both the contributions they have received and the independent expenditures made associated with them. Is saying that the $1,000 someone spent independently on a billboard supporting a candidate the same as saying that person gave $1,000 to that candidate's campaign?

Maybe we could do something like a "activity feed" for a candidate, which could show something like:

10/05/2014: Candidate Filing Posted
10/02/2014: Jane Smith (Austin, TX) made a $100 expenditure supporting Candidate
10/02/2014: Candidate Filing Received, in Processing
10/02/2014: Paul Adams (Los Angeles, California) contributed $500 to Candidate's Authorized Committee, Candidate for a Better America
(etc)

[adamclayman]

@seanherron Yes, an in-state, out-of-state filter would suffice.

I would suggest that the filter recognize the electoral district or state of the candidate who is being reviewed, so that the voter looking at the records can more easily draw a valuable cognitive inference from the filtered data, rather than having to go through the higher cognitive load of selecting "Ohio" from an undiffferentiated set of 50 states, when they're already looking at an Ohio candidate.

Implementing an electoral district test on the addresses listed in FEC reports could help enormously too. The "Ohio" vs. "non-Ohio" test would work fairly well for my Senators, but not for Ohio Congressional Representatives who, for instance, serve a Cincinnati-area district but take 40% of all their money from Cleveland or Columbus. With Congressional Reps, it would be useful to have a ZIP+4 sensitive "In District" vs. "Out of District" and "In State" vs. "Out of State" filter in place.

Then again, ZIP+4 may be the wrong way to go. The Zip Code Tabulation Areas (ZCTAs) are imperfect approximations of Congressional districts, as described in this Sunlight Labs article: http://sunlightfoundation.com/blog/2012/01/19/dont-use-zipcodes/

(Will raise this as a separate support ticket.) Are donor and expense addresses already geocoded by the FEC and individually cross-matched to federal Senate and House Districts? If the answer is "Not yet", that's definitely something that I would add as a separate issue ticket, as deserving of immediate attention. I would find it very hard to believe that Google, Microsoft, and other big players would not gladly geocode all that data for you for free, as part of their public service mission. To get permission for unlimited geocoding by the 18F for the purposes of geocoding and disambiguating FEC contribs and expenses to Senate and House Districts, I would urge you to get in touch with the Social Impact Team at Google. Anthea Watson-Strong (@antheaws) is awesome, and she's available and very responsive via Twitter. I'll try to patch her into this conversation. If she and her team can't get you access to 100,000 geocode requests per day (forward or reverse) on their own, I am sure she will know with whom you ought to be talking to get that clearance. It's the sort of thing that I doubt the Google Team would think twice about. With each contribution and expense tagged by State (which is also Senate) and House District, more important views of the data become readily accessible for end-users in a variety of organizations, and we can just pull the Congressional District and State tags rather than doing the work of geocoding and interpreting that ourselves. I would also suggest that this is well within the statutory authority of the FEC (to collect addresses on contributions and expenses and associate them with a State and Congressional District of origin) and would prove very useful for candidates who want to self-monitor their own election campaign's contribution sources, so that any funds they use remain tied to their Congressional District.

[dwillis]

AFAIK, the FEC does not perform geocoding on individual contributions (end-users have tended to do that).

For contributions geocoding would be fairly straightforward, but expenditures can be misleading in the sense that committees can and do use vendors with out-of-state addresses, even if the work is performed locally. In addition, many committees pay staff using the headquarters' address, not where the person actually is. This is less of an issue for House districts, but some staffers also use DC addresses or home addresses when working for a campaign.

So while I'm definitely not opposed to geocoding, recognize that it does not always accurately reflect in-district or in-state spending.

[adamclayman]

I hear you. There also might be ways for larger PAC organizations to eventually frustrate "Out-of-State" designations by contracting with registered agents in each Congressional District to write checks locally in their name and on their behalf. Nonetheless, I believe the geocoding would be an extremely valuable and enriching public service, both on contributions and expenditures, with contribution geocoding taking priority. The City, State, ZIP reporting structure can be very misleading, and would be extremely time-consuming to filter on or post-process for a district like the one I reside in (OH-11), which has been gerry-mandered to death to include a wide mixture of localities and two major Ohio cities (Cleveland and Akron), with a thin strip connecting them, and multiple ZIP code shares at many of the 11th District's shape file edges.

I was just looking at the FEC Form 3 reporting forms (http://www.fec.gov/pdf/forms/fecfrm3.pdf) in order to disconfirm or confirm whether this would require a form or format change on contrib and/or expense reports, and it seems like you have all the address information you would need to run automated geocoding driven by Form 3 scans.

[dwillis]

Form 3 already collects street addresses, yes, but you wouldn't run geocoders on scans of Form 3. Instead, you would run them on the electronic filings. OCR of Form 3 is a fool's errand (and I've tried). This would leave out Senate campaigns, which file on paper, but the rest would be a good thing.

[adamclayman]

Senate campaigns are easier to geocode anyway, since Senate races are full-state races, so the state designation would work for "In-State" and "Out-of-State" designations on the Senate forms.

By "easier to geocode", I basically mean that you wouldn't have to geocode those contribs and expenses, as long as the contributor/vendor State is listed and reported as done presently.

[dwillis]

True, but the others could be geocoded in a very short period, while Senate geocoding would require the records to be keypunched first. The FEC does enter all Senate contributions and most expenditures, but they don't keypunch street addresses and the time lag is significant (for example, you wouldn't get Senate data for the last 2 months until after Election day in most cases). But that's a different problem that the FEC cannot really solve by itself.

[jsfenfen]

@adamclayman: if you think this is a worthwhile goal, I'd suggest you go ahead and do this. I suspect FEC will see this as out of the scope of their work, and it appears that FEC's unwritten practice is to not make the full street addresses available in the bulk FTP downloads they provide (this may also be to prevent campaigns from harvesting them as donor lists, a practice which is illegal, but perhaps happens anyway). However, the street addresses for house filers this cycle are available in sunlight's bulk Schedule A and B downloads here, though you have to do some filtering to get unique rows.

[adamclayman]

@jsfenfen The bulk downloads you linked me to are an outstanding resource. I'm very grateful that you pointed me to that page, with real-time campaign contribution and expense archive files.

RE: Scope, why in heaven's name would this be judged out-of-scope for the FEC? They're collecting address information for a reason. If they just needed to verify US citizenship, address information does not tell them anything. Why shouldn't contribution data be tagged to the congressional district from which it originates? Why else is such granular address information, in the absence of alternative citizenship verification, being reported back to the FEC?

I love and admire your can-do attitude, and I would be grateful if someone like you (someone who regularly contributes to Sunlight Labs) would integrate campaign contributor geocoding into the Sunlight reporting system. I would not think of that work as relieving the FEC of getting its job done in tagging these contribs and expenses by Congressional District. As a private citizen, I perceive Congressional District tagging of contributions as an FEC duty. It makes all downstream reporting and analysis so much easier, and it's such important and relevant information.

[dwillis]

Again, I think you have to keep in mind that the FEC doesn't just get to decide what it wants to do, but is tasked with specific duties by statute. Geocoding, as valuable as it may be, means the FEC not doing something else. Since 18F's job here is to improve the current web offerings, I agree that this is out of scope.

[adamclayman]

So I'm not asking for geocoded addresses, but rather for In-District vs. Out-of-District address tags.

I am not asking for lat-long, or a contributor map, or any other geographic display information — I'm only asking for a disambiguation of the Congressional District of the street address reported for each contribution and expense. To do that, you may need to do geocoding, but if you find some other method of doing it reliably that's simpler, all the better.

I have a very, very, very hard time believing that if you spoke with the FEC authorities about this, they'd turn down a request to run contributor and vendor addresses through a House District Identification algorithm. If you really think that it's necessary, I would be happy to write all six FEC Commissioners about this to get their formal approval for contributor/vendor House District Identification, and follow-up with a Times or Washington Post editorial if this work isn't approved.

[dwillis]

The FEC gets lots of requests about its data, and it tries to accommodate what it can. What you're asking for is an addition to its data-processing process, which is trickier. I would suggest that you learn more about what the FEC's procedures are before assuming what the FEC would or wouldn't do.

And since I work for the Times, I can assure you that the FEC routinely ignores our editorials :-)

[adamclayman]

There's an electoral race equity argument here as well.

Senate candidates and also House candidates in states with only one Congressional Representative serving the entire state (Montana, North Dakota, South Dakota, Wyoming, Alaska, Delaware, and Vermont) must, by statute, report all their contributions and expenses in a manner that directly corresponds to "In-District" and "Out-of-District". Contributions to House Members from outside of those above-named States are more geographically obscure. That shouldn't be the case. Contributions that are "In-District" vs "Out-of-District" should be equally transparent to citizens no matter whether they live in a 1 House Member or multiple House Member state.

[adamclayman]

@dwillis I did laugh at that. I just feel strange writing an emoticon in this thread. It just seems too informal.

[dwillis]

The statute says nothing about in-district/out-of-district when it comes to reporting. That's purely a byproduct of some geographies. You're asking for a change in what the FEC does. We're not disagreeing about the value of what you're seeking. But this effort isn't about creating new work for the FEC. It's about improving the presentation of what they have.

[konklone]

@dwillis I did laugh at that. I just feel strange writing an emoticon in this thread. It just seems too informal.

Don't worry -- this is GitHub. You can only make it so formal.

[adamclayman]

lol

[adamclayman]

I sent this message to the FEC Commissioners, using the "CommissionerLastName" pattern for Commissioner Goodwin, Peterson, and Walther, who do not list their email addresses at this address: http://www.fec.gov/members/members.shtml. I also left a voicemail with Commissioner Hunter's Office. Her Office is the only one to include a public-facing phone line. In a moment, I will send out a tweet to Commissioner Ravel and Commissioner Weintraub, who have each posted Twitter handles on the FEC contact page.

---

Dear FEC Commissioners,

Would you please authorize 18F and the FEC Data Team to unambiguously decode contributor and vendor addresses for each Congressional Districts?

I recently added a suggestion for an FEC website improvement that may require a moment of your attention. As you know, 18F has been soliciting suggestions on the FEC homepage. What I would suggest is that the FEC translate addresses into House Districts, particularly for House Races, and release the House District as an in-line data point.

Here are the relevant threads: https://github.com/18F/FEC/issues/20 (starting with post #14, halfway down page) https://github.com/18F/FEC/issues/21

In the three Races (OH-11) where I reside, I can tell easily how much my Senators have accepted from in-district vs out-of-district donors, but I would find it extremely time consuming to identify OH-11 contributions by whether they're within the district or outside of it by any method other than ZIP Code, and as you know, using ZIP Codes to identify addresses as "In-District" or "Out-of-District" is fraught with error.

There is a race equity and transparency equity argument here as well. Quoting from #20, post #2:

" Senate candidates and also House candidates in states with only one Congressional Representative serving the entire state (Montana, North Dakota, South Dakota, Wyoming, Alaska, Delaware, and Vermont) must, by statute, report all their contributions and expenses in a manner that directly corresponds to "In-District" and "Out-of-District". Contributions to House Members from outside of those above-named States are more geographically obscure. That shouldn't be the case. Contributions that are "In-District" vs "Out-of-District" should be equally transparent to citizens no matter whether they live in a 1 House Member or multiple House Member state."

Please consider this suggestion carefully and, if you would, please inform the FEC Data Team, 18F and myself of your decision. I would like to very easily cross-compare whether contributors for Democrat, Republican, and Minor Parties were principally "In District" or "Out of District" in any given Congressional District.

Thank you ever so much for your time!

Always,

Adam David Clayman

[konklone]

I edited your address and phone number out of the comment - if you really meant to include it, you're welcome to add it back in.

[adamclayman]

@konklone Thanks! I frankly wish I could edit the last sentence to read "Democratic" rather than "Democrat", but since the copy that reached the FEC Commissioners is now irretrievable, I think it would be irresponsible to edit what I sent in this Github repost.