unitedstates / congress-legislators

Members of the United States Congress, 1789-Present, in YAML/JSON/CSV, as well as committees, presidents, and vice presidents.
Creative Commons Zero v1.0 Universal
2.06k stars 507 forks source link

Drop fax #s? #411

Closed JoshData closed 7 years ago

JoshData commented 7 years ago

In #219 I suggested we drop fax numbers, but consensus was that we should keep them and Sunlight suggested they were going to keep them up to date.

Per #405, they may be out of date. I don't know if anyone is checking them. Sunlight obviously isn't anymore. Our scripts don't update the values. The only fax numbers we're now offering is for senators who were serving last Congress - we didn't pull forward fax numbers for re-elected members.

If no one chimes in to say they are taking responsibility for the fax numbers, I'd like to drop them.

essandess commented 7 years ago

With respect, absolutely not. 👎🏻👎🏻👎🏻

For effectiveness, the fax data is more important than the email forms.

The advice I read about contacting Congress is:

  1. Phoning a district office is most effective, https://storify.com/editoremilye/i-worked-for-congress-for-six-year
  2. Writing a district office is most effective, https://www.ted.com/talks/omar_ahmad_political_change_with_pen_and_paper
  3. Writing or phoning the personal contact of the LA is most effective, like the lobbyists all do
  4. Faxing a personalized letter is as good as a mail letter
  5. Email the least effective, https://storify.com/editoremilye/i-worked-for-congress-for-six-year
  6. Social media not effective at all

Personally, I choose to go with fax because it's as fast as an email and more effective, fax modems cost under $15 https://www.amazon.com/Internet-Modem-V-90-V-92-Sewell/, and my calling plan already covers all US long distance calls.

If I had more time I'd call the district offices. But I don't, so I fax rather than doing nothing.

It would be great if the EFF's https://democracy.io EFForg/democracy.io or Sunlight's contact-congress offered the range of phone/fax/email contact options.

essandess commented 7 years ago

I haven't checked all the fax numbers in the existing data, but the ones I have (numbers in the hundreds) have all been correct, so this looks like a useful, accurate dataset to me.

essandess commented 7 years ago

And per #405, these are all additions of fax data to the district offices, not corrections, so no, they are not out of date.

JoshData commented 7 years ago

I strongly doubt that a faxed letter is better than a message sent through a webform (which is what I assume you mean by an email).

But that aside - If you're volunteering to do the work, then keeping the fax numbers is good with me. Will you submit a PR that ensures every legislator has an up to date fax number listed?

JoshData commented 7 years ago

(Ah, yes, I see I mis-read #405. There are additions (to legislators-current.yaml), not changes.)

essandess commented 7 years ago

Yes I'll submit a PR for fax numbers.

Also, my argument rests on effectiveness, which I've based on what I read staffers say. If you known of any staffer assertions of the effectiveness of webforms/email versus letters, I'd appreciate reading it -- I'll adopt the means that makes the best use of my time.

Finally, would anyone be able to help how to submit a commit-specific PR to this repo with just the lookup script and fax number additions and corrections, if any? My repo is a clone-of-a-clone, which I did so that I can use @TheWalkers district data.

I see from stackoverflow it might involve voodoo like git reset HEAD^, but I don't really know.

JoshData commented 7 years ago

Staffers tend to think that what happens in their office is what happens in every office, and you've probably heard from staffers in offices that happen to like faxes. I am confident we could find a few offices that throw faxes in the trash (metaphorically, since of course the fax is never printed). But I doubt we would find even one office that has a webform that doesn't store every message that goes into it. Whether any of this is effective or not has a lot more to do with other factors in any case (how many messages are sent, are there parallel advocacy efforts happening, opportunity costs for the constituent, ...).

essandess commented 7 years ago

Staffers tend to think that what happens in their office is what happens in every office, and you've probably heard from staffers in offices that happen to like faxes.

This would support the case to keep the fax data, along with the three basic options that legislators provide to contact them: phone numbers, fax numbers, and web forms.

Whether any of this is effective or not has a lot more to do with other factors in any case

Agreed. I would add the personalized nature of the contact to these factors. Faxing a personally written letter provides this option.

essandess commented 7 years ago

I just cross referenced independent congressional fax data with the fax data in committees-current.yaml.

Out of 538 checks, I only found 2 inaccuracies, and made two extra additions, one for the district data. Here's the commit.

I'd say this is confirmation that the fax data is accurate and useful.

When it's useful to do this again, here's a quickie script:

import csv,fnmatch,os,re,yaml

with open('congress_fax_data.csv','r') as c:
    cdata = [[w.replace('\xa0','') for w in line] for line in csv.reader(c)]

with open(os.path.expanduser('~/Source/github/essandess/congress-legislators/legislators-current.yaml'),'r') as y:
   ydata = yaml.load(y,Loader=yaml.CLoader)

check = dict()
for line in cdata:
    try:
        cfax = line[-1]
        names = line[0].split(' ')[1:]
        for leg in (leg for leg in ydata if fnmatch.fnmatch(leg['name']['last'],names[-1]) and fnmatch.fnmatch(leg['name']['first'],names[0])):
            yfax = [term['fax'] for term in leg['terms'] if 'fax' in term]
            if cfax not in yfax: check[line[0]] = line[1]
    except:
        pass
for name in check: print('{}: Check fax {}.'.format(name,check[name]))
essandess commented 7 years ago

Relevant: Apps Make Pestering Congress So Easy That Politicians Can't Keep Up. This and related repo's and their authors are cited.

The consensus among activists and staffers alike is phone calls are better than email, and that showing up in person to district offices or town halls is better than phone calls. But email and social media make a difference too, as long as those communications are personalized.

This speaks to the importance of including both the fax data and the district data, which both facilitate personalized correspondence.

apurvis commented 7 years ago

With respect, absolutely not. 👎🏻👎🏻👎🏻

+1000 to the idea of not dropping support for faxes.

apurvis commented 7 years ago

I strongly doubt that a faxed letter is better than a message sent through a webform (which is what I assume you mean by an email).

Really? Everyone I've ever talked to in any kind of office that has a fax machine checks their fax machine before their spam folder.

JoshData commented 7 years ago

Really?

Really!

I'm unsubscribing from this thread. The only thing I care about is whether someone steps up to take responsibility for the data field and gets a PR merged with complete and accurate data for the 115th Congress.

essandess commented 7 years ago

@JoshData, Per my comment above, I have independently confirmed that all fax data for the 115th congress is accurate.

However, I do see from looking at the current term data using code in #428,

python congress_lookup.py -c Ways -T fax

that none (or little) of this data has been carried over to the current term.

Is this something that is done by hand? Or done by automated scraping? I would have expected the latter. Why doesn't this data appear automatically, and how should it be entered?

essandess commented 7 years ago

By the way, I have direct, personal, recent experience using this script with a fax modem that faxes to Congress are read and taken seriously.

Automated apps are important, but we now have this issue: Apps Make Pestering Congress So Easy That Politicians Can’t Keep Up.

Telephone, letter, and fax will always be of primary importance, and automated apps should include this information to provide these options.

joelcollinsdc commented 7 years ago

@essandess Are you saying that you validated all the fax numbers from the 114th congress and you think that these can be automatically carried forward to the 115th congress terms for those members who currently have no fax number listed? I'd be happy to do that.

Also, can you show how you are validating these so I can spot check them?

konklone commented 7 years ago

@essandess As @JoshData said, he unsubscribed from the thread. To both you and @apurvis, I just want to remind you that this dataset is maintained for free by people who have put hundreds or thousands of hours of human review over the last 4-5 years, and who have many more years than that in extensive experience working with Congress and its data. In @JoshData's case, at least 13 years. Feel free to bring up your own expertise and debate the value of fields, but I encourage you to take extra care in the words you choose and to treat the dataset maintainers' opinions as rooted in as much or more experience and reality as yours are.

Is this something that is done by hand? Or done by automated scraping? I would have expected the latter. Why doesn't this data appear automatically, and how should it be entered?

There is no central server where we run the scripts in the repo regularly, commits are always done by pull requests, and pull requests are always filed by hand. Automating the pipeline is less feasible than it may appear, as a) automating git behavior and pull requests in a safe way is non-trivial, and b) many fields get a lot of value out of human review before even filing the PR. Such is the reality of creating a high quality dataset gathered from dozens of different sources.

Thank you for your work at validating and improving our fax numbers, and demonstrating user interest in maintaining the field. I'm reviewing #428 and leaving comments, and I concur with @joelcollinsdc that we'd benefit from understanding your methodology for validating the fax data.

essandess commented 7 years ago

@joelcollinsdc @konklone

I've validated the fax data using two different methods:

  1. Googled for online congressional fax lists generated independently of this project, parsed them, used the quickie script in my comment above to highlight potential errors, then googled by hand to check and correct any discrepancies.
  2. I use the fax data to contact congress myself. I've haven't faxed the entire congress, but I've sent hundreds. I've only seen one or two errors that I corrected.

Furthermore, Congressional fax numbers are very rarely changed for all the obvious reasons: they're well advertised in print, local political organizations, web pages, civic groups, lobbying groups, and on and on.

I would presume no change in phone or fax contact information for a new legislative term unless there is explicit evidence for it.

joelcollinsdc commented 7 years ago

Maybe having some real input from staffers in congressional offices would be helpful here, but my thoughts on the whole fax number situation is that they are likely to have more negative effects than potential positive effects. I will guess that faxing is a pretty essential part of the "get help with a federal agency" process which is why member offices advertise them (especially the fax numbers associated with district offices, where casework happens). I'm guessing here, but the volume of casework faxes must be much lower than the volume of constituent comments, but each request is very important and if these requests get lost in a high volume of abuse of the fax system, these constituents will suffer.

Also, I think just in general people facilitating communication to member offices should be trying to help member offices deal with the volume of traffic meaningfully. Member offices are able to get the most metadata out of traffic that comes in through the CWC API (i'm guessing?), followed by webforms, followed by anything that get OCR'd, like snail mail and faxes. So we should be good stewards of our role in enabling this process and try our best to advertise the best communication routes first.

Anyhow, I made a PR to pull the fax numbers forward. I did not spot check more than a few.

essandess commented 7 years ago

Congressional contact information is necessarily public. The best anyone can do is to encourage everyone to use this information responsibly. Any abuse is the responsibility of the individual contacting the legislator, and staffers develop strong filters for listening to important constituent versus spam messages.

Real input from staffers is widely available. Staffers comment frequently in news reports and social media on effective ways to contact legislators. Here's just a few such comments from the past week:

Phone calls carry more weight than an email or a tweet, partly because a phone ringing incessantly is harder to ignore than a flooded inbox.

it’s hard for staffers to figure out how much you actually care. “If it takes them five seconds to fill out a form that shoots out a form email,” Ellsworth says. “is that the same as someone who actually takes the time to call?” Oh, and that online petition you signed? You can bet nobody on Capitol Hill saw it, or cares.

JoshData commented 7 years ago

(github un-unsubscribes you when you are @-mentioned, so I am back)

@joelcollinsdc wrote:

I think just in general people facilitating communication to member offices should be trying to help member offices deal with the volume of traffic meaningfully.

I completely agree.

We have a responsibility when we publish information to be cognizant of its intended and unintended consequences and to not publish information that, on balance, does harm. (I'm not saying publishing fax numbers does harm, but we have a responsibility to consider how that might be the case.)

(Btw the NYTimes article that @essandess pasted above relies heavily on the reporter's conversation with ME.)

essandess commented 7 years ago

@JoshData Awesome accomplishments—thank you for all your hard work.

joelcollinsdc commented 7 years ago

Closing as the fax numbers have been updated, thanks @essandess