Public domain photos at Wikimedia

hugovk commented 10 years ago

You can find public domain photos at Wikipedia:

https://en.wikipedia.org/wiki/Current_members_of_the_United_States_House_of_Representatives https://en.wikipedia.org/wiki/List_of_current_United_States_Senators

At least the sampling I checked were all: "This work is in the public domain in the United States because it is a work prepared by an officer or employee of the United States Government as part of that person’s official duties under the terms of Title 17, Chapter 1, Section 105 of the US Code."

Below is a long list of the source files. Wikimedia also has resized images. For example, this one: https://upload.wikimedia.org/wikipedia/commons/8/89/John_Boehner_113th_Congress_2013.jpg Has at least these sizes: https://upload.wikimedia.org/wikipedia/commons/thumb/8/89/John_Boehner_113th_Congress_2013.jpg/490px-John_Boehner_113th_Congress_2013.jpg https://upload.wikimedia.org/wikipedia/commons/thumb/8/89/John_Boehner_113th_Congress_2013.jpg/196px-John_Boehner_113th_Congress_2013.jpg

Also, the link at http://sunlightlabs.github.io/congress/#legislator-photos to https://github.com/sunlightlabs/congress/wiki/Contributing-Congressional-Photos is broken, because for some reason the whole wiki is missing from Github.

Current_members_of_the_United_States_House_of_Representatives

List_of_current_United_States_Senators

hugovk commented 10 years ago

I've processed most of these according to the (cached) instructions.

Here's a zip. The structure is: done/ Rep/ Sen/

done/ has them renamed with bioguide IDs, and resized with the script. Rep and Sen have the remaining photos that still need renaming (and resizing).

H000067 and H001056 didn't download properly. I removed them so they need doing again. Their sources aren't in the Rep/ or Sen/ directories.

(I made this script to rename most of the Wiki photos to bioguide IDs using the CSV.)

konklone commented 10 years ago

Hey, thanks for doing all this! For one, I'd accidentally shut off access to the wiki page, but it's back now.

So far, I've sort of been passing down MoC photo gathering lore and following tradition, which is to get them from official sources (GPO, Bioguide, or *.house|senate.gov) rather than Wikipedia. I'm inclined to keep that policy, but it's also true that these sources can be tedious and slow.

I'd welcome some other opinions: /cc @jamesturk @sbma44 @JoshData

sbma44 commented 10 years ago

Perhaps it's worth blogging about this, adding it to Daniel S's advocacy list, or otherwise trying to get a clear ruling. I agree that the status of these photos has grown apocryphal. I'm not confident that being on Wikipedia means the rights are any clearer than otherwise, though -- appearing on house.gov could easily be interpreted to mean it's the product of a federal employee, when in fact it's just licensed from a contract photographer who owns the copyright, through no fault of the Wikipedian's. This is particularly true given that several of the above photos appear to have been rotoscoped to make the backgrounds match, which strongly implies that they're not all originating from a single source/process.

JoshData commented 10 years ago

@sbma44 That's been my policy -- assume info on Wikipedia (esp about copyright) is wrong unless verifiable. So I collect all photos by hand.

nickom commented 10 years ago

All the images on Wikipedia seem to be coming from the same official sources, but thanks to a much larger pool of contributors the photos there are more up to date and doesn't have the large batch of black and white images that we have in our collection.

The underlying questions about copyright plague all solutions (aside from getting the terms of each photographers contract) and so utilizing Wikipedia seems like it would save a lot of work and improve the quality of the images.

JoshData commented 10 years ago

The metadata on Wikipedia says the photos are from http://memberguide.gpo.gov/, and that seems correct. Would be great to add a scraper to get the photos directly from there in unitedstates/congress. (Bioguide IDs are listed there so the mapping should be easy.)

konklone commented 10 years ago

Wow, everyone made such great points. It's time to centralize these, probably in congress-legislators - it's not like we're going to stop using them because of the copyright, might as well use a better source.

The situation seems generally blog worthy, and I bet @mwweinberg would enjoy that.

hugovk commented 10 years ago

If you can accept Wikipedia's claims of public domain (and/or CC etc), another question is can you assume the photos in http://memberguide.gpo.gov are PD and fetch them straight from there?

Anyway, at least Wikipedia tries to always have an image of some sort, and they prefer images that can be re-used commercially.

I checked the licences.

If you click photos from the Wikipedia lists, they show pages which generally contain source and copyright info for each image. For example, this is public domain:

https://en.wikipedia.org/wiki/File:Richard_Shelby,_official_portrait,_112th_Congress_crop.jpg

Of course, as pointed out, this info could be wrong. Some (most?) I just manually checked come from http://memberguide.gpo.gov/, and I don't see any specific copyright claim for the images there (but the front page says: "Access data concerning House and Senate Members from various publications including: the Pictorial Directory, and the Congressional Directory. These publications can also be found at www.gpo.gov.")

Anyway, I made a script to check the copyright status Wikipedia claims they have. I took the source of each Wikipedia list and got lists of the pages for each image ("File:*.jpg").

Of 512 pages, 510 are claimed to be public domain. The other two:

https://en.wikipedia.org/wiki/File:Rep._Mike_Quigley.jpg is CC-BY-SA-3.0 but the source is fgiven as http://www.memberguide.gpoaccess.gov/112/RP/Quigley so why is it CC and not PD?
https://en.wikipedia.org/wiki/File:SenatorGillibrandpic.jpg is CC-BY-SA-2.0 and is from her official 2010 campaign Flickr account. CC is correct. There's also a photo in memberguide.gpoaccess.gov.

These are 404s:

File:Linda_SA1nchez_113th_Congress.jpg
File:RaBAl_Labrador_113th_Congress.jpg
File:Luis_GutiA9rrez_113th_Congress.jpg
File:AndrA9_Carson_113th_Congress.jpg
File:Ben_Ray_LujA1n_113th_Congress.jpg
File:Nydia_M._VelA1zquez_113th_Congress.jpg
File:JosA9_E._Serrano_113th_Congress.jpg
File@3ARubA9n_Hinojosa_113th_Congress.jpg
File:Beto_O7Rourke_113th_Congress.jpg
File:JoaquADn_Castro_113th_Congress.jpg

mwweinberg commented 10 years ago

This is both super interesting and super annoying. And I agree that it does make sense to tie it into the things we discussed yesterday in person and that we hit on in this blog post (http://www.publicknowledge.org/blog/what-going-usage-restrictions-media-congress-). Has anyone tried to just contact the GPO via the email on the site? I'm happy and inclined to try it, but I don't want to duplicate efforts. I assume that they won't have a great answer, but it seems worth a shot.

And yes, I agree that any copyright declaration on Wikipedia should always be taken with a grain of salt.

JoshData commented 10 years ago

@mwweinberg: I emailed the contact address on that site in 2011 and as far as my archive shows I did not receive a response, which is consistent with GPO's behavior. :)

mwweinberg commented 10 years ago

A truly shocking development. I just called the GPO press office and got another email address. The woman on the phone said that the person on the other end of that address will send me some copyright info, but I suspect that it will be hoop #1 of more than 1. I'll share as soon as I track something down (or once I have a sufficiently entertaining tale of woe).

In any event, I agree that this is blog worthy but I think that it is probably more than that. Along with my GPO digging, I'll try and reconstruct the plan that we had 40% developed last fall. This is probably a good excuse to get that moving again.

mwweinberg commented 10 years ago

I just got a callback from the GPO telling me that the images are all in the public domain.

nickom commented 10 years ago

Wow, nice work @mwweinberg !

mwweinberg commented 10 years ago

Sometimes the inherent disruptiveness of a telephone has its advantages.....

But this entire thread really highlights a larger problem. With more and more contractors involved in USG work, the traditional (and totally reasonable) assumption that something that looks like public domain USG work isn't as robust. It seems like there should be a larger policy solution to this because that is, in fact, a problem. Does anyone have other (non-c-span) examples of this actually manifesting itself? Having a list would be helpful in describing the problem and trying to fashion a reasonable solution.

sbma44 commented 10 years ago

Lemme try paging @ghelleks for ideas about situations where contractor involvement has confused the question of whether federal work products are public domain.

ghelleks commented 10 years ago

Chapters 7 and 8 of the DOD Open Source FAQ are a great overview of this issue. It's tailored to software, of course, but it's all still copyright. I'd pay special attention to "Can the government release software under an open source license if it was developed by contractors under government contract?"

You'll also enjoy the Software Tech News article (PDF!) on page 32, which describes just about every circumstance government and contractor rights, and how they restrict licensing.

The issue of how the government curates and catalogs its own intellectual rights is a wonky mess.

mwweinberg commented 10 years ago

Uf. Thanks. Looks like I have some reading to do, the result of which will be very little enlightenment.

ghelleks commented 10 years ago

As you wade into this swamp, @david-a-wheeler has forgotten more about government intellectual rights than any of us have every known. He wrote that Software Tech News article and curated the DOD OSS FAQ. If you have any questions, he's your guy.

ghelleks commented 10 years ago

Sorry. One more suggestion: CENDI.

It's a volunteer organization of gov't lawyers, has an authoritative copyright FAQ which could be valuable, and way more approachable than reading the FAR.

konklone commented 10 years ago

This has been so terrific. But as for actionable steps (since this is Github), does anyone see any practical issues with making a scraper for the GPO Member Guide that downloads their photos, and us offering those to people for reuse without restriction?

sbma44 commented 10 years ago

That seems like the best way to proceed to me.

konklone commented 10 years ago

Okay, then discussion on that scraper and its output is moving over to https://github.com/unitedstates/congress-legislators/issues/160. I'll close this thread as a todo, though continue to discuss any copyright/etc issues here, and anyone who'd like to work on the photo gathering is welcome to contribute something!

konklone commented 10 years ago

There's a new repo at unitedstates/images to hold images gathered from GPO, with code by @hugovk. Star or watch it if you're interested, and let me know if you'd like write access.

hugovk commented 10 years ago

By the way, I found this API after seeing this composite photo of all the members of congress:

http://new-aesthetic.tumblr.com/post/75685167922/twitter-skomputer-just-for-context-here-is-an

I decided to make one with the faces aligned (by detecting and cropping them with OpenCV) and got the photos from Wikipedia, resulting in this:

http://www.flickr.com/photos/hugovk/12329158515/

I then found the first one got was made using Sunlight Labs photos:

"It was created using a Ruby script to manipulate images from the Sunlight Foundation's Congressional Pictorial Directory, which actually contains 813 images.

"Skomarovsky noted that he hasn't sorted out which members of congress are missing or duplicated from that set, so the composite isn't exact -- but it's still a pretty telling snapshot."

http://www.huffingtonpost.com/2014/01/29/congress-average-picture_n_4688163.html

Because I'd already downloaded them, I thought I'd share them. And here we are!

konklone commented 10 years ago

I'm glad something so horrifically disturbing could beget such a productive outcome. :)

propublica / sunlight-congress

Public domain photos at Wikimedia #432