Matching up scraper and Lugar committee list

[beamalsky]

Lugar's committee/subcommittee list doesn't match up with the scraped data. They should. Why don't they? Could we share the tables below and ask Jamie where they're drawing theirs from?

• The Lugar Senate data does not include subcommittees
• There are 153 House committees and subcommittees in the Lugar data and 298 in the scraped data.
• Some Lugar committees have more subcommittees than the scraper, and some have fewer. For example:

Scraper: Committee on House Committee on Homeland Security (Select)

Subcommittee on Intelligence and Counterterrorism Subcommittee on Emergency Preparedness and Response Subcommittee on Economic Security, Infrastructure Protection, and Cybersecurity Subcommittee on Cybersecurity, Science, and Research and Development Subcommittee on Border, Maritime, and Global Counterterrorism Subcommittee on Prevention of Nuclear and Biological Attack Subcommittee on Infrastructure and Border Security Subcommittee on Management, Integration, and Oversight Subcommittee on Emergency Preparedness, Science, and Technology Subcommittee on Emergency Communications, Preparedness, and Response

Lugar (copied directly from data sent): 110,Homeland Security

1101,Border and Martime Security 1102,Counterterrorism and Intelligence 1103,Cybersecurity and Infrastructure Protection 1104,"Emergency Preparedness, Response, and Communications" 1105,Oversight and Management Efficiency 1106,Transportation and Protective Security 1107,DHS Management and Acquisitions 1108,Fusion Centers 1109,Weapons of Mass Destruction 11001,First Responder Communications 11002,"Chemical, Biological, Radiological, Nuclear and Explosives" 11003,"Science, Technology, Research and Development"

and

Scraper: House Committee on Armed Services

Subcommittee on Projection Forces Subcommittee on Research and Development Subcommittee on Seapower and Strategic and Critical Materials Subcommittee on Readiness Subcommittee on Strategic Forces Subcommittee on Investigations Subcommittee on Terrorism, Unconventional Threats and Capabilities Subcommittee on Total Force Subcommittee on Seapower and Expeditionary Forces Subcommittee on Military Installations and Facilities Subcommittee on Tactical Air and Land Forces Subcommittee on Procurement and Military Nuclear Systems Subcommittee on Military Acquisition Subcommittee on Military Personnel and Compensation

Lugar: 103,Armed Services

1031,Emerging Threats and Capabilities 1032,Military Personnel 1033,Oversight and Investigations 1034,Readiness 1035,Seapower and Projection Forces 1036,Strategic Forces 1037,Tactical Air and Land Forces 1038,Panel on Business Challenges within the Defense Industry 1039,Panel on Defense Financial Management and Auditability Reform 10301,Panel on Defense Acquisition Reform

[fgregg]

Thanks,. @beamalsky. would you mind identifying a couple of examples of subcommittees that do not appear to be in the scraped data at all.

For example, ",Border and Martime Security" probably is in the scraped data under "Border, Maritime, and Global Counterterrorism"

Please provide a two examples of subcommittees that do not appear to be in scraped committeees at all. Thanks!

[beamalsky]

The two sets of tables above, narrowed down to ones I can't identify or am not sure about in the scraped data:

Lugar (copied directly from data sent): 110,Homeland Security

1106,Transportation and Protective Security 1108,Fusion Centers 1109,Weapons of Mass Destruction (maybe this is Prevention of Nuclear and Biological Attack?) 11001,First Responder Communications (maybe Emergency Preparedness and Response?) 11002,"Chemical, Biological, Radiological, Nuclear and Explosives" (or maybe this is Prevention of Nuclear and Biological Attack?) 11003,"Science, Technology, Research and Development" (or maybe this one is Cybersecurity, Science, and Research and Development?)

Lugar: 103,Armed Services

1038,Panel on Business Challenges within the Defense Industry 1039,Panel on Defense Financial Management and Auditability Reform 10301,Panel on Defense Acquisition Reform

[fgregg]

Thanks!

[beamalsky]

I also pulled up the other names of the scraped Homeland Security subcommittees, which makes some of them more clear:

Name	Other Name
Subcommittee on Intelligence and Counterterrorism	Subcommittee on Intelligence, Information Sharing, and Terrorism Risk Assessment
Subcommittee on Intelligence and Counterterrorism	Subcommittee on Counterterrorism and Intelligence
Subcommittee on Economic Security, Infrastructure Protection, and Cybersecurity	Subcommittee on Transportation Security
Subcommittee on Economic Security, Infrastructure Protection, and Cybersecurity	Subcommittee on Transportation Security and Infrastructure Protection
Subcommittee on Border, Maritime, and Global Counterterrorism	Subcommittee on Border and Maritime Security
Subcommittee on Management, Integration, and Oversight	Subcommittee on Management, Investigations, and Oversight
Subcommittee on Management, Integration, and Oversight	Subcommittee on Oversight and Management Efficiency
Subcommittee on Management, Integration, and Oversight	Subcommittee on Oversight, Investigations, and Management
Subcommittee on Emergency Preparedness, Science, and Technology	Subcommittee on Cybersecurity, Infrastructure Protection, and Security Technologies
Subcommittee on Emergency Preparedness, Science, and Technology	Subcommittee on Emerging Threats, Cybersecurity, and Science and Technology
Subcommittee on Emergency Communications, Preparedness, and Response	Subcommittee on Emergency Preparedness, Response, and Communications
Subcommittee on Infrastructure and Border Security
Subcommittee on Emergency Preparedness and Response
Subcommittee on Prevention of Nuclear and Biological Attack
Subcommittee on Cybersecurity, Science, and Research and Development