everypolitician / everypolitician-data

data for national legislatures worldwide

http://everypolitician.org/

238 stars 54 forks source link

Guatemala (Congress) -- disappearing members and parties #25282

Open ondenman opened 7 years ago

ondenman commented 7 years ago

Problem with Incoming Data

Legislature

Guatemala (Congress)

Problem

Lots of membership changes. Statement count down. Party names changing. Members being removed.

The source site no longer provides data for term 7.
Live data contains incorrect birth dates.
It appears that the site has been updated and some fields are no longer being captured.
There are fewer incoming term 8 members than term 8 members in the live data.

Steps to be taken

[x] Archive term 7 members
[ ] Update scraper to capture missing fields (area, email birth date)
[ ] Make sure birth dates are correct (no 2016 birth years)
[ ] Archive off missing term 8 members.

ondenman commented 7 years ago

Description updated.

ondenman commented 7 years ago

The scraper is only picking up term 8. Looks like we're losing term 7.

ondenman commented 7 years ago

158 members are being added to term 8. 158 members are being removed from term 7. 162 members are being removed from term 7.

ondenman commented 7 years ago

The term is hardcoded in the scraper. Term 7 data no longer exists on morph.

ondenman commented 7 years ago

Many members have an incorrect birth date. For example:

id,name,party,faction,term,image,area,birth_date,phone,email,source
325,Alsider Antonio Arias Rodríguez,FCN,FCN,8,http://www.congreso.gob.gt/manager/images/C3FB6058-38D8-7A18-FC66-797FC6D77297.jpg,Chimaltenango,2016-01-19,,congreso@congreso.gob.gt,http://www.congreso.gob.gt/perfiles.php?id=325
326,Héctor Melvyn Caná Rivera,UNE,UNE,8,http://www.congreso.gob.gt/manager/images/C86388D4-51F5-6988-6F58-798C92F28DDC.jpg,Chimaltenango,2016-01-19,,congreso@congreso.gob.gt,http://www.congreso.gob.gt/perfiles.php?id=326
327,Víctor Manuel Estrada Orellana,PP,Reformador,8,http://www.congreso.gob.gt/manager/images/EA3D3649-2E56-C564-B1AB-7994971286BF.jpg,El Progreso,2016-01-19,,congreso@congreso.gob.gt,http://www.congreso.gob.gt/perfiles.php?id=327

ondenman commented 7 years ago

Email and area columns are empty in latest data. Also, birth date.

ondenman commented 7 years ago

There are fewer term 8 rows in the latest incoming data than term 8 rows in the live data. It's possible that some term 8 members need to be added to the archive.

ondenman commented 7 years ago

I've archived term 7 members in: https://github.com/everypolitician/everypolitician-data/pull/30356