lgas.csv hiccups

[myf]

@prabhasp so far i have found many lines of lga_id 82, and there is no lga_id 83

[prabhasp]

Do you have access to the dropbox folder. Can you look in the history for lgas.csv and find a good version, and make sure the edits since then aren't lost? Thanks.

On Thu, May 15, 2014 at 5:46 PM, Myf Ma notifications@github.com wrote:

@prabhasp https://github.com/prabhasp so far i have found many lines of lga_id 82, and there is no lga_id 83

Reply to this email directly or view it on GitHubhttps://github.com/SEL-Columbia/nmis/issues/186 .

Prabhas Pokharel http://prabhasp.com US mobile: +1 347 948 7654 twitter/skype/facebook/whatever: prabhasp

[myf]

I can access the folder through starscream. I didn't know that you cab do history on dropbox? On May 16, 2014 9:20 AM, "prabhasp" notifications@github.com wrote:

Do you have access to the dropbox folder. Can you look in the history for lgas.csv and find a good version, and make sure the edits since then aren't lost? Thanks.

On Thu, May 15, 2014 at 5:46 PM, Myf Ma notifications@github.com wrote:

@prabhasp https://github.com/prabhasp so far i have found many lines of lga_id 82, and there is no lga_id 83

Reply to this email directly or view it on GitHub< https://github.com/SEL-Columbia/nmis/issues/186> .

Prabhas Pokharel http://prabhasp.com US mobile: +1 347 948 7654 twitter/skype/facebook/whatever: prabhasp

— Reply to this email directly or view it on GitHubhttps://github.com/SEL-Columbia/nmis/issues/186#issuecomment-43329869 .

[prabhasp]

Yup: https://www.dropbox.com/revisions/Nigeria/Nigeria%20661%20Baseline%20Data%20Cleaning/lgas.csv?_subject_uid=616249

Actually the version on dropbox has 775 lines, and 774 entires, 1 - 774. So looks like something went wrong in the process of transferring to git repo.

On Fri, May 16, 2014 at 9:23 AM, Myf Ma notifications@github.com wrote:

I can access the folder through starscream. I didn't know that you cab do history on dropbox? On May 16, 2014 9:20 AM, "prabhasp" notifications@github.com wrote:

Do you have access to the dropbox folder. Can you look in the history for lgas.csv and find a good version, and make sure the edits since then aren't lost? Thanks.

On Thu, May 15, 2014 at 5:46 PM, Myf Ma notifications@github.com wrote:

@prabhasp https://github.com/prabhasp so far i have found many lines of lga_id 82, and there is no lga_id 83

Reply to this email directly or view it on GitHub< https://github.com/SEL-Columbia/nmis/issues/186> .

Prabhas Pokharel http://prabhasp.com US mobile: +1 347 948 7654 twitter/skype/facebook/whatever: prabhasp

Reply to this email directly or view it on GitHub< https://github.com/SEL-Columbia/nmis/issues/186#issuecomment-43329869> .

Reply to this email directly or view it on GitHubhttps://github.com/SEL-Columbia/nmis/issues/186#issuecomment-43330104 .

Prabhas Pokharel http://prabhasp.com US mobile: +1 347 948 7654 twitter/skype/facebook/whatever: prabhasp

[myf]

same problem on this file, lga_id 82 is repeated twice and there is not lga_id 83

[prabhasp]

Found an old version in email (11/18/2013) that has 1-774, and unique. Not sure about whether all columns exist, handing off to @myf. @zmyao @schafik, have there been revisions to lgas.csv since then?

@myf Could you add a test in T2_SanityChecks that makes sure that:

anyDuplicated(lga_data$lga_id) == 0 length(lga_data$lga_id) == 774

assuming lga_data is the data.frame read in from lgas.csv.

cheers, --prabhas

On Fri, May 16, 2014 at 10:53 AM, Myf Ma notifications@github.com wrote:

same problem on this file, lga_id 82 is repeated twice and there is not lga_id 83

Reply to this email directly or view it on GitHubhttps://github.com/SEL-Columbia/nmis/issues/186#issuecomment-43340514 .

Prabhas Pokharel http://prabhasp.com US mobile: +1 347 948 7654 twitter/skype/facebook/whatever: prabhasp

[myf]

@prabhasp care to share? i assume everything would be the same except for additional data in 83

[prabhasp]

It was attached in the email. Trying again by ccing you directly.

On Fri, May 16, 2014 at 12:11 PM, Myf Ma notifications@github.com wrote:

@prabhasp https://github.com/prabhasp care to share? i assume everything would be the same except for additional data in 83

Reply to this email directly or view it on GitHubhttps://github.com/SEL-Columbia/nmis/issues/186#issuecomment-43349682 .

Prabhas Pokharel http://prabhasp.com US mobile: +1 347 948 7654 twitter/skype/facebook/whatever: prabhasp

[myf]

I get it now, thanks! On May 16, 2014 12:47 PM, "prabhasp" notifications@github.com wrote:

It was attached in the email. Trying again by ccing you directly.

On Fri, May 16, 2014 at 12:11 PM, Myf Ma notifications@github.com wrote:

@prabhasp https://github.com/prabhasp care to share? i assume everything would be the same except for additional data in 83

Reply to this email directly or view it on GitHub< https://github.com/SEL-Columbia/nmis/issues/186#issuecomment-43349682> .

Prabhas Pokharel http://prabhasp.com US mobile: +1 347 948 7654 twitter/skype/facebook/whatever: prabhasp

— Reply to this email directly or view it on GitHubhttps://github.com/SEL-Columbia/nmis/issues/186#issuecomment-43353423 .

[schafik]

I don't know of any revisions to the .csv file.

On Fri, May 16, 2014 at 5:48 PM, Myf Ma notifications@github.com wrote:

I get it now, thanks! On May 16, 2014 12:47 PM, "prabhasp" notifications@github.com wrote:

It was attached in the email. Trying again by ccing you directly.

On Fri, May 16, 2014 at 12:11 PM, Myf Ma notifications@github.com wrote:

@prabhasp https://github.com/prabhasp care to share? i assume everything would be the same except for additional data in 83

Reply to this email directly or view it on GitHub< https://github.com/SEL-Columbia/nmis/issues/186#issuecomment-43349682> .

Prabhas Pokharel http://prabhasp.com US mobile: +1 347 948 7654 twitter/skype/facebook/whatever: prabhasp

— Reply to this email directly or view it on GitHub< https://github.com/SEL-Columbia/nmis/issues/186#issuecomment-43353423> .

— Reply to this email directly or view it on GitHubhttps://github.com/SEL-Columbia/nmis/issues/186#issuecomment-43353530 .

[zmyao88]

I'm not sure about revision, but there were a couple of edition to the lgas.csv since last November.

On Friday, May 16, 2014, prabhasp notifications@github.com wrote:

Found an old version in email (11/18/2013) that has 1-774, and unique. Not sure about whether all columns exist, handing off to @myf. @zmyao @schafik, have there been revisions to lgas.csv since then?

@myf Could you add a test in T2_SanityChecks that makes sure that:

anyDuplicated(lga_data$lga_id) == 0 length(lga_data$lga_id) == 774

assuming lga_data is the data.frame read in from lgas.csv.

cheers, --prabhas

On Fri, May 16, 2014 at 10:53 AM, Myf Ma notifications@github.com<javascript:_e(%7B%7D,'cvml','notifications@github.com');> wrote:

same problem on this file, lga_id 82 is repeated twice and there is not lga_id 83

Reply to this email directly or view it on GitHub< https://github.com/SEL-Columbia/nmis/issues/186#issuecomment-43340514> .

Prabhas Pokharel http://prabhasp.com US mobile: +1 347 948 7654 twitter/skype/facebook/whatever: prabhasp

— Reply to this email directly or view it on GitHubhttps://github.com/SEL-Columbia/nmis/issues/186#issuecomment-43343253 .

[prabhasp]

additions on what columns we are using, correct?

On Sat, May 17, 2014 at 8:39 PM, Zaiming Yao notifications@github.comwrote:

I'm not sure about revision, but there were a couple of edition to the lgas.csv since last November.

On Friday, May 16, 2014, prabhasp notifications@github.com wrote:

Found an old version in email (11/18/2013) that has 1-774, and unique. Not sure about whether all columns exist, handing off to @myf. @zmyao @schafik, have there been revisions to lgas.csv since then?

@myf Could you add a test in T2_SanityChecks that makes sure that:

anyDuplicated(lga_data$lga_id) == 0 length(lga_data$lga_id) == 774

assuming lga_data is the data.frame read in from lgas.csv.

cheers, --prabhas

On Fri, May 16, 2014 at 10:53 AM, Myf Ma <notifications@github.com javascript:_e(%7B%7D,'cvml','notifications@github.com');> wrote:

same problem on this file, lga_id 82 is repeated twice and there is not lga_id 83

Reply to this email directly or view it on GitHub< https://github.com/SEL-Columbia/nmis/issues/186#issuecomment-43340514> .

Prabhas Pokharel http://prabhasp.com US mobile: +1 347 948 7654 twitter/skype/facebook/whatever: prabhasp

Reply to this email directly or view it on GitHub< https://github.com/SEL-Columbia/nmis/issues/186#issuecomment-43343253> .

Reply to this email directly or view it on GitHubhttps://github.com/SEL-Columbia/nmis/issues/186#issuecomment-43428163 .

Prabhas Pokharel http://prabhasp.com US mobile: +1 347 948 7654 twitter/skype/facebook/whatever: prabhasp

[zmyao88]

Yup, addition was made and possibilbly corrupted old lgas.csv during the process.

On Sunday, May 18, 2014, prabhasp notifications@github.com wrote:

additions on what columns we are using, correct?

On Sat, May 17, 2014 at 8:39 PM, Zaiming Yao notifications@github.com<javascript:_e(%7B%7D,'cvml','notifications@github.com');

wrote:

I'm not sure about revision, but there were a couple of edition to the lgas.csv since last November.

On Friday, May 16, 2014, prabhasp notifications@github.com<javascript:_e(%7B%7D,'cvml','notifications@github.com');> wrote:

Found an old version in email (11/18/2013) that has 1-774, and unique. Not sure about whether all columns exist, handing off to @myf. @zmyao @schafik, have there been revisions to lgas.csv since then?

@myf Could you add a test in T2_SanityChecks that makes sure that:

anyDuplicated(lga_data$lga_id) == 0 length(lga_data$lga_id) == 774

assuming lga_data is the data.frame read in from lgas.csv.

cheers, --prabhas

On Fri, May 16, 2014 at 10:53 AM, Myf Ma notifications@github.com<javascript:_e(%7B%7D,'cvml','notifications@github.com'); javascript:_e(%7B%7D,'cvml','notifications@github.com<javascript:_e(%7B%7D,'cvml','notifications@github.com'); ');>> wrote:

same problem on this file, lga_id 82 is repeated twice and there is not lga_id 83

Reply to this email directly or view it on GitHub< https://github.com/SEL-Columbia/nmis/issues/186#issuecomment-43340514> .

Prabhas Pokharel http://prabhasp.com US mobile: +1 347 948 7654 twitter/skype/facebook/whatever: prabhasp

Reply to this email directly or view it on GitHub< https://github.com/SEL-Columbia/nmis/issues/186#issuecomment-43343253> .

Reply to this email directly or view it on GitHub< https://github.com/SEL-Columbia/nmis/issues/186#issuecomment-43428163> .

Prabhas Pokharel http://prabhasp.com US mobile: +1 347 948 7654 twitter/skype/facebook/whatever: prabhasp

— Reply to this email directly or view it on GitHubhttps://github.com/SEL-Columbia/nmis/issues/186#issuecomment-43428901 .

[myf]

i got the correct version og lgas.csv in the branch "water" now. it has all 774 unique lgas so that's good. the debate right now is whether or not to put all the external data (such as area_sq_km, pop_2006 and all the mdg goals) into the lgas.csv, @prabhasp suggests

• have metadata of lgas, such as population and area in lgas.csv.
• have all the mdg_goals data in external_data.csv
• merge with the water, education and health data via csv2json.py

[myf]

since the new lgas.csv is being merged to master, I am closing this issue. reopen if we still have problem with it.