search

CodeforNepal / nepalmap_app

An application that maps census and other official data for Nepal to make data more easily accessible and understandable to the public. Want to help us? Check out the Wiki.
https://nepalmap.org
MIT License
64 stars 42 forks source link

Scrape municipality budget info #200

Open ravinepal opened 6 years ago

ravinepal commented 6 years ago

http://municipality.gov.np/

We need a team of volunteers who can scrap and translate Nepali data into English. The link I provided has content in Nepali. We are hoping to look at expenses and revenues of each districts, scrape those and upload on https://nepalmap.org

ivanyschen commented 6 years ago

Can you add more detail? After I clicked the link, I couldn't understand anything not to mention what data you wanna scrape.

ravinepal commented 6 years ago

thanks, @ujhuyz0110 for your interest. The link I provided is in Nepali. We are hoping to look at expenses and revenues of each districts, scrape those and upload on https://nepalmap.org

samundra commented 6 years ago

The pattern for the budget link seem to be */ne/budget-program for all the sites. All the sites seem to have similar structure. So coming up with general scraper can be easy. Download all the pdf files from the links.

e.g.

  1. visit http://www.attariyamun.gov.np/ne/budget-program
  2. visit links given in first column
  3. download supporting documents
    • most documents are in PDF format.

Note: These documents are in Nepali and may need to be translated.

e.g. http://godawarimunkailali.gov.np/sites/godawarimunkailali.gov.np/files/Fourth%20Council%20Minute.pdf

cliftonmcintosh commented 6 years ago

@ravinepal is this district-level data? VDC/municipality data? If it is municipal, are they the same municipalities that are in the 2011 census? Is there data for every VDC/municipality in the census?

If this is for the new municipalities with boundaries that are different from the 2011 census boundaries, we will need a new instance of NepalMap to convey the data.

If it is for municipalities, we also need to make sure we have data for every municipality and VDC because Wazimap requires data be present for all units at a geographic level in order to display correctly.

If we are only concerned with district-level data, I believe we should be okay since the district boundaries are the same.

ravinepal commented 6 years ago

hi @cliftonmcintosh - they are VDC/municipality and same as 2011 census. i believe there is data for every VDC/municipality. I guess we could move this to https://github.com/Code4Nepal/data/

cliftonmcintosh commented 6 years ago

@ravinepal

Thank you for the clarification. Separating out data collection into the data project might allow people to contribute to that step without having to do the full import that is necessary to get it into NepalMap.

ravinepal commented 6 years ago

@nikeshbalami can this be a project during hackathon?