search

public-salaries / public_salaries

Public sector employee salaries
16 stars 1 forks source link

Nevada state, local, school, uni. salaries #10

Closed soodoku closed 6 years ago

soodoku commented 6 years ago

For Nevada, we have a pretty clean website with all the data:

http://transparentnevada.com/

We need to just create one big file adding a column that reflects which level of government the person works for you.

Store script in scripts/nv/scrape_ny.R

Store data in data/nv/state_local_school_uni.7z

How?

  1. Don't click on all salaries. Go to: http://transparentnevada.com/agencies/salaries/

And then iterate over all the tabs, and links within each tab.

Whenever you click on one of the links, it takes you to a webpage like this: http://transparentnevada.com/salaries/2016/boulder-city/

And there is a link called 'download records' up top and that is what needs to be ingested.

You would not want to forget to add metadata on kind of gov. type the employee is working for in the final file.

  1. You can also iterate through http://transparentnevada.com/salaries/all/ It means 23k pings to the website. That is not too bad. Just sleep for a couple of seconds between the pings. It will take about 12hrs?