Import bids.state.gov data

18F / fbopen

[DEPRECATED] An open API server, data import tools, and sample apps to help small businesses search for opportunities to work with the U.S. government.

Other

101 stars 45 forks source link

Import bids.state.gov data #70

Closed leahbannon closed 10 years ago

leahbannon commented 10 years ago

kaitlin commented 10 years ago

Links:

cleared and unarchived leads: http://bids.state.gov/geoserver/opengeo/ows?service=WFS&version=1.0.0&request=GetFeature&outputformat=json&typeName=opengeo%3ADATATABLE&Filter=%3CFilter%3E%3CAnd%3E%3CPropertyIsEqualTo%3E%3CPropertyName%3ECleared%3C/PropertyName%3E%3CLiteral%3E1%3C/Literal%3E%3C/PropertyIsEqualTo%3E%3CPropertyIsEqualTo%3E%3CPropertyName%3EArchived%3C/PropertyName%3E%3CLiteral%3E0%3C/Literal%3E%3C/PropertyIsEqualTo%3E%3C/And%3E%3C/Filter%3E

all leads: http://bids.state.gov/geoserver/opengeo/ows?service=WFS&version=1.0.0&request=GetFeature&typeName=opengeo%3ADATATABLE&outputformat=json

kaitlin commented 10 years ago

Hmmm it looks like sometimes the Project_Number is not unique, and has a random value like "No Number Assigned". Need to check this with project owners.

kaitlin commented 10 years ago

also, matching a field with a dot in it won't work because elasticsearch uses dot notation to express nested objects. Might need to change source naming that we decided on earlier.

kaitlin commented 10 years ago

This data is being loaded and tests are written for it. The id being created is a hash of lat, long and project title.