search

ActoKids / AD440_W19_CloudPracticum

3 stars 1 forks source link

Mapping data from crawlers to data model #20

Closed mrvirus9898 closed 5 years ago

mrvirus9898 commented 5 years ago

As our crawlers pull in data, we will need to send this data (via API, or direct server interaction, to be determined at a later time) to the designated database. A common nomenclature will need to be established to guide us in how we map and sort the incoming data to the database. Furthermore, we will need to work with the API team to ensure that our data being sent to them is complaint with their database or API (Ints mapped to Ints, Strings mapped to Strings etc).

Related to #2

Please indicate the time spent on this, any issues that you are having, any good references you found for this subject, and credit anyone helped you out.

mrvirus9898 commented 5 years ago

2 hours committed this far exploring the synonyms used by organization websites.

mrvirus9898 commented 5 years ago

4 Hours committed to further looking into organization websites, and creating a spreadsheet for tracking App, API, and Organization specific mappings.

mrvirus9898 commented 5 years ago

Another hour added updating the key names to match API keys: https://docs.google.com/spreadsheets/d/1I0o9Spry_ulR0ebbPX-3oiljenpBIlbqa8P4gyJZPns/edit?usp=sharing