Closed letsintegreat closed 2 years ago
LGTM!
@letsintegreat
Out of curiosity, what was your approach to compiling the data from this API to finalData.json
? I was wondering about approaching this via running the python script that we have in the code currently. I am actually concerned that compiling 2021 data from external API rather than using the script might create discrepancies. Please feel free to argue with me on this and justify your point.
@Sparsh1212 I saved the entire data from the API in a JSON file, read the file with python, and converted text to a dictionary object (newOrgs
) using python's json
package. Similarly, I converted all of finalData.json
into a dictionary(oldOrgs
). Then, I created an array (oldNames
) containing names of all the organizations in oldOrgs
.
Now I am iterating over all the organizations in newOrgs
.
For each organization, if its name is in oldNames
, I added its data in oldOrgs
to update project
, year
, top
, and tech
.
If the organization is not in oldNames
, I appended it in a separate array.
After this iteration is completed, organizations that already had a place in finalData.json
would have been taken care of.
Now, I just need to take care of the organizations in that separate array, these organizations are new, I compiled their data in a dictionary and added in oldOrgs
. Each of the organizations in that separated array was inserted into oldOrgs
lexicographically according to its name.
Lastly, there would be some organizations that were a part of finalData.json
but not present in 2021. For those organizations, I had to add a 0
in its project
array corresponding to 2021.
@Sparsh1212 I saved the entire data from the API in a JSON file, read the file with python, and converted text to a dictionary object (
newOrgs
) using python'sjson
package. Similarly, I converted all offinalData.json
into a dictionary(oldOrgs
). Then, I created an array (oldNames
) containing names of all the organizations inoldOrgs
.Now I am iterating over all the organizations in
newOrgs
.For each organization, if its name is in
oldNames
, I added its data inoldOrgs
to updateproject
,year
,top
, andtech
.If the organization is not in
oldNames
, I appended it in a separate array.After this iteration is completed, organizations that already had a place in
finalData.json
would have been taken care of.Now, I just need to take care of the organizations in that separate array, these organizations are new, I compiled their data in a dictionary and added in
oldOrgs
. Each of the organizations in that separated array was inserted intooldOrgs
lexicographically according to its name.Lastly, there would be some organizations that were a part of
finalData.json
but not present in 2021. For those organizations, I had to add a0
in itsproject
array corresponding to 2021.
The approach looks totally fine to me. I can easily digest it :)
Add 2021 data of organizations, as compiled from here. Update tech, topics, and number of projects. Also fix 2 organizations' old data.