hugolpz / Sparql2Data

Given SPARQL queries, save responses into corresponding persistent files, daily. Served via github page, accessible quickly via xhr.
https://hugolpz.github.io/Sparql2Data/
MIT License
0 stars 0 forks source link

Create command to join files via common `propery:value` pairs into one #10

Closed hugolpz closed 1 year ago

hugolpz commented 1 year ago

Given json files :

./data/LL-LanguagesActive.json ./data/LL-LanguagesGenderData.json ./data/LL-LanguagesRecordsData.json ./data/WD-LanguagesPopulationData.json

Join file by common wikidata property if value is equal. Idea :

jq -s 'group_by(.wikidata) | map(reduce .[] as $item ({}; . * $item))' file1.json file2.json file3.json > merged.json
felixknds commented 1 year ago

Suggestion of working JQ based on these 4 files:

jq \
  --argjson languagesGenderData "$(<./data/LL-LanguagesGenderData.json)"  \
  --argjson languagesActive "$(<./data/LL-LanguagesActive.json)" \
  --argjson languagesRecordsData "$(<./data/LL-LanguagesRecordsData.json)" \
  --argjson languagesPopulationData "$(<./data/WD-LanguagesPopulationData.json)" \
  '[ $languagesGenderData + $languagesActive + $languagesRecordsData + $languagesPopulationData | group_by(.wikidata) | .[] | reduce .[] as $obj ({}; . * $obj) ]' <<< "{}" > ./data/languages-all.json
hugolpz commented 1 year ago

The following was eventually implemented :

jq -s 'add | group_by(.wikidata) | map(add) | map(select(.records > 0)) | sort_by((.records | tonumber) // 0) | reverse' \
    ./data/LL-LanguagesSpeakersData.json ./data/LL-LanguagesGenderData.json ./data/LL-LanguagesActive.json ./data/LL-LanguagesRecordsData.json ./data/WD-LanguagesPopulationData.json \
    > ./data/languages-gallery.json