grantmakers / profiles

Grantmakers.io Profiles - Summary profiles for all US-based foundations who have filed electronic IRS Form 990-PF
MIT License
1 stars 1 forks source link

Amended returns create duplicate grants #2

Closed chadokruse closed 6 years ago

chadokruse commented 6 years ago

Function located here: https://github.com/grantmakers/profiles/blob/source/algolia/pull_all_grants.js

Example: https://www.grantmakers.io/profiles/v0/010522171-the-borman-family-foundation/

Impact: 106,145 grants GCE shows 3,483,322 Local shows 3,377,177 Low priority as Algolia de-dupes thanks to use of objectID

db.grants.aggregate([
  {"$group" : { "_id": "$objectID", "count": { "$sum": 1 } } },
  {"$match": {"_id" :{ "$ne" : null } , "count" : 
  {"$gt": 1} } },
  {"$project": {"objectID" : "$_id", "_id" : 0} },
  {"$sort": {"count" : -1} }
],
{ "allowDiskUse": true}
)