search

ripgarpr / garpr

the revival of GarPR. may gar rest in RIP
17 stars 14 forks source link

Remove/move 'raw' property of tournament documents #102

Closed jschnei closed 8 years ago

jschnei commented 8 years ago

Each tournament document in mongo has a 'raw' property, which stores things like the API dump from Challonge/SmashGG. These dumps are huge and often dwarf the remaining attributes of the tournament documents in size (in fact the 'raw' attributes take up the majority of space used by our db).

Disk usage is not currently an issue for us, but I suspect shuffling around these giant tournament documents in mongo/python is responsible for some of our memory problems. Since the 'raw' property isn't actually used by any of the API calls, we should move this data somewhere else (possibilities include a new mongo collection, somewhere on disk, or some service like AWS/DropBox) and just store a pointer to the new location in the tournament document.

Since #93 uses the raw property to grab URLs for old tournaments, this should wait until that's been merged/migrated.

BrandonCookeDev commented 8 years ago

Are we able to do this now that @Jamesargy6 has implemented #93

jschnei commented 8 years ago

Closed by #126 .