This is a possible optimization of cohort management that would be closely related to #229 (basically, more attempts at getting away from cohortInds getting superhuge and slow). Previews of downloads presently involve creating the download (which creates a table of structure not wholly dissimilar to that of a cohort) and then partially retrieving it; actually downloading involves getting all the contents of that table and marking it in diverDownloadTables as having been actually used.
Something similar could also be done for cohort creation as a possible optimization - in particular, it'd let us:
make creation after preview much faster (since we have the information already)
greatly cut down on cohortInds size and, along with #229, potentially eliminate it (as a very large table that gets large updates, it's a potential locking concern)
potentially allow for a cleaner separation between Application Structure and User Data (right now it's a little dubious)
Possible concerns:
adding a lot of tables might cause slowdowns in other areas
doing lots of previewing with no commitment will fill up the database (we can have ways around that tho - DROP TABLE can be pretty quick).
This is a possible optimization of cohort management that would be closely related to #229 (basically, more attempts at getting away from cohortInds getting superhuge and slow). Previews of downloads presently involve creating the download (which creates a table of structure not wholly dissimilar to that of a cohort) and then partially retrieving it; actually downloading involves getting all the contents of that table and marking it in diverDownloadTables as having been actually used.
Something similar could also be done for cohort creation as a possible optimization - in particular, it'd let us:
Possible concerns:
Still, something to noodle.