With the move from sqlalchemy to django orm we lost the use of vertical tables and replaced them with json dictionaries (for variable properties). This is leading to scalability problems, and the obvious solution, using standard tables, will be very costly in terms of memory insofar as what should be a sparse matrix will be stored as full matrix but with most elements empty.
We need to work out how to nicely implement via the django ORM, vertical tables.
With the move from sqlalchemy to django orm we lost the use of vertical tables and replaced them with json dictionaries (for variable properties). This is leading to scalability problems, and the obvious solution, using standard tables, will be very costly in terms of memory insofar as what should be a sparse matrix will be stored as full matrix but with most elements empty.
We need to work out how to nicely implement via the django ORM, vertical tables.