etenlab / docs

Documentation over all projects in the ETEN Innovation Lab repos
MIT License
1 stars 0 forks source link

Impove performance of map parsing routines. #285

Open ForeverYoung1208 opened 1 year ago

ForeverYoung1208 commented 1 year ago

Now it takes several minutes to load and parse map with words quantity around 100. It looks like word node creation methods not quite optimized for creation of several words, so we need to investigate and optimize these routines to make map parsing quicker.

ForeverYoung1208 commented 1 year ago

I tried to improve performance of loading maps. (Which is mainly depend on creating word nodes with all necessary properties)

I tried several approaches, such as

Also I noted that there no big diference between typeorm inserts and raw sql inserts, so i think for future performance analysis it is worth to look at more deep restructuring of praph services, and try to reduce the amount of queries for high - loaded operations, i.e. make sql queries to insert all property_keys and all property_values at once, if it is possible.

Maybe, even try to find some replacement of the stack sql.js + localforage + indexed db (if such a replacement exists) But it will require additional research.