Schema revamp - Githubissues

carmenfan commented 2 years ago

Description

revamp a couple of schema on the scene graph to allow better querying

Goals

update metadata schema as described here
add revision id to node objects

Tasks

[x] update meta schema
[x] add revision id to nodes
- [x] remove current array
[x] migration scripts
- [x] have to get rid of old index and add new ones
- [x] add rev_id to all nodes
- [x] add shared_id to mesh mapping
- [x] rework metadata
[x] 3drepo.io:
- [x] re-interpret new meta schema to old
- [x] remove all current reference and replace it rev_id in scene graph query
- [x] add indices? (or do it on io?)
[x] Update toy project with latest schema
[x] update mongo dump for test to latest schema

mosment3DRepo commented 2 years ago

As discussed with Carmen, we will not index fields >1024bytes. Although this may create edge cases, the current issue of 3D Repo hanging or crashing on larger metadata queries means that it is worth the risk

mosment3DRepo commented 2 years ago

Test Case:

All Groups API request: https://staging.dev.3drepo.io/api/mosment3ddev/b60e7190-205d-11ec-a7d1-f183b9e6b73a/revision/master/head/groups?noViews=true&noRisks=true&noIssues=true

Currently takes 22 secs

Contains 70 instances of the same group:

Expand to see group JSON

``` { "_id": "bc532f40-3bcf-11ec-9f22-ddb5fc69007e", "name": "Test 1", "color": [ 255, 0, 0 ], "rules": [ { "field": "IFC GUID", "operator": "IS", "values": [ "1rK1_1s3_9HPKdro5Qic3h", "0tIzhKQBMTHPXv3GDMn6lj", "0NrvGXEDz_IRmefgz6_DAO", "24cKqMWecvHv$12jcxR00K", "092bHB$VtaGgdNsl1DByFQ", "2uAGTrAopDGR9aFIik_lwd", "00tMo7QcxqWdIGvc4sMN2A", "3DPzOGwfl8IR7Wxd$HACeW", "1JJcbnKp4RH8qDZ0iDUPu_", "2sK94ngIKCIBs1T2UK0MXB", "20FpTZCqJy2vhVJYtjuIce", "3WISpiq1mDJQsdRdOnYg3L", "2N5E0o6j83HOGGt3PPXJ5$", "0NoeABfdC2GQMIcjcxrAoC", "0t1suqpeL2IR4VwAXU7Jrm", "2FjskDayF_IhAjJ0cBOtep", "3jcC8FBS0HGRZTHvEEQkTd", "2uwr8NsZHrJfk8SMsziI57", "26qsiRPQobJw6NY6Xz54Bh", "12YYvOO1qfIeIBSlyjPW21", "0W7cDVldUJGup4tn42qDdy", "2qvd0aiydeJ8DpPdtE4ZCq", "3A8Nux7GtCG8DWVO3m7AMI", "0LTEcw3rZ0Ju5UEM$fp3qa", "1BWcG5lW5zJP3Inzidriv0", "2Sl_OEyrqDHf_6lADZP_kc", "2CmwNMerbVI8c1MKJDUNas", "1_W7Jv__saGevPEhgT9gjd", "0GSHwKBeOnIQ$0uKm0eFrf", "2wn9igZkDoJe6K_MZ5bAXW", "2GOfaAFjopJ8y2KzqB9I79", "3BauYxuXtsIAVT5E4xY1st", "35AfmdirXIIBXOHS0pwr8M", "3TDTH_70KqJ8IKddKKUEeU", "06eWbjXBhmHR_H5k_Sdzez", "3BxVgpDDXFJBAI3uTgu2hJ", "20M2pd9SELGQlF7oBxQk1K", "1VGi6v09UYGAPdhhaOJmnR", "07awbM2jspG8ZSvcORFkD6", "0hGPNYfFuYHBtQO9x94Xjg", "2uMGt5KDiTHvnRrJUB3Jzo", "0cnirL7AxeJx9yDTIx4Ndo", "2AiS8zrGrnJe9CZDA57JC3", "3xf4uqj8hQHfeTRcyTheXH", "2nnyQj4rIIHvPSONxNi$tD", "2OpUpt9JwEJ8kQ3uwObS4q", "130BTvGSlBGPjazx2E7DIY", "3UgkPt2DAbG9Yqj1ub2t2j", "0upxKiz8XiGBohpAxLafyN", "0BgCmwv5HVIeXraWNFxQuM", "32GMG1bsSMGeiCNOqAdN3q", "0xbke6A0LJGRGpadxWBzzD", "3Hjq9VaD5wJhCJkcmsS90G", "3quaZtPT9YGhx$AkPldRxc", "3pobFMmTGOGgjSUzdUz$Rf", "1wppSkY6bmH84_$qmDsn6_", "2TuqZOWqcOHe5f7cCHO9wz", "0pJWdol$KjIRz1gmY0a5iw", "3G7tAkbUrBGOhXFFNxFvJX", "20pc3be0nHJuBg$iX34nBa", "2jOIATJ7cQHPqQT9qLbiw7", "230klV_DkKJgtzKxJOKx1y", "0PX1RvirloJvUkSEj826Pr", "1OsUuY7KO8GvwCvVSrbQop", "0MSwUULeHfHBLeASNd1I9g", "2uEnYYz4RNIuOrDUOmk0a$", "3b3g1HTOpgIhm1llh3K8wW", "2Q0dUJGkXnJB31mBHuu$6I", "1WDTNDhCbCHusXqZNllHnY", "2nj$$2mlMZH8JFDYZtq7zH", "2xihfUqV7OIgv0lY0LQ5Zi", "0U9eruYxhSIeylYuEWE1$v", "2Cfs$HllbpIen3VkRBxAWE", "0P_4d0xKsXHuIqDwhYe4IY", "0Deg006MIjGAs6JJ4sGmoa", "0hvXjyetK6IQRQBhCQQ_zc", "3XcCL2P84kGxcSjl9Omkxu", "3sAJAwGAWRHP8H2OWUrIDP", "0gpfhxCzC8GhHOZPGipYM_", "0Kg2BvK5XWH9HGVyNOeubg", "1LgtCzyfCwIBXxdozLgtc2", "1Oy6oYt5JbJfrlWS5Qqmx5", "2Brj_XRZa2Jg73rTCWpe5z", "0KTdom_iG0JuNVwphVeZwJ", "1V7EWuFBnuIQEgJOnWMQuq", "28GYrpN$WaIAk1iRtPipia", "1F7T3_tKO6HAFv3zKiB79L", "1k8ZP5rykRG9jZZigYv4Am", "3UJMESSWh2Hu2ZWA3E4m23", "1HpDvWEouXGuuSg816dSRp", "24DyolJbNMJ8mBAm1$dIVy", "3STckGyliVIv1Lk4FMGc92", "3ltlXFG9GqGv19VdKH1hA7", "12PhvPm0u3H9PWOOkU1cQH", "1$iBjnKxr7HwMvDimUDNbC", "2ULvwgivEBJeFC0ny5chDb", "3VtT1QiuZ0IA_LS$w8VXSM", "28b45cl$pWIgn0$v4Fo1Et", "3BHgKQmpxzIOazzLg4fL2Z", "2Xpy1lC3lDIPKgEGcBUiXF", "2GszOkEg0OJA5OPQJCGnsQ", "3ezyVAvxqMJuBxwW0Uuwf9", "0yl2_Y$WaxHw2lspSCVKv$", "0N5rIiG1FvJA2f_YsyJfkM", "08JxOpxzVCIA8VYBpPdgNp", "0fHnCQV48yGQUR_M7hjv96", "2GLPX7gQjvGehYPCNejn1Q", "2wXw7__i4SIRUSgtES_h8N", "3lVlu8C2hDJOHxQf8bhz_i", "0s5EpdcnIQIwoP9isAwnXE", "3JBgkqm5nKGusAYfwCUihS", "1UNIEW7PRiGABfHGSXo2qN", "0yyDMwOGLVHBG$q2sRuK1B", "0FZ3l_tCQ7HQNGv$pZCvPB", "3uqf2nepebJu2W$44r2Bjl", "1VZizlmZJpJhdNS2WTGWNr", "29iN49KOZVHO5_viPgobJd", "19dqiti_gOGfxfIyPxWtwB", "2m0aCw3YKRI8_oqG8XOB77", "39dZ_JsRsqHAjFzt1EGhXc", "22$$t4kw0HJ9I9a4A72Gjd", "2oq_Jm24zzGR8g_FjUuu5K", "2etyBHeXCtJPNywmPCuVb6", "09SvUbdJIJHgEUhJqhVswE", "03p5CMWg_iIxDD7K5vKIlh", "07ipTzxl_pHxFnFqHPHlw4", "1GPS7O5sNTGeUA5hymqXRh", "38P8D3QIcEIPy1S7zYNHhX", "18wBVlCPWLI84dhm88WaYY", "3UcJNevbyaGBgrUnXetk_f", "1QQrlXcJDKGvuzkGUqTx6Q", "3ER6WHuGrnJAWH8nebTg5v", "045vEEofGlIeBy_shO6Nvk", "22gfFZZ8A3IhZulnZ2gQUw", "3G$$U2eJVzJeponxqZ632T", "0aT74W$ur7Ifui$FeSKkNR", "3u2McEIorAIB0pzJb_d5eN", "3ipcVn4ADgGAe3RsVHhrH1", "0G6eYb1FULGhtrx71Pr9oC", "2t$cYtvYM7Hf2Rb7Pipvfg", "3Y7B6XyykXHeaFh4dnoar1", "20FpTZCqJy2vhVJYtjuIce", "1LOZnWI94YG9HYDoloNjeE", "3mXUvHUVg4GPYzp687NAUP", "1jtDqEdKCsIxInSscavnfC", "1SoFEnQVTeIOBLM2NOIYH8", "0J_ytboJIYGRJziUHU$KGi", "3ZjXT4H7B4JuAcKLhynDnb", "2VVSSP9DmwIwqbmIr0pSaD", "0yuILoFngBJhxwDnh89mW9", "3uigvfysrTG8nd1AzOw77l", "1ugWbcHAvLGBoE4D5UwK0G", "3E2RbJkxyDJunJ35z7mFzt", "1E6EP2ZvzjGR8ZNcBMNidE", "2M22rrDUnsIv35CRS_2Jxq", "2TcUGaKNrmGw7Eo9pZjPrI", "1oiiDhB2suIe7aiv2iNwOL", "2IzensOgTFGw0nyJds9ToE", "2tk_9cWaMFH9$ziiLVGFpY", "206Wh4i1jjIflFrl_1FN2i", "2KWR5xit_uJBrIqP6nZ72V", "1aYgxzLXCAHOoCRKlCLFQo", "0R1rsfODWtI8JrpY1H42No", "05CqfUZ_pyHeWf5ZwHZa_c", "17tOeFwXjvIfoP7Bo6Zwiv", "0rZi5CRigCIRKuB6ycR16a", "3dp7IF77ReIfRV37K2gIxP", "1D1GRFicmFGhfPg_3qptVs", "2bMfvo_HKBHRgmoZ7$2Zm6", "0ATMfOG5DdJB9Xg3HrR51X", "2DPTBTJFkfJeBPMI96xAoq", "2$LnPEVaU1JOAcgLesU455", "2_rqNOITW3GwkwPSe1ahKY", "3yix2rG43XHOqT9GXIQPcC", "3xgBzQ$9PbGvF96v7vFAJC", "1uvbzrc1xXH9vc$5ntK5qK", "2mEb6b_NEgJPElyPPldAon", "0pOA8tp1sXJfHYUINz4O8P", "3wOJ_VM8nYJOfnXz7rzsM0", "06qTn9RYe0GOx40rdtKQfQ", "1q4PsU6T_oIA9HPDmM3lND", "39mTzUffDUJAgJo_EZX$5f", "3zteXBR3ouGA0FJchvk4qL", "0rwjYcSb2zIvjY4IwneKSP", "2Nc2EnjQzBH8o2XJnrLi4H", "29DLgoePaXJh5_S0gFmo63", "1bF7_IalhyJAzGDY73tJuf", "28A9hCutSgIf2WrD4$6wfj", "05gZO8iwXSIweWEaIiJuEj", "22vWqkrL6YJhJnPrxcgHje", "1YuhrC559oGRPBzz0gVY4H", "1A18E30a74HOmh3yqffMUG", "1JF0Sux_mWJuESEkJV7FoG", "32QIYeSj4aI8FPqnJAhrs1", "1JMKoF2okQH9m4QRStm3vn", "0sMuB54EPwIPLh_BrrrnO$", "0ZS6BIhQXOGQsC3QRSwSUT", "2kWVI6Ib8fIPNwyuS_rZBo", "1iZDxmAsrlI9tFS4LdKAJ$", "3MuTooYfGMGRJv1$RkOJRU", "3oBW5VC_c3HPsPf$6ifUOn", "07k1vIDfSfHg82r1aJDsAb", "2MmyjnpV3oJf_pP7GzmBMr", "0$DsQjTeqrIQzdj9SWmffU", "0ORGDmCQH_JPZOFuW8hEwx", "32IErZm4bmHRli4wJk_iSn", "04sk0cHTyXHQMnuVJFlotp", "0xxTSmsxrSJgnMAokFz52y", "0nsFyOMJxnHh7ULCqr3Imd", "3_hdn0mJ3nJevhGJNpAJKU", "34cqQogCQSIhUCAgkedDb_", "2diqjrVYYQJAQCCeKZ$yNf", "3WoWz4w34IHhHkBn79gdwX", "3E9QOqO1hQIfoo8BFrYPeT", "3ufWKED91CHO4jKs2NbYlr", "2LKMR4MULAI9NhD0wh3grn", "31gOvau$Z8JPJ3WBOwDr7M", "3ai5dm6kcQHO_4qHh23Cpb", "1bjD2HtKkeIPIN$K6wf$lQ", "1FuWeYvAQFIAT0VIGslVL4", "3w3y1uo72yJf9$pkruE4hM", "1HcjHQirgXHhtYulxY9A40", "1_aJxLRE84HA1TAySFCUtG", "2oakMfTuhZIBamcMJwdZzm", "3bYvL3eyXAJ90fzAjU9apv", "1$tAcxqVQLGeASoGsFTiLk", "3tzRq_tbzRJhgWpx1ILj1D", "3DqDqeJRbzHRCr96A2A0Pq", "12q97XBYm6HgirWVP7XaPG", "3giJswEL0CJvaIBwjYCJHS", "2t0HEuipacGvYh29YMQBOk", "3YPIINcBRdHg_szmIU8uwy", "1StMu5AEzvIOY7pAE8CfEc", "0KI8o_Tz36HPleeOCj4d92", "3o8R9gZiVkGhNmZsMThuHI", "3jvf94MBLnHPhc2bEIOHov", "1nzcqr47mVGQrcNsBLOe6X", "0H332wKetjIO10TsIXoycM", "1AHp1_m57mJepirvdKFbjx", "0a0zA16FToJurzyvFPSxBW", "0qAEnKgdvJG8QozQALOWtB", "0y3w6$rAgPH8lXTZbsVPXi", "0PBu3fBhqBHh0hTUsJM0nI", "0csgv17FZmGuzP9oX9aKZC", "3j6Jpo0YTeIRDuznLGeCFQ", "1beFGnICXRHvMSOrG27WhP", "0PzPemAW9HIwhdAg0qeLuC", "0Z3rIt19a8HgSj7B5prOJ0", "3HpudUe0xRHO4drd3Apl_B", "3xNEwVq2trIO$osZYnC7bZ", "0L08t1zm_wIOkvhwVUE8Eq", "23Eq1hbeCxIBl5Um4TNLRX", "2OvcSsXY91HPWEp43IhjJ5", "0VX9JO0mTNIOliK36ng_n5", "0LnWvbnHiaHhawBkqdEitP", "2bMxOhp4jmGhvcrUBzhTmU", "0LgaLkOnNPI8UIcqg53eD6", "1_BgnBN7qNIuIxG9qwLTuO", "2KPN1VQ1vaIw6YnhvsM7f1", "1PpyI4WUTdHARJxrP3yGwu", "0H1mXkaQkIJh$YctKHsTlf", "0SGrli7XTEGgPlXNVmSIE7", "1L0P_808ZxGBzx45_whtjd", "0yr_0lU1ZcHAXdGNGAPCjA", "1txpBwaloeI9RYuDHTOqLk", "2jXjw$Zy3jIA5iTMl1YaKg", "0B97aBkeAuHeytNaMZaPJH", "1C1gxtBeibIvTMzUEs$YP0", "1vzVt0FZvEG9vTrAe04tnM", "0rDKojiid2HA3OqUBOuX1Z", "34tQvmrXt9GPmlzi9PQSsa", "2LfD8aTQA_GO9BvsoVmfN2", "25Voi4SfeyJxlG5L2XQfL$", "2HqlYFQ3kYI8cVkL_SYZO6", "0lGNjcZA4eJxPpOq_jc9Rv", "2H27wSbM20I9Y3MtpFK7B5", "17mlsUaA3vHvF9gDHdHzov", "2BiTWX5MNuJAjmk9fjpU_C", "1rQMnHnGWdGQCLiw_DUqIE", "0$bFa1aedqHehV3wxODiZn", "2Qp58hNitpJ8VYJxfx1q15", "1vTxkFs1LMHBgrvxXuwug2", "2AYHGPypHKGgNHvz6_FIcQ", "1BYJ3n5cE$GBFmDyeIM3Ds", "3e5MoZaWFQIwhoivHTSln$", "2A4ARYYBVSGALRu55pqalP", "2Av9$29whGIuQZ7JK_ZpCY", "3SgTTZAp9tIfMSBW9SNLyL", "26pqd5IJp2GQmFODm0vG3d", "2z5PnXewgkIO8qaXAZ6_IM", "3Cf6BknjLwHe0gv92Oic5H", "3vMOTKHTlCJPk$SG7M82a7", "1BK91zzk6ZGwbHpjjchKT2", "13Z0qFiTm4JAHLNUdrOnVp", "0978DaOYyUHRBALxmJpK6j", "1D2uc15wSRIPCUKHlUnXyi", "3M0ycBa0xgHeLFgvvQ4P_2", "1L43TyiOFjGO2zy_HSl_hL", "3wQNkPhwyHHgzHiwTSx6hf", "0tbl0kX6QZIefj7uI9mZpl", "31zRNPUzzOJfhAdlyEb8GI", "2wZT2J5Qt4HBUy52BZwuzI", "0$2eFTNASBGPdJEH7u_lQf", "339MnZx5M4JxBFpuinPRNM", "0ofA8HUys8HQFIhBiKmRzQ", "3DOGJPIUY_JwjYBx3Glso7", "28fCbHrcCkIhyIvB8FQv7H", "2F4k5Sx3xMJ8d_9tdD9JTd", "0ZV9yaQIxgIvzDAmfSvbHu", "1gRP7xapOGHPS5oQrjR9AE", "0a4gu3DZY8GPAIiQdNvDlR", "0qLqELc3mEJ80dhQTbXJrj", "0cscBDH7kvHgEZxejAXR8E", "3jky_DBDa$JvCA4HsKVQ_e", "1fgH_uv27ZGxtEsaI_Nvx8", "3v7sU8s30SH8dDvJLY0Zk6", "3ni9uYJvWwIPFAO7Wbcr7o", "26Sp5sp_NeHA_1JvdpQv4w", "32VjdRynYsIxD7UvUFIKKp", "3REnbJfyT3JhnUWjFOssCn", "1BVG0ro$FnH9VQ2g$Yt48c", "36WO0mF_P2JerQB7hzRbdd", "1H1Fg67eoGIh_Xvl7vFxGY", "3nKLH7K15dGAvudpLPFnCB", "0$f7eS_MW$GBzoW18c8tq6", "11DAW0jD0vG8Pu5vIHsrV7", "0m$rWNc1gOIvy3BLbDlUlk", "33vS$loK9xIPdDaCmxW9fN", "207k0y3klCHx8Y3iKAhNU4", "1fUhihpd61GQXGuzegNbGH", "0iYK88vqvaJOiLnrcNo2FH", "0a167IyU7eHPBmhR0zfKnf", "2S0Q5_djAkJQ1AB_kDvSNy", "1dXVyzRAdYIRvdmebPEVo5", "2H4q043FB8JvgLVBPsU0D1", "3xk9UTARH5GQRqP4uZJhIL", "1oZ0wPs_PE8ANCPg3bIs4j", "13$iOZNFnCGh44MlpQDt3M", "1pRWJirt7THPsOmQ_bhJS6", "1ib96xdIdWIAJTBvhl2FD2", "21Rctj6pbRGQFD5zQjvY$1", "1KqhkCGUSdHBPZatPadIJx", "3E_pAcK9WMIxyluzWHr_8B", "2kpMk$x3zLHQb7QOSIpJ7L", "21s1vkihZ8Gu6Xit4iinMr", "3RwKJRxNfLIPacWe1sjHpQ", "1wTqWsFdiwGPf3R8RpVdPK", "1LrKMWZoaIJBI7Qzu0zAMz", "0NEVnR9ldKGf3I2KVJWlRf", "2$9NhC5jDyGxjiC4hb4QiV", "0R3yqouA_AHQd3JMnuXjer", "3tcqORzCjgHOUHNK7Kbmk1", "1agmPpp6Y2HvQpzzH_iVfL", "3BxdB1kCYkJw67RfqJznOr", "3T_ilnIKT3GRCcO27w93GY", "1HG17KBvckIPlCZAGwfTwR", "0SAO4oI1cXGep84n$tvzzU", "3HhIhtNWtzIwYXLCloUJSl", "2M77pK9EqWJeCq_48WRjXP", "1S1AfqR0chIwPQnE_oaPxc", "2p3q_Tkq5UIxBJKvqI5DMQ", "3IVXdYdFvfGuy$5OrDfmk1", "3NaGaHtHXLGfbwU_awDta8", "2I3EySkv02JvZtckosCXhR", "2xzops0DCQJ8BCbhoJFTZI", "0g54Sd3Cs0J9UTS7xFrzHw", "2GMJc_CE4mHeIzR49NU8Vk", "251qBlEPKAHOiTxOuSBuXg", "0jX3SeRbilHvyjwRdfmf$g", "1GG6sRzmbzHv1jlpnZo74K", "3cFI_0RthoIQhpz4m0HJqn", "1yYs6ncb8eJw13SPj8AKRc", "2fAoC22UKOIhNfl9YoyPSw", "1wIvhFenjPJwHsk4qC6qnd", "0EBO2hbaM$JA_ngk1smw6A", "0umIaVlL1gJwFzC4rdVijw", "0EAjJgiDIvGeRy$CjugM4J", "0WHuICC7qTG8oNFZ9AvcS0", "3_RbutVE$VGPjtnamuOzRG", "0dpOluMo2DI81UCH00hLAZ", "0eeD6ZsyOMGwfuljnFBtTM", "0UWg$h0DZCIf1UV6LrHGCW", "3QQYl3a43nIRluXjnHr9x9", "00tMo7QcxqWdIGvc4sMN2A", "1UaRjqyMypGR7tZqpmweiC", "1cdTGCQaW4JQQyDXPjP75j", "28J5BfNP4MJxryO7xbNdKB", "1SkEI6JJ_MGQKS0CfGXWRG", "2p1p0j7HdJGx4AdaQj5kJ1", "0Jz_XMa1YGIhtUlEIIjEUg", "204X9tgwJpJRGBwIXjj28t", "0JOJCQI807Hxh7L2t0sqPY", "2cZuepA$SOI8196QJ1m7qT", "2hRJw5YvzXJBT_X401Bhvc", "244uR_3jZfJ8SJHD48RZvg", "1lbS8QujH6GfEcMbbI3YCg", "2Kw1m1mnUzIe_2cSPrGdgG", "0kmULclDF3IgioZLRyE7g1", "2zclV51jifIQwuiDNZU9dX", "3Z10A76a0oIxGfQkrSPpyR", "08kEiBicP2JhlZcU2_4Dxs", "0CujeCA8oxJBULoP$RV1Rk", "3OWWMplO_qHxb_57oyOzTf", "27g_4K3$kpJf57C1us1F8D", "0yRV89jxwkIh1fK0MWjdt0", "1M6rhSzvZRGvjNxqySgNDV", "0w8346lFNlGfjNUSQi$c4i", "2MyNzarHbXJQBGSPPy1ibX", "24uCj6j1vXH9SFSwUKL$vU", "0zNgo$1uM$HBbic0BTFVvr", "198io1y6tLJ9G$fcQVydOV", "0cVqjhfLhhG9te5vSUg24f", "0FYxfasNnxJusbReqWcyxu", "3Gf4orC9vFIfzriagMaOu1", "0rndB2fiw1IB2sTrMQoDKC", "0KBXviXAKPG8ZTo0HNK1$n", "0fh$uoZhKjIxr4k7dl_erf", "0Pzkv8oDumGhtlFoY67e4w", "0kIPXmJufSGOadJHXXpDuC", "0ON1FREMPcIPBkD8Cn_0kl", "3jFjcQrfSsJBJ89nQsAxUY", "2Ja4ooUuuXIgGESA2K7xB1", "0TRXin_RxJHBy35E9fk7K6", "2iZaitqNXEJgNEZqJn6erv", "1OViLfh8NKGOqZBg_OyppZ", "1NGnnNCItZH8JUc1Qidf9N", "07EoEz44rfIPFxuQpL_l0k", "0Ce0svALZ2IvIcaHB8bUNt", "2HG6KlVDXHHf6kGc5rnkEn", "071raLA$ooHuj$JIQK_HYl", "0Paw6JEN$gIA4C_GmYuIUf", "3nU_j1wuqhGfuGWC61J819", "0sc4R1HPd7GeCJIT3J7YZ$", "3rPcvhxt30HR6qyu2z3Fvp", "1VOw4zgXelG9cHCTrvTxVM", "37ZOAvRzlpJ8IbNBGryiH9", "2muvwgp49oGeO8XraftVib", "36XCdOUHccHuG6IjI_be9W", "1DpGjv89oZGQRYY3oCEOwj", "3pnYUQbOwvIxhJsiA0lxRB", "3BkqHTsKp2GQJYscUGvBaL", "2h3bBpWQL6JwFhCFZioGBd", "2k51R3uFblIen7cdEeiZ8b", "0aP$pVBIVKJfFWf7jhgsov", "38ISLpfhSRI8_MCPKsrGTs", "16MqnvtDAAIeuhnYEQF4YS", "0J8onPrhtHJv9X4KoyrDiB", "3lhxeSgbLJHPoa9koBN0Ll", "0iiaTUIihrJ8e3clUgPVtK", "1R3Ysk4CDLIu7faV$Sx2wX", "3tEFFsCO0RIurt7AaoxXZM", "0PcD0wDCndIQIpSMAGT89j", "1msuLGoYeqHu0miA6z9p6P", "2KKDM$zIPTG89MKT2shx_l", "167gtFZ$XOJuEC9vY_m2B3", "0Hb5QDupxzHupDb7xLviv_", "0sjF1rehLjJ8uyhbskYGya", "07Km4jCxhaIeP7PKElrgzE", "0_tNfQPcr2J9MFPUN_ufux", "2WfeV$YJ6GHhIQrjX1ZGrD", "3$Z1L9bt5EIALNgBFTaih7", "2y_ErEgUNmHQOXn5b6SXjr", "2BO_AS_pRtIPZE5D1$ud2z", "0RoktMLMcuJhCzGzRffohL", "3LpzoNc3pqJvJmCnC0T89R", "3v7PNQoWEcIgbgKCeIseUO", "3W6Lf0eqI8JgRK7Ncu$o14", "08L5zIH_1hJx9pkNWzmRMi", "30AFu7TUmnIPxQ5kwV5uoN", "1d$u8XVBE0GwqdZx20$Rqa", "0DiLp14pFUHR7FLVqW3X3d", "0GY7s1wKl0Ihli7z0cFcbA", "1vBjAZBZnfIPT34Xe67w2C", "0VYtQ$GzXnGQ1Axq8dhqJz", "0uHr9iEud9JPn4a438QAA8", "3kp1RmTvO9HP$vi0_7Ar0O", "1TW5cy1EE7GQkx7H_k2Ddr", "0Q8GiUC7aZIxA1Bbh5KtSz", "1OAew5Ij8xIwKXcR7tsgRs", "3sSnmPHu6EGOJuLZHag987", "0jWLzoxVXdJeyPDZFdYy36", "2Je3k2DQABI8WcqhauWcKw", "0HU6rdOaolGhyVw5sNgqwD", "0lUQx0y10CH9ziRNI0i9tz", "3aUBd4SXJ7JgczuCmXN4rm", "0yz09RJdZrJRJpC8cc3jlN", "0ubej8FiaDJO44GtJ_WcWe", "3dmOBXSuxqH8wkDxzPQqUg", "0U6uQA97iwHvUN8hEi50DN", "1vPfhhGyCGJhTEO3AXTP7I", "2ExxUwYDrwJPcg25F$YYta", "19O7iB3DWfHRBot4ySvWBQ", "1ChZDUebT7Hum$mSn3WylK", "3_0tIrpGUpJv6qWACHscUH", "1oD$gWKYRJGBgiHrpCJ$H5", "1SmE8n8ea$HPmzezs9yufv", "2hzuxKv4VPJQJGr$choCbH", "0m$xz_82_uGvinj88ghm8T", "20rAo0cy0tJguhqzjv1L9m", "0NpUge$5hHGPD3Bsm7VPmc", "1yK85tZ4d8IBaP9EO4VuZ0", "1BoOOAFGLpGRinTfDBX_EA", "3b02Bhk1PKG96KDI3c8l7_", "1eAcTM2Q8KIfsInQIqJ7YD", "0a4sdXcLQgGAihLxwfKx7R", "2R9qA79UEtIh7rdr2GHSRh", "0u9jpuRkODJ8svRUFhKLD7", "0c2eljbsZmJxGp075l0$WP", "3Wlp31jQ10JAxEFlW$K02w", "0OeO1fEucKGxaVO$yAAH7h", "3uMm$g0hCDJ9IerReb$xqV", "3Xy2UoCzrHI9mIMcx2m0oQ", "0OGA6WztB1I9FoLXMmEmLc", "0$YHPrpvOrJQfuDICQszWd", "00pxmdieXsJvFWaR3YMamx", "1wvlVheydmHQ3dFr_9jdY8", "17nSu6sT2CIeMSh829shWg", "34DSubc$t2JBsglXewOukW", "0q1n$MMf5uGerIPs02asWh", "3nuNoWrNR$IBrCdOwMt3u5", "3AY55BMmuIJOf8u6mrkcST", "2lxY1t2TrNGwAe$MqGQaai", "1vM934FsN9HPYzsDWLubOn", "0seJMzuY9bGvbPXPfbcKZv", "0ucX5pG_2gJ9mY_3cMXYZ6", "1oyMGrBrcXIeM0J7M4yP25", "3sp9R9QqfKGBvLAN3SFlY2", "3raMRoFKXgIOZ8AzcQBPRC", "3CyNN3n4$6GBLNDHtDBTnj", "09ae_mS3GCIe7kaQf8rplv", "2R_fFYNnu1HhVv9qcYoAxn", "3AwPBBCT6lJvgtVWUGBxi0", "1Ss0JkuSmkJO0gspoGbX2c", "3dv__GNz5_HRnROIcnlEbo", "2l0RezIdvPGPvfiJqf96GO", "0pbbVBz$iDIAKfe2t0U0YX", "0ROWIwTa2yJ9riNUFwh4po", "2OI41TsREsIB$yvCQKs13b", "3lOfFmH5vtJRIPP0Wmww94", "2BSHa$JyZ2GRTu8gJ2CrY7", "2Tj$S2N7YZHgFPGgyG5P1u", "3PDoY9bXd$HRcF3wH4D2dW", "3k7qw55rd2G9jrWVuHOIvB", "2xuYcHVHwjGgkHZlhT3iLQ", "2pkmw_ok19Hwifkv6U0U$0", "04BSt1cZWaIuQppTwvax8H", "1HwsPiapcDH8SJuheHy2Pv", "0D9PuJlUC2HuKkomsUkIwt", "11AQiSiZKBIBYP9JLZ_YwO", "27FjVdCggAHOC2el_zaXrt", "1aOJxrWM5kIgR3OEGUi$WU", "1OYjOnBb_JGhmwv459S3k$", "09UbDVAQ40GeOjRsogbdju", "0RExv3UeNhGwEYT1EiBuJ$", "27GUoBptbhJB3LAjP74K9R", "0m9tXzU_zCI9zVgossUXDZ", "0rtzyPUiMSIv04mDpt_fKk", "0W6__x50YqGQlbyiFtFarx", "1Y8x3$VaS$I9_wZQiAdTwe", "0n6dfMXJFqGuPVmU3NKGkF", "0mTuHvzfuIGf3EIHGTKrbT", "3xJsYKrYNQJx5Dm__Nyosc", "2F0V3QPDDFMRTpJqxqPeZR", "0arx$JTfTGJ9PSsBM1FIIp", "1qipXZC$VFJuXGajTrCv3h", "3O16CH_jLQGPokk$dXdyWK", "1COQs$cKC9JwRMesKbzN5W", "0v4ma49yZJHAnBKQrqmCSa", "0kWRT8MjM$JeTdLhJ5zmtM", "16CU7Um3OTJBLxDeV3SMZI", "21H6aDSltMJ99fDFWj9eZY", "1tEOL$wSkDJANM6dqTObTs", "1Vwh6H0RTmHB7niaQ$z8ZJ", "3cQHvgvbIpIhxTXYamUsPW", "1i$rRga2I3J9fUWxJNIan9", "14Jfu_Lb$NHuKVuXBvGkLN", "3v5_scPJEwGBIorhxFsnZs", "1hy1isaBBfGQT0oIM7uGy1", "2RjlBga3ejHxtlMCXL7ZbE", "2_4pOfCLIiHxfZdOSUjKPZ", "2pk19xAukMHfNpngZK0giu", "17IpwE0PwNJOnBqKu1mtoi", "1oKv_WR65zJxH1S3WOVzyt", "0X8S1vfJIDGv0TeQiV7NV3", "3Wye_xdmssHwm_yZ2oFwtt", "2k9sgS7qDbHPxDDoTSDwcC", "2f6EDRCRynIuXuqfVyk7g7", "1R2lXeG03$IfKF356sVtJb", "3hEr$1_SfhH9sWiJSCbX3$", "0LIeyARIzaGeBS2G5c5amD", "0m9JFtvaMuIPzuTJY3Ivg6", "2Oc6vmluBSJg0D072GKcLx", "2L9GjOGafAHuBDaI5zTXf4", "17CKeNn3m7IQxU2NIb6Ykj", "3hue$sg1M$Gxd061MMiEWp", "2$plTCygz$Hg8QHup94RHR", "2sL6QRv__3HwQv629_yO7X", "1dcZbG5$h4GBWhoOXPZmRF", "3O$ka3lHkYIRKSDSO0b9Bc", "1FIzJyAW5ZHuMn2oTTH$HG", "2w81Te$loyHOXQLmdq8e4R", "2uAGTrAopDGR9aFIik_lwd", "2YLgjI$vpkIe1439YO27Wx", "3kxDkgiSAjJAgpduumT$0P", "2JTd3$BT9YHuK8W7eCdVCY", "1RE$6lkLzaGPiSMo7tZlb5", "2$0_TDKcPdHO87mhhwg$4C", "2S1kREl8a3Jg04Fdb9JW2k", "1BBFXNKnG3GenTNtkMhAEz", "3rY3UFg7jGIOWMw7OtNSes", "003P9UyNMjIem17XaZTuJu", "1jYAroFiBYJhijKwVX4110", "0Y6kZw4u7mG9N9xSZ6yfg8", "1rdW0vviHjJfx$KEQFH_23", "3Vj0csGrQFIPGY2r1W33PN", "2g$e9LbAdvIfhCV88OrRtS", "3RKU89BnlzG9pIdsh_zDJM", "1lLsdSyy_CIhd9By0t_Jhd", "2MJNLHxVbMGw2x3hSaD45w", "2QNITVd5XEH8GslAYJha7H", "17jzZjHXn0Ih3$NkQMhXcH", "0TIGXtV6Q8GOGoxVKouB5R", "3dBe27Vy0KIvF$0CdWmpol", "2m9FNOOqDsH9h3mmwTDYmk", "2cVqbGcucrIetcNF1mOIqI", "310PRnZ7SzHANSget5VDSG", "3kOFHflor$JBED7IkMdIVS", "0KZppEr8goGRXTgSOjDA6p", "24uLdm0$wIIQc7$qRtqrEF", "2mVF31zTgVGOfqkELObnDG", "1q3_dTJak2GvRuneqXAh99", "2e_CNTkcnuIvC4_3zlKXD6", "0EJseoOVE7JwXjbx6UL1Cs", "2gizt6VcaQJAWTPPN2rNyv", "3bgvJ492haGPJ2aEer1ory", "0gzMK1Ir4nGAi7XmJCNSUJ", "3bn9uMCi2YIRxeRG_zEqO$", "3aK02PB43XJhlnqhkXOY76", "08QPz_CDZoIv2oXILbRxBF", "3SfNZ$Gea4JxX6sbvVIqnr", "0L9SNk6nDCG8nElzV5wfDi", "22zsMbip68HeKyu_8nJ_P1", "2U4iyGC9aaGRexa8qT$e6V", "2enmXP5GCSJQE94xPa0WUA", "3JzYmvo0doIf0Nrtk01bHU", "0tfvOBqXSHHg$hoAWXtQ3K", "2KgAGn1TjEGwKaomd4XAB2", "2Hr8Z7oU44GvHPFlLwPARY", "1I$40zLyC7GBZ_$5dh0w1i", "1IKuuRMTBEJPs5N2Fxi8CQ", "3IV656GviDJBwTU0v7DUwF", "1XdA8AKFqAHvA6E9LdGwAL", "2oghcwwalJIOkSmBjnJdXP", "3qKCkKEeumGwDFTYEI_hyA", "1ZcSItLQ4fIeWUYWrCvR7H", "28weao_oL9J8ukBP1W0OGi", "3LwloLvmioJQH4RPiC8tyX", "0EoJIfDHDlIPsq9xTo5PyX", "1PEktnuOksJ9xh5eHd4exl", "2YlLsUc63SJglIFc_R5qqo", "1CrqUK6jr7JgmcQMP4KOSJ", "0tfuAc7MlnGgFdwqvOEEgE", "0f8QktFzARHO0dQi1jGiDt", "1S_ELflgV8JObVghESC3I9", "1RZvjZ06l6GexZW2FIDUCk", "3wJvcP2TMyGALPJr4Fr3KS" ] } ] } ```

carmenfan commented 2 years ago

https://staging.dev.3drepo.io/api/mosment3ddev/b60e7190-205d-11ec-a7d1-f183b9e6b73a/

fun fact - when I import these groups to local database and run it locally, It comes back in ~ 2s.

For this particular case, the overhead might not be the queries... 😆

carmenfan commented 2 years ago

Test Case:

All Groups API request: https://staging.dev.3drepo.io/api/mosment3ddev/b60e7190-205d-11ec-a7d1-f183b9e6b73a/revision/master/head/groups?noViews=true&noRisks=true&noIssues=true

Currently takes 22 secs

Contains 70 instances of the same group:

Expand to see group JSON

now that dev has upgraded to to v5. I've just tried the query again, this runs in ~5-6s under the new schema

3drepo / 3drepobouncer

Schema revamp #514

Description

Goals

Tasks