search

linnarsson-lab / loom-viewer

Tool for sharing, browsing and visualizing single-cell data stored in the Loom file format
BSD 2-Clause "Simplified" License
35 stars 6 forks source link

Shorter URLs #91

Closed JobLeonard closed 7 years ago

JobLeonard commented 7 years ago

There turns out to be a library called lz-string that has this function called compressToEncodedURIComponent that, you guessed it, takes any JS object and converts it to a base64 encoded URI safe LZW compressed string. This might help with shortening the URLs for sharing purposes

JobLeonard commented 7 years ago

Preliminary tests indicate that it works for very large state, and even then only a little bit:

JSURL:

http://localhost:8003/dataset/cellmetadata/Ventral%20Midbrain/Midbrain_Human_embryo-repack.loom/~(heatmap~(zoomRange~(~8~15~23)~fullZoomHeight~4999936~fullZoomWidth~506112~shape~(~19531~1977))~col~(order~(~(key~'*28original*20order*29~asc~true)~(key~'Cell_ID~asc~true)~(key~'Cell_type~asc~true)~(key~'Timepoint~asc~true))~filter~(~))~row~(order~(~(key~'*28original*20order*29~asc~true)~(key~'Gene~asc~true))~filter~(~))~cellMD~(searchVal~'))

http://localhost:8003/dataset/sparklines/Published/cortex_repack.loom/~(heatmap~(zoomRange~(~8~13~21)~fullZoomHeight~1279488~fullZoomWidth~769280~shape~(~4998~3005))~col~(order~(~(key~'*28original*20order*29~asc~true)~(key~'Age~asc~true)~(key~'BackSPIN_level_0_group~asc~true)~(key~'BackSPIN_level_1_group~asc~true)~(key~'BackSPIN_level_2_group~asc~true))~filter~(~))~row~(order~(~(key~'*28original*20order*29~asc~true)~(key~'BackSPIN_level_0_group~asc~true)~(key~'BackSPIN_level_1_group~asc~true)~(key~'BackSPIN_level_2_group~asc~true)~(key~'BackSPIN_level_3_group~asc~true))~filter~(~))~cellMD~(searchVal~')~sparkline~(colMode~'Categorical~geneMode~'Bars~genes~(~'Tspan12~'Tshz1~'Fnbp1l~'Adamts15~'Cldn12~'Rxfp1~'2310042E22Rik~'Sema3c~'Jam2~'Apbb1ip~'Frem2~'BC005764~'Deptor~'C130030K03Rik~'Klhl13~'Tnfaip8l3~'Ascl1~'Atp1b2~'Tmem132e~'Prkar2b~'Necab1~'Nr2f2~'Stmn1-rs1~'Shisa9~'Sub1~'Vat1l~'Pcdh18~'Dpysl5~'Agtr2~'Cxcl14~'Ptprf~'Cers5~'Nacc2~'Cntnap2~'Asic4~'Tph2~'Rit2~'Scg2~'Nr2e1~'Bmp3~'B3galnt1~'Gm9846~'Kcnn1~'Fam19a2~'Samd4~'Stmn1)~showLabels~true))

http://localhost:8003/dataset/sparklines/Published/cortex_repack.loom/~(heatmap~(zoomRange~(~8~13~21)~fullZoomHeight~1279488~fullZoomWidth~769280~shape~(~4998~3005))~col~(order~(~(key~'*28original*20order*29~asc~true)~(key~'Age~asc~true)~(key~'BackSPIN_level_0_group~asc~true)~(key~'BackSPIN_level_1_group~asc~true)~(key~'BackSPIN_level_2_group~asc~true))~filter~(~))~row~(order~(~(key~'*28original*20order*29~asc~true)~(key~'BackSPIN_level_0_group~asc~true)~(key~'BackSPIN_level_1_group~asc~true)~(key~'BackSPIN_level_2_group~asc~true)~(key~'BackSPIN_level_3_group~asc~true))~filter~(~))~cellMD~(searchVal~')~sparkline~(colMode~'Categorical~geneMode~'Bars~genes~(~'0610007N19Rik~'0610007P14Rik~'1110007C09Rik~'1110008J03Rik~'1110008P14Rik~'1110017D15Rik~'1110032A03Rik~'1110038B12Rik~'1110059M19Rik~'1190002N15Rik~'1500004A13Rik~'1500009L16Rik~'1500011B03Rik~'1500015O10Rik~'1600029I14Rik~'1700001C02Rik~'1700001L19Rik~'1700001O22Rik~'1700001P01Rik~'1700007G11Rik~'1700007K13Rik~'1700009P17Rik~'1700012B09Rik~'1700013G23Rik~'1700016K19Rik~'1700019L03Rik~'1700021K19Rik~'1700024G13Rik~'1700025G04Rik~'1700026D08Rik~'1700026L06Rik~'1700028P14Rik~'1700037H04Rik~'1700094D03Rik~'1810011O10Rik~'1810037I17Rik~'1810041L15Rik~'2010001M06Rik~'2010107E04Rik~'2010107G23Rik~'2010204K13Rik~'2010300C02Rik~'2300009A05Rik~'2310001H17Rik~'2310002F09Rik~'2310003H01Rik~'2310015B20Rik~'2310022B05Rik~'2310042E22Rik~'2310057M21Rik~'2410004P03Rik~'2510003E04Rik~'2510009E07Rik~'2610002M06Rik~'2610008E11Rik~'2610017I09Rik~'2610034M16Rik~'2610035D17Rik~'2610044O15Rik8~'2610203C22Rik~'2700046A07Rik~'2700097O09Rik~'2810047C21Rik1~'2810417H13Rik~'2810468N07Rik~'2900011O08Rik~'2900026A02Rik~'2900052N01Rik~'2900056M20Rik~'2900060B14Rik~'2900093L17Rik~'2900097C17Rik~'3110035E14Rik~'3110043O21Rik~'3110047P20Rik~'3110099E03Rik~'4632428N05Rik~'4632434I11Rik~'4833424O15Rik~'4833427G06Rik~'4930426D05Rik~'4930444M15Rik~'4930451C15Rik~'4930452G13Rik~'4930523C07Rik~'4931406P16Rik~'4931428F04Rik~'4932438H23Rik~'4933409K07Rik_loc3~'4933412O06Rik~'5031426D15Rik~'5033406O09Rik~'5330413P13Rik~'5330426P16Rik~'5330434G04Rik~'5430435G22Rik~'5530601H04Rik~'5730522E02Rik~'5930403L14Rik~'6030419C18Rik~'6330403A02Rik~'6330403K07Rik~'6330415B21Rik~'6430548M08Rik~'6820408C15Rik~'8430408G22Rik~'8430427H17Rik~'9030619P08Rik~'9230112J17Rik~'9430020K01Rik~'9530080O11Rik~'9630013A20Rik~'9630033F20Rik~'9830001H06Rik~'9930012K11Rik~'9930013L23Rik~'9930021J03Rik~'A030009H04Rik~'A230057D06Rik~'A230072E10Rik~'A230073K19Rik~'A230077H06Rik~'A2m~'A330021E22Rik~'A330048O09Rik~'A330049N07Rik~'A330050F15Rik~'A330076H08Rik~'A430035B10Rik~'A530058N18Rik~'A630001G21Rik~'A830009L08Rik~'A830018L16Rik~'A830082N09Rik~'A930003A15Rik~'AA465934~'AF251705~'AF529169~'AI413582~'AI464131~'AI504432~'AI593442~'AI607873~'AI846148~'AI848285~'AU021092~'AW112010~'AW551984~'AY074887~'Aaed1~'Aak1~'Aass~'Aatk~'Abat~'Abca1~'Abca4~'Abca8a~'Abca9~'Abcb1a~'Abcb7~'Abcb9~'Abcc3~'Abcc4~'Abcc9~'Abcd2~'Abcg3~'Abhd12~'Abhd17c~'Abhd3~'Abhd4~'Abhd8~'Abi1~'Abi2~'Abi3~'Abi3bp~'Ablim1~'Abr~'Abracl~'Acaa2~'Acadl~'Acadm~'Acat2~'Ace2~'Ache~'Ackr3~'Acly~'Acot10~'Acot11~'Acot13~'Acot7~'Acpl2~'Acsbg1~'Acsf2~'Acsl3~'Acsl4~'Acsl6~'Acss1~'Acta1~'Acta2~'Actb~'Actg1~'Actg2~'Actn1~'Actr1b~'Actr2~'Actr3~'Actr3b~'Acvr1c~'Acvrl1~'Acy1~'Adam12~'Adam17~'Adam22~'Adam23~'Adam33~'Adam9~'Adamts1~'Adamts14)~showLabels~true))

JSURL + lz-string:

http://localhost:8003/dataset/cellmetadata/Ventral%20Midbrain/Midbrain_Human_embryo-repack.loom/H4CgFgpghgLgtlADqAXgezXASlAdgcwlGAA5gBGAVmACYBmASmADMBXAG3YC0M4AJCAEt8YGMAAsATmmS6ANhYduvAOqCAJjDDBKABjnlyNYAGcwSIiAqTKdctYDsDhkwDGadqDQAndRG-EIADWEACewADkAFQ0JD7CgrhQ7DG6Pn7eMZLAUCauwDDerBBMwWGRAMIQnAD6AJIAIjl5BUUloCHhEVW1MKGIRLn5hcWlnZEAKoJwEIhoiWJDraNMzILsMP7ELsDeaADuXr5bVmVdMXHeCUkpNGnHmTTZSyPtZ5EA4hC4gy2vO2sNidgDtXNV2ABZJogEzQbyuMAANWSkRcQA

http://localhost:8003/dataset/sparklines/Published/cortex_repack.loom/H4CgFgpghgLgtlADqAXgezXASlAdgcwlGAA5gBGAZmACZyBKYAMwFcAbNgLQzgAkIAlvjAwKNAOwBOACwkyrDt0wB1AQBMYYYOIBskmiQAMwAM5gkREMGmTJZSocMBWRmthQAQmha41J4gC05IYO0sABlJI6ZFQ0UaSSuowATmgA7gCyaGpEAOQAKhAAHqIAxmhsWTnABcWi6JjAksClELgwEMmgbLDhhgB0Dk7OlOLBkqNOwGwEwAPkJJTSwTrS0k6U5DQujOVsoGjJOV1WIADWEACeNQBUBodCArhQbHeGh8d3zVAmpcAwyRYEEY5yuNQAgoRgD8-gCgSCLtdch4oKUzgBlAAKAEkAHIAfTYEAAbhA2PjDPj8KkWMgYf9AcDQIiaii0Vi8YSSWT8eQqTS6b8GfDmWDkaiMTiCUTSeSaPzvILYYz6IwmAI2B0TsBVcBUmkDkdOsRQUi7iQHvgni83h9Ol9oUK4UzTayJRzpdzyZTqYrHcqRa7xeypVzZbyFbT-cKXSzg5LOTKefLfVH6c6EWK2QnPeHKJGlTHderNcarLrWhwMgARUAmaDJUpgABqLxqjBMiCgyTObCelj2VTyAGFYBB8A9Sm3CLgIEO3cl-DOIP4rAVO3gtjV8mYUOQagAxXAAI0Q5H2uXBbjgMBM5CmuWHbDUuC3uSwRSYZ5qNE2jmkNAAKI0DQWACGcNTohACCUH8uQAFJQHANAQogx7HuQAjILkB7JNBKHIsOjhOLoYS5NWECIDAhw1MOVCOA4ADSIRgRBuSMWwYBsFQ264EwUBYSQbDUJevzcRCMBnseBH5HA0GxHkmI9t2NDHjUuIQFOGHqckNBMAR6LwK+ASLvuuTomAAgmFAzTmSw2m5K2MDnjUmKlGoYALDU1aIJcJhsA+kIAgRw5FKU3FkZiknJEwtGdCYD64qipQhe0zyIAR4ImAIpRkfkiBgARYEwAZpT4ARuK6RAZkeHAiAiR4lD4C87RmQA4nAdjSDoNSMaUuCvoeSHkJIUAGUhahkYZcCvh2YDpAAMlAx5kv4GbTHgfhTogA4YEc1odOCMAAquNT4jA6K4oBZnnZdwG7HtagHRAbVtCuxC5O20xoPgvwvJYjAAFYCMdZZqhqWqcJ0aCrowPwZLAyQCEUzAvPWLQVIcR0Arc9xI1azyvDQ7xGskDp7Icr2zjUGNsIc865PwsAIIgjDLjWdYNk2rYXmzb1-TtoDlB8z3Y4uH23VdN0XVdNCMHTv1TkSoBAyDWoq8wEOdFDqSw46CMAsjqNsOjFPJGLuMWvj1pEyTnxxLT9PZHkTPwEgqpAA

http://localhost:8003/dataset/sparklines/Published/cortex_repack.loom/H4CgFgpghgLgtlADqAXgezXASlAdgcwlGAA5gBGAZmACZyBKYAMwFcAbNgLQzgAkIAlvjAwKNAOwBOACwkyrDt0wB1AQBMYYYOIBskmiQAMwAM5gkREMGmTJZSocMBWeowDGaNqDQAnNRB9iEABrCABPYAByACoDXyEBXCg2WMNffx9YyWAoEzdgGB8WCEYQ8KiAQUIcvIKiktBQiMiAISg3YIBlAAUASQA5AH02CAA3CDZBw0H8HzQWZFz8wuLSpqi2jp6B4bGJwfIZuYWa5fq18tb2rr6hkfHJmiP5xdqVksYmATYYAOJXYBzADu3j8fysZWasRI8XwiWSqXSASypzqq0al02Nx293201mL1R7wuzSx2zue0mhwJJyWaIakI213JuwegyeNNeZ3RjKuW1urP2lGetLe50+31+gSsALcEzYAFkACKgEzQHxuMAANWSUUYJkQUB8wTYiUsHkVaH8UQAwrAIPh4m5dYRcBAFVaiFcfCZgK6IL6rJFDDpyI5DOJ+uRJFgBMEoiGw45xN1yNJY-HIuRs+HxDbDDG41Fs0nDCQAFKGSgZ4s5xwkVPpotZuvkcRK8hOGstpOUGgVKvdkuOSgkFrkGhDutOSQK6NTyThmhRrvNzvhwzSCpUIdODeSAAy5B0u-D2Zag7Xe8cnYA8mGhzol5Jemmh+IN+R85O1x+z0fC0zNtP1vGgfyAv8b26QxyHfDdxAAcWzODcwAaR3X991TcQUJgmgL0A4tIKoBCaGrTCbx0dDCKzYjD0vCCl3IajcJoaQkPIxjHBoJwEM3VidCVMsBIPENWIbN8KKrcReH4qSZCEzjixIJNs3vQwhxUkdxFfHC1y0zdyCPVdMxoGCzwVMTmzMsMw3EABROTTPMuzSKUyIbMMMzpHQ9zPIcQxv27Mj9wHEyojI0tyF4NtgsoUsaAAMQLOLS0oWTYOs+KbycFozNS7j8OcArNxoeywJKpxxAVOhgukUtpGgvynDSxym1Mlrw0kRy9NM0Ml0sk9rP6+t7OQ4bVJ0lKJpHaQ5yGvreycDteoikbpGke8TLIDz+rMygbQq6zIOkHQB1WjzIMkcRb2m0yDOkPNarjcgIq0+qZIw+6w1Okh+gjYLFxvchbpIQGlzOrzwccJxlxg6HnB0GqNOsoGQ0Mcd2oitHJEoI8LpoHG81i5t4qWsasciMnHGkShQMyzNqc3FN8tJutbEc9zTr7aQDH+8LIm5tjKGkV8GaiWRKBFtitu7SXpcQqzMxsBxecE4rmxVzcNrnAWtekFqbU7OXcc3WGOJNhxYYOgHNdxtMQ1TBaJft3mSGSymVeFkheDIy2RYLVDbeCYY0DcahBdxkWJ1u53Ij3eK1Y7AWE4DnRbpopwpcMyhU3crPVZoHQne7AvNxFvjKacWny94o7MycLP0eipyoiqq2wMc8C29N6QqyPSmn1V6MjbB5sdGzvvKAHbvIgn1WqyDi758M3LnszHQa+rkhLLHjeSG8ssjYFkga77khSNn0-C8+i7FwcUNJGgveon0BxsxocsSczGQArMoPxaREkE3Mshh7yAMkBPG809WY-ygVWSgiVYGvxIAFGCsk462AChOdCEDTYwTxn7ZsWDuLkErO5AcaDJCyUphUEKzh2xK0qPQ8QZUHzNjoQFcQlAWIcJYTJJhkQ6FwEqNnLy5ByqzwqGI2QGduzSICjYf6F0FEw0MIlY2HCxG6Fki-IRNcqy5XYZmCoICnB-XIHoio8CYKkUARUVBXVRJWMcTBEgR444OICgff6NEKj4KrNuAWFQKinRnCLSoSCWoficJE2GkhjzZCEb0eqlBzE0EqCkzeVBXrJL3BtPsmTwkbQyckp84gSDcMyafUMshqmyAMLEoRABVcRBZSkVGUB-cylRlCN2jKfSoABNCMshKmVCgBANQuSKhQGCDM3IvohGwBMQAI1gJUVZzoZlbKgNITZzoSBQAOVAJJFQtmrPIMcoRFzxAHNWWcrZ4cDluH2TctwbhHluDUB0rZ+AI7nLANM35QK2z5BuUCgFqygVvMBWoHa5yBA7IEL8gQUK0WrOQDc00cAdmBBuT4doXghHOigB050ahiUVApSIklsByUQHJZASoHQfAArcGwZo1K0AwDDCynl2Z+W8vZTyu5JLEBsHJSYVZ+AZluBMEwKVbB2UmDYLC+VbAdAspMCYOVMAoB6rJSymAqzjWyrNeSmAuA9U+HIKaklhRLVsuNWy+11LRi2vBe6nwbA5VhBmWoKAuKOmBtxWKiooawKVEjQC0NUto1BrOaGmAuqE1wBTWmfUYA0BAgPFAVZExfTEnoEAA

JSON + lz-string:

http://localhost:8003/dataset/cellmetadata/Ventral%20Midbrain/Midbrain_Human_embryo-repack.loom/N4IgFgpghgLgtlADiAXKAXge03ASlAOwHMJUBtADgBoBGAVioCYBmAXSpADMBXAG14Ba2OAAkIASyJgYqACwBORfOYA2Dj35CcAdXEATGGFR0ADCpo1GHAM5gkpFGRry6zGrXkB2T6wC+HAGNMXlRQTAAnPQhw8lAAawgAT1QQAAoIyXECKF4AAgio8IBKEA4oawDUGHDuCH94pJSAYQh+AH0ASQARUpByypRq2vqQBOSUEBb2mETEUjKKqpq6qgbxkAAVcTgIREwsmQWBobr2LnFeGGjyPw5wzAB3UJAC68c1lPTwzOy81+Lev0lsNVqNGhMAOIQAjzPqLQbLXxnTgXK4xRy3EABVq8ACyPTQIGs0HCATAADUcikQL5fEA

JSON.stringify + lz-string
http://localhost:8003/dataset/sparklines/Published/cortex_repack.loom/N4IgFgpghgLgtlADiAXKAXge03ASlAOwHMJUBtADgBoBGAZioCYaBdKkAMwFcAbHgLWxwAEhACWRMDFQ1GAdgCcAFgrVOvAUIDqYgCYwwqOQDYFjCgAZ2AZzBJSKMkoULqdCxYCsLAL7sAxpg8qKCYAE66EGHkoADWEACeqCAAFOESYgRQPAAE4ZFhAJQg7FDW-qgwYVwQfnGJyQCCJCUgZRUoVTV1IPFJKCAAQlD+sQDKAAoAkgByAPo8EABuEDxzFnNEYZhcyKXlldW1VPX9QyPj0-OLK2s0m9u7re2H3Se9DQPDo5OzC8urOaMB47PZtA6dI4+NicMQ8GBRci+djbADuIRA+URjlOyTSYQyWVyWKKzwhXWOuK+F1+1wBaw2W1BZI6FJ6fWS30ufxugPuTKe+1ZUPeHOpPyu-1uQJBgvBwreVPOEp59LmdFlYJekO6MI4cIR0UcyJA-lWPAAsgARDHWaBhfxgABq2WSID8IGsiCgYViPEyDlAgUtmEiyQAwrAIER0v5XewSAQIBbQw5zmFrK1ExBM44QAAVL2EWStQtgdA0VoAMQIACNEDRguxGrooHAYNYaJ5WuGeLoCCX2LgAB4cButRh0GgeJSMACijEYuDEsVaYwgCDoFXYACk24xWo1ELXazQxGCq2ENwf2INwx5PCYlK0rRBEDBwj36B53ABpCx0Muq7sL+PBgDw9ClgQHBQOeFA8HQh7lBBh4wA2tY3gWcAbvQjCkOwEy+j6jC1q0MwQHGp5kWEjAcJhYzwAOAC0GaVuwYxgGI1hQAoa5cFR7AujAjatBM-i6GANAUC+iAJNYPDds2RBVJh4bDv4EHPgRaFhBwPZRNYikgDMIz+KpBAwFkiCYY01hiP4WkFogYCYcuMD0f4RCYTMNEQGxQxwIgiG3nQRDZBZ-kAOJwK4SjGK0v7+AQA7Vm2NAKFA9FtrojkMXAKUwrYmCogAMlAtarLmbI+EAA

http://localhost:8003/dataset/sparklines/Published/cortex_repack.loom/N4IgFgpghgLgtlADiAXKAXge03ASlAOwHMJUBtADgBoBGAZioCYaBdKkAMwFcAbHgLWxwAEhACWRMDFQ1GAdgCcAFgrVOvAUIDqYgCYwwqOQDYFjCgAZ2AZzBJSKMkoULqdCxYCsLAL7sAxpg8qKCYAE66EGHkoADWEACeqCAAFOESYgRQPAAE4ZFhAJQg7FDW-qgwYVwQfnGJyQCCJCUgZRUoVTV1IPFJKCAAQlD+sQDKAAoAkgByAPo8EABuEDxzFnNEYZhcyKXlldW1VPX9QyPj0-OLK2s0m9u7re2H3Se9DQPDo5OzC8urOaMB47PZtA6dI4+NicMQ8GBRci+djbADuIRA+URjlOyTSYQyWVyWKKzwhXWOuK+F1+1wBaw2W1BZI6FJ6fWS30ufxugPuTKe+1ZUPeHOpPyu-1uQJBgvBwreVPOEp59LmdFlYJekO6MI4cIR0UcyJA-lWPAAsgARDHWaBhfxgABq2WSID8IGsiCgYViPEyDlAgUtmEiyQAwrAIER0v5XewSAQIBbQw5zmFrK1ExBM44QBZjDQPBY5DMaApcGJYq0C0WPHIJjQlJXq+waO3i3JwxYK1XWu26xYKAApCx0Fv9jseCiN5t9ttTmhyK00TwThd1uiMRpj9cgAceOgUQayPcHrwKC3ls-l4uMMtr+f7zzFixKRr0M8v4sKAAyNGML9i3bQZdyfVdgM8AB5Isz2MO8FCmJszzkV8aG7RgULQ-9e1bfdUOAqDGEw8CCI8GgJgsGgsM7ABxdsaPrABpT9SNfBRGzkRjZFA3D+zI+haMYcc2PI4wWL4tsBL-MC8KXO8aAkxjGCUeiRLksjGE8Wi32U4wrSHPTfwLZSZ2Q0SxzkYRdIs5QDPU-sKDrdsYIsM8nMPOQkK48CPLfGh-0fPDGCo4CLRMp8QqLIs5AAURs4LQpioSHPYKKLBCpQWNSkB0vcCwML3YT2J3ILWmEwcaGEJciroQdGAAMR7WrBzoazqMiuryM8QYQpajxiNAsq0q6t9GFi4j+q8OQLWYIqlEHJRKJyrTWviudgs8QcFHinzgsLO9wsAyKDunWKGJO5yvOay7DyUK9jv2zdPBXPbytOpQlBgoKKHeosQrocNJsisilGMHc3rSsiFDkKCbuCvylC7OaqxocqPIWqzWIRoswYoGYSyKhRgJoOGKCJu9wYyimPE8e8qJprxjFmtzIuJjx4JPDbyvZns6H-SHct5mHwxqp86ue87ufYCWPCUOgiI6vDZbfBs+vFqcXHinKwa3JRzAJ4aQF1lS6CUJCldaFQ6DNlTvr3a3bbkHTHqthR3H1-SvAd9230+q8jecD2ttFwPfaUOm1J99w6cBwmnyDpsC0bV32ET-WKCa6Xjfd02KGEYTo7NnsmPj2IFkwfw6Ddm2FsYOHU5AF86s9lcjeb4vjDhySm5t-y6EbHLPD7z2U73YePbNnTs88eW3zobTgbwzxh4LKjrJnuQY+I+KSOX8Ox3-bP4I98tRfJp9jBHscdz31or49sdS8Fh--J6lG8OMOfZ4ocKL8-igmUhyhz3BQOeSghxCTvuwMBHt5DVUFsTdwhYOKGSfGYdw7ZGDDjFnhZQ+UQql0tuwBQq8hwWBgsQkACgr7kToI0dWeDaFjjoA1RhrRXD5XXhFPBvsqKMBYlQlw+V6C-kLugvhzBRw5R3FwhQG89wMPyp4ZcPDWhKPrONWCT4NEljoEpHRxUSxWTUewBhcB1F9wyjQCa0CQCNCsSobuijHEKAJoLBxyiLANVXC4-KJhrL-3UXPMcPVtF4UaGQzw+MaBBLMcwqiQkqGNAoHI4ycT7GpPIhQf8jcUn5UAQTHujQ+E318ToxoYNPDuyUOothW1UKeDqXTBQAEFDqKmAtBegCOlgy6WjMxUwXyfS3B06pZt9YdPgnICgW8OlgMLCoeZKhzBNLMQAVWsT2Rg6itBYNCrsle5YwHqIAJolhULM9RUAIC6AGfYqAsR7mNDKJmMxsAIkACNYDqM+XGZ5fyoC1LMYCigUBflxnaSC-wnyaDguhZ8uQELPlQvsX8quEL-DArRf4fwqLGh-N0Ds6FRBq4grAHc4laKKVLgqOS3QZLqW6GxQSilv0QViABWIKlBKxCMt5XQT5YICX+jgAC6IIKwgjGCGYuMUAeVxl0DK+xiqLGytgAqiACrIDqNGGEfl-geBJFlZgGARZdWmvbBas1BrTVItlYgHgCrrCfKIM8-w1gODOp4Aa6wPAWUep4MYXV1hrDupgFAcN8rdUwE+TGt18aFUwAIOGsINA42yqqEm-VMb9UZpVUsNNdKC1hB4O6hIzzdBQDFTyqtYr7X2LrcRdRTb+V1pti26t+K60wDDZ2uAvamwgBhLYTAqJfxQE+asXMbIfBAA
Method test0 test1 test2
JSURL 442 1058 3341
JSURL + lz-string 444 1392 2389
JSON + lz-string 480 1054 2446

Test 0 is the first viewsettings you see when loading a dataset (cell metadata). Test 1 is a medium selection of random sparkline genes. Test 2 is about 300 selected sparkline genes. As you can see, the benefits of lz-string don't really kick in until we have a huuuuge selection of genes, and even then it's debatable if any human would notice the difference.

What would probably work better is to ditch the usage of strings in the viewstate object; I consciously encoded attribute and gene keys by string to keep it human readable, but if we ditch that requirement I might as well just save the index into the array of the attribute/gene key. That would mean we'd be able to replace something like 9930012K11Rik with a number below 28000 (since we never have more than 28000 rows). For large selections of genes this probably adds up quickly. Applying lz-string compression on top of that might help even more, but this would have to be tested.

JobLeonard commented 7 years ago

So I came up with another way of shortening the URLs: custom "compression" making use of our knowledge of the data contained in viewstate.

Because we know the structure of the viewstate object, we can encode it as a positional array, with key names replaced by either position, or index nrs. Boolean values can be replaced with 0 and 1. Undefined state would be indicated with 0 ("null").

Furthermore, since the state to be saved is usually either attribute names or row/column keys (respectively genes or cell id's), which are stored in the metadata as arrays of strings, we replace the strings in the encoded viewstate object with indices. The indices should refer to a sorted array of keys to ensure stability.

The net result would be nested arrays of numbers, which are much shorter, and given the reduced use of symbols likely would compress better too.

For example:

    {
        heatmap: {
            zoomRange: [8, 13, 21],
            fullZoomWidth: 769280,
            fullZoomHeight: 1279488,
            shape: [4998, 3005]
        },
        col: {
            order: [
                { key: 'Plp2', asc: true },
                { key: 'Plp1', asc: true },
                { key: 'Aplp2', asc: true },
                { key: 'Aplp1', asc: true },
                { key: 'Class', asc: true },
            ],
            filter: []
        },
        row: {
            order: [
                { key: '(original order)', asc: true },
                { key: 'BackSPIN_level_0_group', asc: true },
                { key: 'BackSPIN_level_1_group', asc: true },
                { key: 'BackSPIN_level_2_group', asc: true },
                { key: 'BackSPIN_level_3_group', asc: true }
            ],
            filter: []
        },
        cellMD: { searchVal: '' },
        sparkline: {
            colMode: 'Categorical',
            geneMode: 'Bars',
            genes: [
                'Tspan12',
                'Gfap',
                'Plp1',
                'Plp2',
                'Aplp1',
                'Aplp2'
            ],
            showLabels: true,
            colAttr: 'Class'
        }
    }

Could become:

    [
        0, // first version of schema
        [ // heatmap
            [8, 13, 21],
            769280, 1279488,
            [4998, 3005]
        ],
        [ // col
            [ // order
              .., 1,
              .., 1,
              .., 1,
              .., 1,
              .., 1
            ],
            [] // filter
        ],
        [ // row
            [ // order
              .., 1,
              .., 1,
              .., 1,
              .., 1,
              .., 1
            ],
            [] // filter
        ],
        0, // unititiated landscape view
        [ // cellMD
          '' // one of the few cases where we have ot use a string
        ],
        [ // sparkline
            0, // colMode has preset choices, so can be indexed
            0, // same with geneMode
            [ .., .., .., .., .. ],
            1, // "showLabels: true"
            .. // 'Class' is an attribute key
        ],
        0, // unititiated geneScape view
        0, // unitiated geneMD view
    ]

Here .. indicates an index to an attribute or row/col key. Given the upper limit of 28000 genes, we can give a pessimistic average of five digits per row. For any gene name with more than three letters we are guaranteed to save space in this schema: two quotes + number of letters vs five digits maximum. For example, if we represent these unknown indices by 99999, then the minified version of the above examples are as follows:

{heatmap:{zoomRange:[8,13,21],fullZoomWidth:769280,fullZoomHeight:1279488,shape:[4998,3005]},col:{order:[{key:'Plp2',asc:true},{key:'Plp1',asc:true},{key:'Aplp2',asc:true},{key:'Aplp1',asc:true},{key:'Class',asc:true}],filter:[]},row:{order:[{key:'(originalorder)',asc:true},{key:'BackSPIN_level_0_group',asc:true},{key:'BackSPIN_level_1_group',asc:true},{key:'BackSPIN_level_2_group',asc:true},{key:'BackSPIN_level_3_group',asc:true}],filter:[]},cellMD:{searchVal:''},sparkline:{colMode:'Categorical',geneMode:'Bars',genes:['Tspan12','Gfap','Plp1','Plp2','Aplp1','Aplp2'],showLabels:true,colAttr:'Class'}}

[0,[[8,13,21],769280,1279488,[4998,3005]],[[99999,1,99999,1,99999,1,99999,99999,1],[]],[[99999,1,99999,1,99999,1,99999,1,99999,1],[]],0,[''],,0,[[99999,99999,99999,99999,99999],1,99999],0,0,]

This would make us less flexible when it comes to changing the viewstate schema, since:

Versioning the schema would actually be fairly simple: the first entry into the top array is a number, which we increase if a breaking change is made to the schema. Since this is a floating point value, we can use point-increases to indicate non-breaking changes to the schema.

JobLeonard commented 7 years ago

So, I implemented the whole custom compression/decompression thing. Here's the "base case", opening every view once without changing the default settings:

custom encoder + JSURL.stringify
http://localhost:8003/dataset/cells/Published/cortex_repack.loom/~(~0~(~(~-0.3050537109375~0.18341064453125)~963~2~3011~2~9)~(~(~0~1~3006~1~3007~1~3008~1~3009~1)~(~))~(~(~7~1~1~1~0~1~2~1~3~1)~(~))~(~2508~1~(~4695~4689~1701~140~964~3842~48)~0~1)~(~(~5016~5017)~0~0~1~1~0~0~1)~(~0)~(~(~3019~3020)~0~0~0~0~0~0~0)~(~0))

custom encoder + JSURL.stringify + lz-string
http://localhost:8003/dataset/cells/Published/cortex_repack.loom/H4CmAZShacDoDM4CsKEHYCM4CcHkRyYAcCALNgGxlnIKYBMyAlMDpQsA8Eppl21ZgwkfknCVgY8OHRSeM4vPE4pQ4M3Vg5-XRHncxaqJqhNwS-mDKUcBG8VWZ04XTm7syPYmW5lirKJawKh8Ia7ggfp6kEFQkVBgvKpIDAmx+hmx6pHMQA

custom encoder + JSON.stringify + lz-string
http://localhost:8003/dataset/genes/Published/cortex_repack.loom/NoBgNMwLQgdAzCArM+B2AjCAnOpY4MAOeAFiwDZTSl4MAmJAXTGwvjHrEQw09ZaRwfRCApgRIEGgncpRWaOwTBTQcBl8tBWVxEqIaiIxAK+wUhWz5LRZRjQg+9aqyrcipLqSIth6lF4wQJA-HW1wfwhQiGAeZUR6GMidFMjBUKYgA

Looks like the built-in JSON + lz-string gives the best results.

Below is an example with a large selection of genes, equivalent to the example in the previous post actually. Gene names added again to show how much characters are saved

Selected genes: 0610007P14Rik, 1110008P14Rik, 1110017D15Rik, 1110032A03Rik, 1190002N15Rik, 1500009L16Rik, 1500011B03Rik, 1500015O10Rik, 1700001C02Rik, 1700001L19Rik, 1700001O22Rik, 1700001P01Rik, 1700007G11Rik, 1700007K13Rik, 1700012B09Rik, 1700016K19Rik, 1700024G13Rik, 1700025G04Rik, 1700026L06Rik, 1700028P14Rik, 1700037H04Rik, 1810011O10Rik, 1810037I17Rik, 1810041L15Rik, 2010107E04Rik, 2010107G23Rik, 2010300C02Rik, 2300009A05Rik, 2310001H17Rik, 2310022B05Rik, 2310057M21Rik, 2410004P03Rik, 2510009E07Rik, 2610002M06Rik, 2610008E11Rik, 2610035D17Rik, 2610044O15Rik8, 2700046A07Rik, 2700097O09Rik, 2810417H13Rik, 2900011O08Rik, 2900026A02Rik, 2900052N01Rik, 2900060B14Rik, 3110035E14Rik, 3110043O21Rik, 3110099E03Rik, 4833427G06Rik, 4930426D05Rik, 4930444M15Rik, 4930523C07Rik, 4931406P16Rik, 4931428F04Rik, 4932438H23Rik, 5330434G04Rik, 5730522E02Rik, 6030419C18Rik, 6330403A02Rik, 6330403K07Rik, 6430548M08Rik, 6820408C15Rik, 8430408G22Rik, 9430020K01Rik, 9930012K11Rik, 9930021J03Rik, A230072I06Rik, A230083G16Rik, A330008L17Rik, A330050F15Rik, A330069E16Rik, A630010A05Rik, A830010M20Rik, A830019L24Rik, A930003O13Rik, Adgre1, Adgre5, Afg3l1, Afmid, Agap1, Agap3, Agbl3, Agk, Atp8b3, Aup1, Aven, Axdnd1, AC123873.3, AC124606.2, AC132444.2, AC132444.4, AC133095.1, AC133095.2, AC140325.3, AC151602.2, AC163611.2, AC164084.3, AC168977.2, AF067061, AI115009, AI182371, AI427809, AI464131, AI607873, AI846148, AU018091, AU020206, AU021092, AU022754, AU023070, AU040320, AU040972, AU041133, AV320801, AW146154, AW495222, Abcb5, Abcc2, Abcc3, Abcd4, Abhd10, Abhd13, Abhd16a, Abhd17a, Abhd5, Abhd6, Abi1, Ablim1, Acad11, Acad8, Acadm, Acads, Acadvl, Acat2, Acbd5, Acbd6, Acbd7, Acd, Ace, Ackr1, Acnat1, Acnat2, Aco1, Aco2, Acot5, Acot8, Acot9, Acpt, Acrbp, Acsl4, Acsl5, Acss3, Actl11, Actl6b, Actn1

encoder + JSURL.stringify:
http://localhost:8003/dataset/sparklines/Build%20161109/Forebrain_E9-E18.5.loom/~(~0~(~0~14022~-1~22439~-1~null)~(~(~0~1~44873~1~44874~1~44875~1~44876~1)~(~))~(~(~0~1~1~1~2~1~3~1~4~1)~(~))~(~14022~1~(~0~33~36~41~54~64~65~69~85~94~96~97~114~116~144~168~223~231~242~254~296~491~505~507~530~532~539~569~570~589~612~627~642~649~650~659~662~678~688~705~718~719~720~722~739~743~753~796~868~906~1021~1175~1184~1205~1319~1348~1363~1374~1375~1386~1401~1414~1486~1526~1527~1554~1556~1562~1571~1572~1613~1650~1653~1667~2094~2096~2172~2175~2179~2181~2184~2191~3109~3151~3160~3180~1819~1821~1832~1834~1837~1838~1847~1851~1853~1856~1861~2160~2231~2232~2236~2238~2247~2254~3137~3140~3141~3144~3145~3146~3147~3148~3159~3171~3173~1748~1756~1757~1766~1777~1782~1786~1788~1795~1796~1798~1804~1869~1871~1874~1875~1877~1883~1887~1888~1889~1892~1893~1901~1907~1908~1909~1910~1919~1922~1923~1933~1935~1943~1944~1954~1963~1965~1969)~0~1)~0~(~0)~0~0)

encoder + JSURL.stringify + lz-string:
http://localhost:8003/dataset/sparklines/Build%20161109/Forebrain_E9-E18.5.loom/H4CmAZQ4EYBZwCZHALQ2MuBmAnGjAOwFcAbUgSijEgzjgA4B2bWYe5uNjpgVm8ZMAbLCpgKYqLTYYUGVnVFQJUeEjlTg2VthFwMvLkKP8h+Bv1xdcI3E1jwHI+FxhCGmRKy+y4KRIaYNuy4BuD8vOD2vNiQMSgx+LxmwLxMcQz4QjAoQoj2xrlwWZHAQrxZeWVMHu4e6fxMMPUw+EyIkO0oLG04wHysTMEM7sC44M5IGDAwfA4Mrh38MNitsNiM60IK2Eyuu8vYI7AI0-qujM68iFf5sLyBMA9XVU9N9+2w2QrltOU-QnsHSsmHAwUQs38s34EKGmGasmaXAhoS0MDBaN48jckFWDFoDDWzQhsAY2DkZNcZPszSOpLgNIs0wsCgszhGiKEkGQq085L5Ih5HiwQICXFWuzRCCl2PoUv4qzgIkV9kVHlWFTR71WLFgew80OcfBpwiNTBNDDkNSNDANQ2WQyNuAN+KpKWa7w9VLmHsZZNJzADLsypNwFNwCnG03GNPGBvG+Fa6NgoUTYbkYcj2hT2GWVkjctajxskfKKbMVFolak1fAFCAA

encoder + JSON.stringify + lz-string:
http://localhost:8003/dataset/sparklines/Build%20161109/Forebrain_E9-E18.5.loom/NoBgNKYIwCwgTPMBaKZEwMwE4VoHYCuANsQLoSRowwAcA7JtGDQzM6-QKwd30Bs0CsDLCqzNEjRNqQiKIiwEUiOExNMgmGi7t+env1y0e2dtkHZ60WDcGx2UfrXTwmbyTCTxd6Cy2wdEB4uEGsuTHAIpAjcLiMwLnoo2lx+KCR+eGt9TJg00LB+LjSsovoXZxdknnooaqhcenhwZqRGJqwwbiZ6f1pnMGwQewQ0KChuG1pHFp4oTEboTDpl-hlMekdN+cwB6Dhx7Uc6ey54M+zoLl8oG7Oyu7rr5uh0mWLwJwi3-msWszoED+eCTbyTHigvroeqSersUGBMCLYHIu7SJxqepfWhLeqg6C0TBSImOInWep7QkwCkmcYmGQmewDOH8cCIRauYlcwQclwYf4+diLTZouBijE0MU8RYwQSy6yylyLEpo56LRjQLYuCH2bgUgR6+gG2hSCp62g6vrzPp67A62ggMkJerPV1kqau2lEwkMX0O1KE7Ak7AyYbjYYU4Y64a4RpQL6BOPBqTBsPqaCh+ZmMNSxq3Cxh4qZowUL5l1QVkBkIA

In raw number of characters:

Method test1 test2
encoder + JSURL 251 846
encoder + JSURL + lz-string 179 573
encoder + JSON + lz-string 174 569

Remember that this was previously 400-500 characters, and 2400-3300 characters respectively.

So still a large URL but not quite as insane as before