yuzutech / kroki

Creates diagrams from textual descriptions!
https://kroki.io
MIT License
2.87k stars 211 forks source link

Make URLs shorter via encrypted cloud storage? #964

Open borekb opened 2 years ago

borekb commented 2 years ago

Hi, I've recently came across this amazing 😍 service, it's so nice that I don't need to install all the tools locally!

If I understand correctly, the hosted https://kroki.io/ doesn't persist any data – in only uses compute (VMs by Exoscale) but there's no long-term storage of any data. This means that the entire diagram has to fit into the URL, via deflate + base64 as described in the README.

I wonder if you would consider Excalidraw-like approach which uses a clever technique:

  1. Encrypt a diagram in the browser.
  2. Upload the unreadable blob of data to an S3 bucket or similar.
  3. In the browser, construct a permalink that stores the decryption key in the anchor / hash part of the URL: excalidraw.com/#json=...

This means that Excalidraw URLs are fixed width regardless of diagram complexity:

Diagram URL
Simple
simple
https://excalidraw.com/#json=5645858175451136,8w-G0ZXiOfRYAn7VWpANxw
Complex
complex
https://excalidraw.com/#json=4680957890134016,q6NWURUuPP_VThmbRQ89Jg

For comparison, here is the same scenes via Kroki:

Diagram URL
Simple
simple
https://kroki.io/excalidraw/svg/eNrtmltXm0gcwN_9FJ70tU7nP_fpm9fG7dZbqrHu6fEgIQlKgAIxSXv63XdADQkJirZaulseNMyFGWZ-_-vwbWV1tZFMQqfxdrXhjG3LczuRNWq8TsuvnSh2A99Ukew-DoaRnbXsJ0kYv33zJu-B7GBw08vxnIHjJ7Fp94-5X139lv2dGSdy7MTye56Tdciq8qEYV8XSvcDPhgUuMeGCEJi2cOMtM17idEx11_JiJ69Jixq0s03aTscbbhwdnMedjcnWmrTyYbuu57WSiXfzUpbdH0Yzk4qTKLhy2m4n6aejF8qn_eLALEHeKwqGvb7vxPFcnyC0bDeZpGUYT0tvVuHtal4yNneaIyoJlpgC4dOatC9RGGlJhFkEXpjOZuAFUTqdV-RCWXTmNS4s-6pnZuV3pm2SyPLj0IrMPuXtRncvKhliShE1O3rfcXv9JK2lCulibexkO6AFZpxgIqYV6bjhbieD4XNxAftWFN4uVCNOb2bmnE53-4aku-5Z3ffXy5lKnHGyDCcicSlOkjGupSKyMk7buxMdr1_j0-Hk-Cxo-e5Z73i37jgBBgRUC6YYlUWeJCKMAcPPCRSXixgtgYcyIeF50Ml3KPCTlvs1XSSC50p3rIHrTeYWOWPKDLDpuXPvdWHFjuf6GUNqrvW65_ZS6hqe053HMXGNppxWJ0HYeBLOgEkZzgQYo0CFqkzzid0_GTtDHnah3w_p35eyO6J1p5kqgjDDmOo5ZjOaJUflFNuaWMR6OsVSVaEYtKRaMyaghiDvOckoiK5elOR7jT0FKMOZagmEcKIr43y9teGpD14E-6MgiCYda63lq6fhTF4OZ6kR1qxo09POlGikQNyvm3F2_YCxxwzxojBN-aamknKYdzdu1TUwQQVltD62HspdR8GAUEVFdc8xOp4cfPLfb23qT7td7Z6u984N8XVXjpqaDSNkGU3pXiqq552An42TJFWUJBcgpRKE1lBH2qb6pa39vTrS7GepB0uI0pQC5pWxPqXRu4_jCIuL9yetVnh99JfYPqm9kuQUGXvABNZFD5YyhkBRrp-TatBGR3I1P0geERFEGS9Ea7ekE6DmMv61fnk1mUG7LMKmtFRNGheKcK1xZZ4GHTk43B73z8E6sj8enx0P2s3rEp7sKIjjtb6V2P06MCXNti03vIwiDPwHze4rJZSjxRKaCJJMYSbUUpoA4QJpd64l10bazYSf4FpmU6uoNj0rTjaDwcBNzLYfBK6fTssfet7MAFaUrEdRMOo7VqdY6_id0rowfVyeGkqv_NdqvkXZzfT359dLW5tAgEqY0wnptaYQAM0SFA89IRVdSuYkN3uCRmD0arYLFR6xsJtZOVKacDk_i5XZ_493b8Q9mTGhjTcmqqcyrtXBBfV70b43uTi6_NDfEZNLu_apDMWQMXl40bkhiD2UF3sZ34ZgRkzowmZs8v_bubFSXbA0-NO81A4Z3cmJIqwyzs6n3mEL7zWPqPh6uXnU4V_a7XHdcTY6CnGNFVnM9FLjc0haTLP-5JQG0RgJqZcbIoKIJvN1d4bIiJGZsLFHv5UhKpL4c23R4lJmliRfxUdagVKxEaTUDChtzBCRsnrOxP9wGlx13_t-YLdsQq1TvofDuosNF8L4_FgUUiOZ2EgTDgitCSj5jHKjKJKSsqXhAEYLM7uVGxMIaMkxFX_EJpePxaXMyvNVfKTc3B9LQ6nsMCJNGDkbqD0YSjMIx01XDr_Q8HBIN9u93UNVe9nRBAFeyAPdpc8xls9qcZ5-tgjcOLmcA65RwpGq0vQ1KADgxhOszlOLHm-qyOH7obfedE7Y4f45n9SdJ6GlkV_MC9zcHVYTE5Q8rw8jKnnlqTepOFe4hk55y4kMPDU6lGHlX2BQjI3jpx9xyAjtrR34mw6CVsvubRxFbqD2j-uecDTQIsHBxNNzfsRNqAmokLupzZkMYM6ZUTozXzT8-kOZ8iiPYCU1SHhE1gIbD_XSPttvsjNhnUTNvZ2v0Kq9jmQSYaXUUpo4UsAYp_DLMxdGOypqJqj_JC4qeJJCl5_KMCZV-v1WZaxFM3wXdneI7I8ugETJ4NKJ9muvJLFAmjMopMpvMugcPfRV0a87lAFCKDXxI6vTocw936kBKE4149V56p2AI955_sfT5PAMmm3_4Mvl6e9wKiMAI42XG17GECkeAdbiXEYrwgGEYn_OZf4T5zIrtw9tWGHYSqwkxfRGkM3Gup1b45WvX-PadUYbS0jrZldqZb6vfP8Xx186Rw==
Complex
complex
Doesn't even render:

Screen Shot 2021-10-20 at 11 30 16

I understand that Kroki.io is simpler when it doesn't need to worry about data persistence but I hope that this example shows that the benefits of Excalidraw-like approach are substantial.

BTW a practical limit for URL length seems to be only around 2000 characters / 2kB, probably a bit more in modern scenarios but it's still way too little for powerful diagramming tools like Excalidraw or draw.io/diagrams.net (which is not currently implemented but would be super-nice if it was, and there's an issue for it: https://github.com/yuzutech/kroki/issues/405).

ggrossetie commented 2 years ago

Hi @borekb,

Thanks for your kind words abour Kroki! πŸ€—

If I understand correctly, the hosted kroki.io doesn't persist any data – in only uses compute (VMs by Exoscale) but there's no long-term storage of any data. This means that the entire diagram has to fit into the URL, via deflate + base64 as described in the README.

That's correct.

I wonder if you would consider Excalidraw-like approach which uses a clever technique:

This is a good idea and something I've considered but I'm relunctant to persist data (even if they are uncrypted). And to be completely honest, if we end up providing this feature it will most likely require a subscription. Why? Because we are now responsible for the data privacy/security and backup/recovery which come at a cost.

I understand that Kroki.io is simpler when it doesn't need to worry about data persistence but I hope that this example shows that the benefits of Excalidraw-like approach are substantial.

I'm fully aware of the benefits of such solution but also its constraints πŸ˜„ If we want to provide a good user experience, we would probably need a diagram editor where we would be able to "save" a diagram using a shorter URL. We could even use a user-defined name (i.e., https://img.krokio.io/borekb/client-server.svg). Lot of exciting possibilities but we are now talking about a fullblown product that requires funding. The question is, are users willing to pay a subscription fee to get such features? So far I've been fortunate enough to get Exoscale support but it does not cover all expenses.

Having said that, anyone can implement/provide this feature on top of Kroki.

BTW a practical limit for URL length seems to be only around 2000 characters / 2kB, probably a bit more in modern scenarios but it's still way too little for powerful diagramming tools like Excalidraw or draw.io/diagrams.net (which is not currently implemented but would be super-nice if it was, and there's an issue for it: #405).

Yes, the max URL length is a well-know limitation (described in the documentation: https://docs.kroki.io/kroki/setup/configuration/#_max_uri_length).

borekb commented 2 years ago

Thanks for the response, I realize it would be quite a big change for the infrastructure / hosting and fair enough if this is too much for Kroki.

I had this idea in mind for the setup:

You're right that anyone can build this on top of Kroki but I personally don't have the time right now (and in the near future), however, I'd be happy to sponsor a service that results from that, as in to cover the monthly costs. If this is something you'd like to discuss, feel free to reach me privately at borekb@gmail.com.

ggrossetie commented 2 years ago

You're right that anyone can build this on top of Kroki but I personally don't have the time right now (and in the near future)

Me neither, I don't have time in the near future to take on this effort.

however, I'd be happy to sponsor a service that results from that, as in to cover the monthly costs. If this is something you'd like to discuss, feel free to reach me privately at borekb@gmail.com.

Thanks, I will keep that in mind πŸ‘πŸ»

I didn't mention it but another workaround is to use a POST request: https://docs.kroki.io/kroki/setup/usage/#_post_requests

sturtison commented 2 years ago

From a privacy and IP point of view, the use case of Kroki not persisting data is very strong too. Having Kroki persist anything at all would make it a problem for many users and in many organizations this would shut down the discussion about using it.

That Kroki is able to be deployed in containers locally, with no external traffic, is a great architectural decision and its implementation has made using diagrams-as-text exceeding useful.

That said, the idea of persistence in the way you described is appreciated.