Import from Github gist

lemmy commented 5 years ago

dreampuf commented 5 years ago

That's an interesting feature.

Originally thought was try to not use any external services. So It is a simple URL encoding as the query string.

Gist definitely would be a cleaner solution and associated with versioning and multi-files support. But I'd like to consider it be the right solution rather than a simple Gist reader. How to put your current work into Gist should also need to think of it.

lemmy commented 5 years ago

At first, why not just have a url parameter that takes the given resource as input. Could be a (raw) gist or a .dot file hosted on a webserver.

dreampuf commented 5 years ago

Because of the Same-origin policy, it restricts very few sites that could be as an input in Javascript, unless the target site enabled the cross-origin script access. AFAIK Gist v3 API provides the JSONP that could be a choice.

leonelgalan commented 4 years ago

Another option Im considering to shorten the URLs is compressing with zlib (pako) and base64 encode. The entire chart shared by @lemmy comes down to:

eJzFl8tu4zYUhtf2UxBetYDCkof3FhpMil42gyxmpt0kReHYnNiNYruW0rQo\n8u7zk7IydqLYSRC08MakpMP/OzeSdbOeTxo2nV+sx6sZ+2FeX/6c//07HLRT\np+fn5UgUopCkTKG14l6PiuFgsFhOYx1XpeDKDAe/fTfMM+y0Gp/HqhydnYzS\nXJxeYG6yrJbrejKLV7FcjefrOJWUntbX5+0qk+q6buL694tu8e3VfYGfC4VW\nknuZFx9ki+XNbN5EDJOtQfqq+2w9XlyW9fgq5ieDI22d1yKQcEYaEYQIBtPs\ndBbnF7OmlFxome0ONvK/OTtjf8E18ytWspPrqjpbpKnlzSKuMfNVI9m3b9hP\nx+8+/MjevmUN3Q2/bt+8Gde/dt/ve7vFGayWdTnS5AvlLVd6M1s3/1Sx/DSv\nqjhtZ27m02ZWGm6Vyx4c3O4nV9YZ7y0FYRUpHST5Z4A38hWwP77/ZS81KSrI\nGa78aJfRyKC7+PkgdQjaKSOM8gLJ+BwMev3oJaodDGv0AYwDoTqSgESCUrBe\nOW+tDfq/htyN1QNGkugDVnOt7jFqIU1LYY1BcDTSODjy1jgT/t+Eexgo5w5A\nHAiUDyGQ0cFahyBZsL5CnPqq5MW5mErKKK6de7SktLJkpRdOK4N/Tir56nHK\nul7cF6xVByBSnPpaOzt6w3ob32YnKikb/LRcNNvjvGosKGge0IglfKBgByNZ\noO9wS0xppAlGKnDjGQxz7EuKHPZApgRGFt/ZvPM9qqy/mT1JmvHEjcMSgmvD\nYAO+gDTJSTEDMaKVZgNDenIFZZY7z4yDGzU+A4puxfW4J2vr7UFPc5t0aRES\nkgfLiDQ3piDluCeMiIeCKOnFQHKJF0lwExjJwDU2AHi709bnoCSut7U8SRsw\nOFGnzVqZwrbRhjRLXtyIsxYd3HbiUOFcml1xPR5K4nqbQidO3ROndhyXXGWL\ngHQTcFVyo8Fpy3LvsndkIaWHM1kS5W0hBSKss0+Nx0ggW1ttPQ5K0npL/UnS\nEr9TnTTriJPtpMEOpztt1gkuQqfNWs9D2NHW45/DxaDvidM7xYBaJOQRjqQB\nhSlRg4XX3PtU+9yZIqBVBFQw6hkQSZpWTEuXCVCLGjUiwGOISw8vOSRHIZ3l\nFs+caNNFMijMqYzAKGLwcKrylC6BnQ2NMomUtGgxe3x9sBvtpdyAZUpUsoZj\nKWFCCepIhSJ1K7JoBgpdAJR4QWf5HSW4XKKU28yJ0mdnbChVAJ5uKeFBp1Mz\nyZQOmLn9JRWIchvQ257TexUv4mL64Pju0bFMwL0BksjhGG+6O0S3t5zEvxv2\noRk3kR1jk1gu6s3jzW4kUqbliVU5kgItUKNoumNy1W4MEuHN4/bUXC+r+bS7\nG7RXk75ryBcZfwCirFoLs/Eqlus4Wa6nWybbg3hn83jy5zVsvFtOLu9i2Wc8\nfbVdW3dMRiofttywZbBXVc4IKZHZ25cEqGzyvELzR92p5GaJ25L44qG9PHmi\ncyG6Snuv+B4vX8V2z34iHu3H27a4h0+gF/bxCaRtxyfx/2V8yvuQ+d7HKo7r\n50RP78fbMvg4XVC+D84H07EFL1+IhlORbivzdvgZT6vUjQ==\n

Which is 1635 long as opposed to 4056.

const pako = require('pako')
const atob = require('atob')

// Based on https://stackoverflow.com/a/30240791/637094
function uncompress(base64data){
    compressData = atob(base64data);
    compressData = compressData.split('').map(function(e) {
        return e.charCodeAt(0);
    });
    binData = new Uint8Array(compressData);
    data = pako.inflate(binData);
    return String.fromCharCode.apply(null, new Uint16Array(data));
}

const chart = 'eJzFl8tu4zYUhtf2UxBetYDCkof3FhpMil42gyxmpt0kReHYnNiNYruW0rQo\n8u7zk7IydqLYSRC08MakpMP/OzeSdbOeTxo2nV+sx6sZ+2FeX/6c//07HLRT\np+fn5UgUopCkTKG14l6PiuFgsFhOYx1XpeDKDAe/fTfMM+y0Gp/HqhydnYzS\nXJxeYG6yrJbrejKLV7FcjefrOJWUntbX5+0qk+q6buL694tu8e3VfYGfC4VW\nknuZFx9ki+XNbN5EDJOtQfqq+2w9XlyW9fgq5ieDI22d1yKQcEYaEYQIBtPs\ndBbnF7OmlFxome0ONvK/OTtjf8E18ytWspPrqjpbpKnlzSKuMfNVI9m3b9hP\nx+8+/MjevmUN3Q2/bt+8Gde/dt/ve7vFGayWdTnS5AvlLVd6M1s3/1Sx/DSv\nqjhtZ27m02ZWGm6Vyx4c3O4nV9YZ7y0FYRUpHST5Z4A38hWwP77/ZS81KSrI\nGa78aJfRyKC7+PkgdQjaKSOM8gLJ+BwMev3oJaodDGv0AYwDoTqSgESCUrBe\nOW+tDfq/htyN1QNGkugDVnOt7jFqIU1LYY1BcDTSODjy1jgT/t+Eexgo5w5A\nHAiUDyGQ0cFahyBZsL5CnPqq5MW5mErKKK6de7SktLJkpRdOK4N/Tir56nHK\nul7cF6xVByBSnPpaOzt6w3ob32YnKikb/LRcNNvjvGosKGge0IglfKBgByNZ\noO9wS0xppAlGKnDjGQxz7EuKHPZApgRGFt/ZvPM9qqy/mT1JmvHEjcMSgmvD\nYAO+gDTJSTEDMaKVZgNDenIFZZY7z4yDGzU+A4puxfW4J2vr7UFPc5t0aRES\nkgfLiDQ3piDluCeMiIeCKOnFQHKJF0lwExjJwDU2AHi709bnoCSut7U8SRsw\nOFGnzVqZwrbRhjRLXtyIsxYd3HbiUOFcml1xPR5K4nqbQidO3ROndhyXXGWL\ngHQTcFVyo8Fpy3LvsndkIaWHM1kS5W0hBSKss0+Nx0ggW1ttPQ5K0npL/UnS\nEr9TnTTriJPtpMEOpztt1gkuQqfNWs9D2NHW45/DxaDvidM7xYBaJOQRjqQB\nhSlRg4XX3PtU+9yZIqBVBFQw6hkQSZpWTEuXCVCLGjUiwGOISw8vOSRHIZ3l\nFs+caNNFMijMqYzAKGLwcKrylC6BnQ2NMomUtGgxe3x9sBvtpdyAZUpUsoZj\nKWFCCepIhSJ1K7JoBgpdAJR4QWf5HSW4XKKU28yJ0mdnbChVAJ5uKeFBp1Mz\nyZQOmLn9JRWIchvQ257TexUv4mL64Pju0bFMwL0BksjhGG+6O0S3t5zEvxv2\noRk3kR1jk1gu6s3jzW4kUqbliVU5kgItUKNoumNy1W4MEuHN4/bUXC+r+bS7\nG7RXk75ryBcZfwCirFoLs/Eqlus4Wa6nWybbg3hn83jy5zVsvFtOLu9i2Wc8\nfbVdW3dMRiofttywZbBXVc4IKZHZ25cEqGzyvELzR92p5GaJ25L44qG9PHmi\ncyG6Snuv+B4vX8V2z34iHu3H27a4h0+gF/bxCaRtxyfx/2V8yvuQ+d7HKo7r\n50RP78fbMvg4XVC+D84H07EFL1+IhlORbivzdvgZT6vUjQ==\n'

uncompress(chart);

timabell commented 4 years ago

Some references

Thanks for a great project!

timabell commented 4 years ago

Seeing if I can pay someone to build this because I want it and don't have time: https://www.upwork.com/jobs/~01a9f81b8a394c74fa

lemmy commented 4 years ago

@timabell Is there a way for me to chime in and raise to $100?

timabell commented 4 years ago

@lemmy that's a good idea, I haven't had any takers so far so it might help.

I've set up a payment page so you can send money to my business which is funding the upwork posting; I've set the amount to $60 to cover all the processing fees so that I can still make it $100 total if that's okay.

Head over to https://timwise.co.uk/pay/ and put in a credit card number and I will add the money to the upwork posting. I'm pretty sure I can do a refund from stripe if it doesn't work out. Any problems drop me an email.

lemmy commented 4 years ago

@timabell Tried to chime in and my browser crashed mid-flight. Did you get the $60?

timabell commented 4 years ago

Yep, got it, thanks. Yay computers. ... upwork posting updated, let's see if this works! https://www.upwork.com/jobs/~01a9f81b8a394c74fa

bcardiff commented 4 years ago

@dreampuf AFAIK you can use urls with gist.githubusercontent.com prefix to allow CORS.

ObservableHQ uses those url in many examples https://stackoverflow.com/a/57255140/30948

Another route to make .dot files from gist render in graphviz is to package the js in an easy way to consume it and use https://bl.ocks.org/ to load those gist. People will need to add a bolerplate index.html to bl.ocks.org render it directly but is another way to share graphs with no maintenance cost.

timabell commented 4 years ago

Neat, thanks @bcardiff Here's a trivial graph example in that form: https://gist.githubusercontent.com/timabell/da08616ecb8693d524b8eab3b7b51018/raw/9575a3c887526eb5858587514f6bfb5c173b6449/graph.gv

(from https://gist.github.com/timabell/da08616ecb8693d524b8eab3b7b51018 )

Which comes with these headers:

Selection_20200723-01-d

Which includes Access-Control-Allow-Origin: *, which I think means we are okay for xss protection - reference https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS

In this case, the server responds with Access-Control-Allow-Origin: *, which means that the resource can be accessed by any domain.

dreampuf commented 4 years ago

Thank you, everyone, for using this tool. I'm working on a rework version(relatively slow). It combines monaco-editor(DOT language highlight support and even a fully functional LSP) and an updated javascript portion of graphviz https://github.com/hpcc-systems/hpcc-js-wasm. It gives you ability to link picture files in your graph.

For the shorter share URL, @leonelgalan shared a valuable library pako. I will check it out. Thank you @leonelgalan. Regarding gist, thank you for sharing the knowledge. The client-side import is possible. It's on my list.

timabell commented 4 years ago

@lemmy I've closed that upwork post as it had zero response and upwork removed the link to this issue which was less than helpful. I'll post with bountysource instead which is perhaps more appropriate. If you're not happy with this change in approach let me know and I'll refund your contribution. Sorry to others for clogging up the discussion with trying to get this paid for, I hope you all don't mind. ... and here's the bounty https://www.bountysource.com/issues/83842220-import-from-github-gist :crossed_fingers:

timabell commented 4 years ago

It works! https://dreampuf.github.io/GraphvizOnline/?url=https://gist.githubusercontent.com/timabell/da08616ecb8693d524b8eab3b7b51018/raw/0e205c341b40641206a55c9f96b5db9b8fa581bc/graph.gv Thanks everyone who helped on this, I've very happy this has been done and merged. :champagne:

dreampuf / GraphvizOnline

Import from Github gist #12