search

zgrossbart / jdd

A semantic JSON compare tool
http://www.jsondiff.com
Apache License 2.0
1.03k stars 180 forks source link

Support data URLs #61

Closed lorenzleutgeb closed 9 months ago

lorenzleutgeb commented 9 months ago

It'd be great to support data URLs, e.g.:

https://jsondiff.com/?left=https://jsondiff.com/two.json&right=data:base64,eyJBaWRhbiBHaWxsZW4iOiB7ImFycmF5IjogWyJHYW1lIG9mIFRocm9uXCJlcyIsIlRoZSBXaXJlIl0sInN0cmluZyI6ICJzb21lIHN0cmluZyIsImludCI6IDIsImFib29sZWFuIjogdHJ1ZSwgImJvb2xlYW4iOiB0cnVlLCJvYmplY3QiOiB7ImZvbyI6ICJiYXIiLCJvYmplY3QxIjogeyJuZXcgcHJvcDEiOiAibmV3IHByb3AgdmFsdWUifSwib2JqZWN0MiI6IHsibmV3IHByb3AxIjogIm5ldyBwcm9wIHZhbHVlIn0sIm9iamVjdDMiOiB7Im5ldyBwcm9wMSI6ICJuZXcgcHJvcCB2YWx1ZSJ9LCJvYmplY3Q0IjogeyJuZXcgcHJvcDEiOiAibmV3IHByb3AgdmFsdWUifX19LCJBbXkgUnlhbiI6IHsib25lIjogIkluIFRyZWF0bWVudCIsInR3byI6ICJUaGUgV2lyZSJ9LCJBbm5pZSBGaXR6Z2VyYWxkIjogWyJCaWcgTG92ZSIsIlRydWUgQmxvb2QiXSwiQW53YW4gR2xvdmVyIjogWyJUcmVtZSIsIlRoZSBXaXJlIl0sIkFsZXhhbmRlciBTa2Fyc2dhcmQiOiBbIkdlbmVyYXRpb24gS2lsbCIsIlRydWUgQmxvb2QiXX0=

See also: https://developer.mozilla.org/en-US/docs/web/http/basics_of_http/data_urls

zgrossbart commented 9 months ago

Hello @lorenzleutgeb. Thank you for the feature suggestion.

How do you envision this feature working? I'm thinking about how someone generates these data URLs. I know it's not hard to base64 encode something, but it is a little technical. How would you like to see that part work?

lorenzleutgeb commented 9 months ago

I'd like to link to a comparison of two files that are not easily accessible via a public URL (they are technically public, but just don't have a canonical URL). So, I thought why not just encode them and pass them as part of the URL?

You note that this is not complicated, and you're right:

$ curl --silent 'https://jsondiff.com/two.json' | basenc --base64url
eyJBaWRhbiBHaWxsZW4iOiB7ImFycmF5IjogWyJHYW1lIG9mIFRocm9uZXMiLCJUaGUgV2lyZSJd
LCJzdHJpbmciOiAic29tZSBzdHJpbmciLCJpbnQiOiAiMiIsIm90aGVyaW50IjogNCwgImFib29s
ZWFuIjogInRydWUiLCAiYm9vbGVhbiI6IGZhbHNlLCJvYmplY3QiOiB7ImZvbyI6ICJiYXIifX0s
IkFteSBSeWFuIjogWyJJbiBUcmVhdG1lbnQiLCJUaGUgV2lyZSJdLCJBbm5pZSBGaXR6Z2VyYWxk
IjogWyJUcnVlIEJsb29kIiwiQmlnIExvdmUiLCJUaGUgU29wcmFub3MiLCJPeiJdLCJBbndhbiBH
bG92ZXIiOiBbIlRyZW1lIiwiVGhlIFdpcmUiXSwiQWxleGFuZGVyIFNrYXJzZz9yZCI6IFsiR2Vu
ZXJhdGlvbiBLaWxsIiwiVHJ1ZSBCbG9vZCJdLCJBbGljZSBGYXJtZXIiOiBbIlRoZSBDb3JuZXIi
LCJPeiIsIlRoZSBXaXJlIl19

You wouldn't have to care about generation of base64 encoded data. Just correctly parse a data URL (which probably the browser can do for you).

zgrossbart commented 9 months ago

Thank you for the additional details @lorenzleutgeb. I think this is an excellent feature request and I've implemented it in PR #61. Please take a look if you have a chance and let me know if you have any feedback. I'm going to do a little more testing on this today and then get this change merged and deployed so it will be out live on jsondiff.com

lorenzleutgeb commented 9 months ago

See #62

zgrossbart commented 9 months ago

@lorenzleutgeb, I'm a little confused. What's the difference between #61 and #62? It doesn't look like #62 has any additional changes. Am I missing something?

Forget this. I was just getting confused between the bug number and PR number. It looks like you approved and we're good. I'm going to do a little more testing and merge this.

zgrossbart commented 9 months ago

This code has been merged and deployed. You can test it out on the live site with this URL: https://jsondiff.com/?left=data:base64,eyJmb28iOiAxfQ==&right=data:base64,eyJmb28iOiAyfQ==

Thanks again for the awesome enhancement request.

lorenzleutgeb commented 9 months ago

A follow-up (I don't have permission to reopen the issue): I saw this coming, but now I am getting an error because the URL is too long:

Request-URI Too Long

The requested URL's length exceeds the capacity limit for this server. Additionally, a 400 Bad Request error was encountered while trying to use an ErrorDocument to handle the request.

One way to work around that would be to allow passing "left" and "right" after a hash, that is

https://jsondiff.com/?left=data:base64,eyJmb28iOiAxfQ==&right=data:base64,eyJmb28iOiAyfQ==

should be equivalent to

https://jsondiff.com/#left=data:base64,eyJmb28iOiAxfQ==&right=data:base64,eyJmb28iOiAyfQ==

(the difference is quite small, just one character). The server only sees the part up to '#' and does not complain. See MDN on window.location.hash for information on how to access this portion of the URL.

The question of priority arises, and I think that if window.location.hash !== '', it should be used (and the query string should be ignored).

I am happy to PR this, if you like.

zgrossbart commented 9 months ago

I'd be happy see a PR. It's also fine to open another issue for this. Do you have a sample URL I can use to try this out for myself?

Is it clear who is returning that 400 error? I know there are specific limits for parameter length. I don't know if there is a similar limit for location hash. Is there browser agreement over that?

lorenzleutgeb commented 9 months ago

Try these two:

Your web server seems to be Apache (also confirmed via HTTP response header) using the default LimitRequestLine 8190 (Just a guess, I don't want to dig into the details on how things are counted towards that limit...).

I think there's no standard limit for browsers, but I found a comment saying that Chrome limits URLs to 2MiB and another 11 year old comment regarding Firefox that talks about 100 000 characters.