zgrossbart / jdd

A semantic JSON compare tool
http://www.jsondiff.com
Apache License 2.0
1.03k stars 180 forks source link

Feature: Add option to get the JSON from a URL #10

Closed wildeyes closed 7 years ago

wildeyes commented 7 years ago

Add two textboxes for URLs for each side under the "upload" button, and when one clicks compare, it simply fetches the JSON from the url instead of the user having to input it by himself. would save a lot of time for that use case.

zgrossbart commented 7 years ago

There are two ways to load JSON from another location in two ways. The first is to just put an URL in one of the text boxes like this: http://myserver/myfile.json.

The second way is to add parameters to the URL like this:

http://jsondiff.com/?left=http://jsondiff.com/one.json&right=http://jsondiff.com/two.json

Thanks for using JSONDiff.

district10 commented 5 years ago

It does not work if your json file is only locally HTTP served. Can something this be possible?

https://github.com/mapbox/geojson.io/blob/gh-pages/API.md#url-api

data=data:text/x-url,

Load GeoJSON data from a URL on the internet onto the map. The URL must refer directly to a resource that is:

Thank you.

zgrossbart commented 5 years ago

@district10, the browser doesn't have permission to load data from a different location. Instead we have a small PHP script which loads the data. However, that means that the data you want to compare needs to be available from where the PHP script is running. That's unavoidable.

If you really need to access data via an URL that isn't available then you would need to run JSONDiff somewhere inside your network.

district10 commented 5 years ago

@zgrossbart I think the browser has the permission to request these data, it's just most server would not respond to the request because of CORS. I use a small web server with no CORS constraints as my data source, then I can use my brower to load data.

A python example (Python 3.6+, I think):

import os
import sys
import json
import tempfile
import webbrowser
from http.server import HTTPServer, SimpleHTTPRequestHandler, test
from urllib.parse import quote as encodeURIComponent
from multiprocessing import Process
import argparse
from uuid import uuid4
from typing import Union, Set, Dict, List, Any, Tuple, Optional

def uuid() -> str:
    return str(uuid4())[:8]

PORT = 9999
KEY = f'mocked/{uuid()}.json' # avoids browser cache problem
CACHE = {}

class CORSRequestHandler(SimpleHTTPRequestHandler):

    def end_headers(self):
        self.send_header('Access-Control-Allow-Origin', '*')
        SimpleHTTPRequestHandler.end_headers(self)

    def translate_path(self, path):
        if path.endswith(KEY):
            return CACHE[KEY]
        else:
            return super().translate_path(path)

def run():
    test(CORSRequestHandler, HTTPServer, port=PORT)

def json_editor_online2url(
        *,
        data: Optional[Dict] = None,
        url: Optional[str] = None,
) -> str:
    """
    http://jsoneditoronline.org
        document: http://jsoneditoronline.org/doc/index.html#query_parameters
        -   url_prefix + ?url=encodeURIComponent(path)
        -   url_prefix + ?json=encodeURIComponent(data)
    """
    url_prefix = 'http://jsoneditoronline.org'
    if data:
        ret_url = f'{url_prefix}/?json={encodeURIComponent(json.dumps(data))}'
    elif url:
        ret_url = f'{url_prefix}/?url={encodeURIComponent(url)}'
    return ret_url

def show_in_browser(json_path):
    path = os.path.abspath(json_path)
    CACHE[KEY] = path
    p = Process(target=run)
    p.start()
    url = json_editor_online2url(url=f'http://localhost:{PORT}/{KEY}')
    webbrowser.open_new_tab(url)
    p.join()

if __name__ == '__main__':
    prog = 'python3 open_json_editor_online.py'
    description = ('view json file')
    parser = argparse.ArgumentParser(prog=prog, description=description)
    parser.add_argument(
        'input_path',
        type=str,
        help='input json path',
    )
    args = parser.parse_args()
    show_in_browser(args.input_path)

Usage:

python3 open_json_editor_online.py data/calibration.json

A new web page will pop up in my browser with my data loaded.

The semantic diff idea is awesome and your project is cool. I tried some other choices (e.g. https://github.com/seperman/deepdiff) but... anyway, I always favor the browser and html diff coloring.

:smiley: Really expecting this.

zgrossbart commented 5 years ago

Thanks for this code sample, but your sample would all the browser to make CORS requests to any server anywhere. That has some serious security concerns.

district10 commented 5 years ago

@zgrossbart True, I'm aware of that. In my case, I just run the script, open the web page, then I kill the script after the web page fetched the data. And my resource url changes everytime.