arakelian / java-jq

Lightweight Java wrapper around JQ, a flexible JSON processor available for multiple platforms
MIT License
82 stars 10 forks source link

JqRequest.execute() slowdowns #12

Closed chop0 closed 4 years ago

chop0 commented 4 years ago

I've been playing around with particularly large JSON files, and I've found that after running a very simple query (".[].hostname"), it has been slowing down the rest of execution, namely multithreaded socket operation significantly. I have attached my code below `

    HttpGet request = new HttpGet(
            "https://api.nordvpn.com/v1/servers/recommendations?filters[servers_groups][identifier]=legacy_standard&filters[servers_technologies][identifier]=proxy_ssl");
    try (CloseableHttpResponse response = httpClient.execute(request)) {

        HttpEntity entity = response.getEntity();

        if (entity != null) {
            String result = EntityUtils.toString(entity);
            resp += result;
        }
        request.abort();
    }
    ArrayList<String> serverstemp = new ArrayList<String>();
    JqLibrary library = ImmutableJqLibrary.of();
    JqRequest jqReq = ImmutableJqRequest.builder().lib(library).input(resp).filter(".[].hostname").build();
    jqReq.execute(); //TODO slowdown culprit`
arakelian commented 4 years ago

JQ is not multithreaded. See https://github.com/stedolan/jq/issues/821#issuecomment-114168590

Quoted here: "There is no multithreading. Don't ask for it, it won't happen."