apache / incubator-hugegraph

A graph database that supports more than 100+ billion data, high performance and scalability (Include OLTP Engine & REST-API & Backends)
https://hugegraph.apache.org
Apache License 2.0
2.58k stars 512 forks source link

fix(server): fix server slow log, support loader import & client IP #2466

Open SunnyBoy-WYH opened 4 months ago

SunnyBoy-WYH commented 4 months ago

fix #2468

we support slow log before ,but it cause bug when loader batch import data, the feat PR see https://github.com/apache/incubator-hugegraph/pull/2327

and later we downgrade it ,see pr : https://github.com/apache/incubator-hugegraph/pull/2347

the bug due to:

  1. we need get post body from req, and we need set it back to request, so it changed.
  2. the loader request use the "GZIP" header, after get post body, server cant read it

so we ready to resolve it , use BufferedInputStream to cache the stream:

` BufferedInputStream bufferedStream = new BufferedInputStream(context.getEntityStream());

        bufferedStream.mark(Integer.MAX_VALUE);

        context.setProperty(REQUEST_PARAMS_JSON,
                            IOUtils.toString(bufferedStream, Charsets.toCharset(CHARSET)));

        bufferedStream.reset();

        context.setEntityStream(bufferedStream);

` case: [Slow Query] ip=127.0.0.1 execTime=22ms, body={"gremlin":"hugegraph.backendStoreFeatures() .supportsSharedStorage();","bindings":{},"language":"gremlin-groovy","aliases":{"g":"__g_hugegraph"}}, method=POST, path=/gremlin, query=null

and then i test it image

image

codecov[bot] commented 4 months ago

Codecov Report

Attention: Patch coverage is 92.30769% with 2 lines in your changes missing coverage. Please review.

Project coverage is 49.91%. Comparing base (506850c) to head (071943c). Report is 1 commits behind head on master.

Files Patch % Lines
...g/apache/hugegraph/api/filter/AccessLogFilter.java 77.77% 2 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #2466 +/- ## ============================================ - Coverage 56.94% 49.91% -7.03% + Complexity 827 741 -86 ============================================ Files 612 612 Lines 49672 49697 +25 Branches 6681 6685 +4 ============================================ - Hits 28284 24806 -3478 - Misses 18572 22317 +3745 + Partials 2816 2574 -242 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.