winedepot / pinot

Apache Pinot (Incubating) - A realtime distributed OLAP datastore
https://pinot.apache.org
Apache License 2.0
1 stars 0 forks source link

Support sorted column for bytes #8

Closed xiangfu0 closed 5 years ago

xiangfu0 commented 5 years ago

Support bytes type in sorted column type.

codecov-io commented 5 years ago

Codecov Report

Merging #8 into develop will decrease coverage by 0.03%. The diff coverage is 22.22%.

Impacted file tree graph

@@             Coverage Diff              @@
##             develop      #8      +/-   ##
============================================
- Coverage      67.14%   67.1%   -0.04%     
  Complexity         4       4              
============================================
  Files           1030    1030              
  Lines          50829   50850      +21     
  Branches        7105    7109       +4     
============================================
- Hits           34129   34124       -5     
- Misses         14362   14395      +33     
+ Partials        2338    2331       -7
Impacted Files Coverage Δ Complexity Δ
.../core/indexsegment/mutable/MutableSegmentImpl.java 65.52% <0%> (-3.07%) 0 <0> (ø)
...pinot/core/query/pruner/AbstractSegmentPruner.java 74.07% <50%> (-2.85%) 0 <0> (ø)
...time/converter/stats/RealtimeColumnStatistics.java 47.22% <62.5%> (-2.06%) 0 <0> (ø)
...er/validation/BrokerResourceValidationManager.java 25% <0%> (-56.25%) 0% <0%> (ø)
.../BrokerResourceOnlineOfflineStateModelFactory.java 49.15% <0%> (-10.17%) 0% <0%> (ø)
...elix/core/periodictask/ControllerPeriodicTask.java 76.31% <0%> (-7.9%) 0% <0%> (ø)
...pache/pinot/core/util/SortedRangeIntersection.java 83.82% <0%> (-7.36%) 0% <0%> (ø)
...mpl/dictionary/DoubleOffHeapMutableDictionary.java 65.45% <0%> (-7.28%) 0% <0%> (ø)
.../impl/dictionary/LongOffHeapMutableDictionary.java 83.63% <0%> (-3.64%) 0% <0%> (ø)
...not/broker/broker/helix/ClusterChangeMediator.java 62.29% <0%> (-3.28%) 0% <0%> (ø)
... and 20 more

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 8b1c145...54824b2. Read the comment docs.

kishoreg commented 5 years ago

Ideally, the sorting should not require us to create additional arrays other than an array to store the docIds.

int[] docIds = new int[_numDocsIndexed];
    for (int i = 0; i < docIds.length; i++) {
      docIds[i] = i;
    }
    IntArrays.quickSort(docIds, new IntComparator() {
      @Override
      public int compare(int docId1, int docId2) {
        byte[] bytes1 = _dictionaryMap.get(column).getBytesValue(docId1);
        byte[] bytes2 = _dictionaryMap.get(column).getBytesValue(docId2);
        return ByteArray.compare(bytes1, bytes2);
      }

      @Override
      public int compare(Integer o1, Integer o2) {
        return 0;
      }
    });