Closed hugy718 closed 1 year ago
This commit changes cleans the Compressor APIs. It is the second step to make FieldValue the internal data representation in Cool. The serialization of values is pushed into the logic of specific handlers. At the same time, I have cleaned how OutputCompressor and Histogram are used. OutputCompressor now only exposes a static methods instead of being a concrete instance that can be reset and called to compress data multiple times.
The next step is for the loading path, we parse inputs directly into FieldValue, and feed to write store classes.
Loading path including all WS related logic are updated.
Internal processing logic has been updated.
Address Issue#125.
The old processing code is only designed to support string and int types and they are explicitly handled in the codes. This PR adds a
FieldValue
abstraction to hide the actual types and exposes three types ofFieldValue
for processing:Inputs are converted to these types for COOL to process, and outputs of these
FieldValue
can be decoded into actual values by get methods.The internal code is refactored to decouple HashField and RangeField as much as possible to avoid any ambiguity codes are designed to handle. Type casting is avoided as much as possible and performs type checking beforehand if needed
Compressors API are changed to compress a list of FieldValue instead of directly working on int/string arrays.