COOL-cohort / COOL

the source code of the COOL system
https://www.comp.nus.edu.sg/~dbsystem/cool/
Apache License 2.0
44 stars 15 forks source link

125 extensibility for more value types #127

Closed hugy718 closed 1 year ago

hugy718 commented 1 year ago

Address Issue#125.

The old processing code is only designed to support string and int types and they are explicitly handled in the codes. This PR adds a FieldValue abstraction to hide the actual types and exposes three types of FieldValue for processing:

Inputs are converted to these types for COOL to process, and outputs of these FieldValue can be decoded into actual values by get methods.

The internal code is refactored to decouple HashField and RangeField as much as possible to avoid any ambiguity codes are designed to handle. Type casting is avoided as much as possible and performs type checking beforehand if needed

Compressors API are changed to compress a list of FieldValue instead of directly working on int/string arrays.

hugy718 commented 1 year ago

This commit changes cleans the Compressor APIs. It is the second step to make FieldValue the internal data representation in Cool. The serialization of values is pushed into the logic of specific handlers. At the same time, I have cleaned how OutputCompressor and Histogram are used. OutputCompressor now only exposes a static methods instead of being a concrete instance that can be reset and called to compress data multiple times.

hugy718 commented 1 year ago

The next step is for the loading path, we parse inputs directly into FieldValue, and feed to write store classes.

hugy718 commented 1 year ago

Loading path including all WS related logic are updated.

hugy718 commented 1 year ago

Internal processing logic has been updated.