search

GridProtectionAlliance / openHistorian

The Open Source Time-Series Data Historian
MIT License
172 stars 52 forks source link

Compression algorithm about the OpenHistoian #88

Open deathfromheaven opened 2 years ago

deathfromheaven commented 2 years ago

Dear developer，thanks for your contribution to the GSF. Recently,some questions about the compression algorithm of OpenHistoian come to me. At present, I have used the OpenHistorian to receive the phasor datas ,which are measured by self-developed device. Therefore,I want to make an improvement on OpenHistorian's own compression algorithm(the TSSC algorithm) to adapt to our team's phasor data. My question is : where to find the code of TSSC algorithm in the GSF framework and whether or not I can modify this algorithm. Thanks for your kind attention and look forward your prompt reply!

ritchiecarroll commented 2 years ago

Here's the C# code used for compression / decompression:

https://github.com/GridProtectionAlliance/openHistorian/blob/7d8643dc7af0239eb791020c187e10047148e2aa/Source/Libraries/openHistorian.Core/Snap/Encoding/HistorianFileEncoding.cs

See the Encode / Decode methods.

You can write your own algorithm by giving it its own EncodingDefinition, basically a unique Guid and some type definitions, similar to this:

https://github.com/GridProtectionAlliance/openHistorian/blob/master/Source/Libraries/openHistorian.Core/Snap/Definitions/HistorianFileEncodingDefinition.cs

You are not restricted to the specified HistorianKey / HistorianValue - you can use your own key/value pair definition.

I'm sure if you are willing to invest more CPU cycles, the compression can be improved - however, openHistorian has a design goal to focus on speed, although for synchrophasor data, the compression results are pretty good.

If you have a different kind of data in mind, certainly another algorithm may do better.

deathfromheaven commented 2 years ago

OK，thank you very much！