How to dynamically tune RS(N,K) parameters on run time

[james224]

Hi @pjkundert . I would like to know that is there a way to dynamically tune RS parity and block size on runtime? Like i am already using your code given here . While reading the whole file i want to change RS parity for certain chunks of data. Like for example, first few chunk will get the parity of 32 bytes and last few chunks will get parity 60 bytes. But when i try to adjust it on runtime it sayscannot change value of const expression. And the second scenario is if i want addRS(255,255-k) for first few chunks and for next i want to use RS(511,511-k). But in the current scenario i am not even able to use RS(511,511-k). i get the error output data type too small to contain symbol. Any help would be highly appreciated . Thanks

[pjkundert]

One of the reasons that ezpwd-reed-solomon is so fast (especially on GNU g++) is because all of its array sizes are statically defined at compile time. This gives the compiler a chance to tune the code (I presume) to use fixed size shifts and offsets, instead of multiplies and adds, when indexing the various tables.

This has the drawback, of course, that you must know the RS(,) parameters at run-time. For most applications, this is not a problem -- you usually use 1 (or a small number) of fixed Reed-Solomon encodings. Worst case, you could (for small symbol values, anyway), create instances of each possible value at compile time. Your executable size would of course bloom to contain all of the template instantiation code for each combination -- but, these days, unless you are working in the embedded space, this probably won't cause too much of an issue.

Finally, for the RS<511,511-k> scenario, remember to use a buffer of uint16_t -- 9-bit symbols are supported by the RS(511,...) codec, which cannot fit in uint8_t...

Hope that helps!

[james224]

very much thanks for your response. But here in the code i am unable to find a way to use buffer of uint16 for RS(511,511-k) . Could you please explain how can i make the current code compatible to use with higher 9 or 10 bit symbols as well? Additionally, by using larger bit symbol values, am i only making a RS coding block size large? or is there any other pros and cons of using higher lower bit symbol as well? I mean why people prefer RS(511,k) over RS(255,k) or vice versa.

[pjkundert]

Unfortunately, the Reed-Solomon encoding doesn't produce parity symbols that contain zeros in all bit positions, even if all of the input data contains 0 bits in those positions.

Therefore, when you use an RS(511,...) codec (which processes 9-bit symbols), even if you provide all input data as purely 8-bit data (the 9'th bit is always zero), the produced parity symbols will be 16-bit symbols. They may have 1 bits in any of their 16 bit positions.

Therefore, you cannot process 8-bit ASCII data (such as you find in a std::string), using anything other than an RS(255,...) codec, to get parity data symbols you can output as 8-bit data.

So, trying to use an RS(511,...) codec to process more than 255 bytes of data per block, will always produce parity data you cannot save to an 8-bit character file.

[james224]

This seems like RS codecs greater than 8 bit symbols are just use in theory and there isn't any practical use of these codecs. Also, is it just a size of block that is changing when we use 8 or 9 bit symbol codecs? or is this also effect the error correcting capability of RS Codec as well?

[james224]

@pjkundert could you please help?

[pjkundert]

I am not certain I understand what question you would like help with...

Of course, there is use for these larger R-S codecs. If your symbols are bigger than 8 bits, then you must use a larger codec RS(511,...) or even larger for 10+ bit symbols, in order for the codec to be able to correct errors in the entire symbol.

You just have to use a std::vector instead of a std::string.

std::vector<uint16_t> data;
// fill data with your 9+ bit symbols.  Each element can hold up to 16-bit symbols, so you could
// use up to the RS<65535,...> codec
data.resize( data.size() + rs.nroots() );  // Add the space for the parity
rs.encode( std::make_pair( data.begin(), data.end() )); // and produce the R-S codeword parity
// send/receive data.  Some is corrupted.
rs.decode( std::make_pair( data.begin(), data.end() )); // R-S decode codeword; data and parity

Hope that helps!

[james224]

Very much thanks for your kind response. Let me clear the scenario if i have 1000 bytes in a text file I want to encode it .and it should recover loss of 10 bytes but the problem is when i write the encoded file into another file using your code with RS (1023,k).It just change the whole file. But previously with RS(255,k) codec its was working perfectly i.e just added redundancy to the file. Is there any possibility that i can use RS codec(1023,k) in this scenario to encode the whole RS block and write it back to the file. I don't want to use RS(255,k) in this scenario. or simply in this code can chunk size be upto 500 or more? And it works simply as it was working with RS(255,k)?