Open ManjunathVenkataswamy opened 4 months ago
@dcamper need to sanity check the viz when going from 32bit -> 64bit numbers (JS doesn't support them and they end up being strings)
@GordonSmith good point. What would be the best workaround for values exceeding 2^32? (Or is it 2^31?)
Its actually converted to a decimal with precision 2^56 (from memory), so you start to lose the insignificant values - but - it gets sent as a string which sometimes catches us out.
@ManjunathVenkataswamy: Would an UNSIGNED6 suffice for your needs? That would give you a maximum value of 281,474,976,710,655 (2^48 - 1) and yet comfortably remain a number for visualization purposes.
yes it would.
@ManjunathVenkataswamy: If you have the ability, please check out the candidate-1.10.0 branch (https://github.com/hpcc-systems/DataPatterns/tree/candidate-1.10.0) and test it. Once accepted, I can get it merged into the platform's Std ECL library.
I haven't yet been able to test this. But will do so within 2 days.
The 'rec_cnt' that comes out in the response might need to be changed from being a unsigned4 to unsigned. We have noticed it created an issue with a pattern count that was greater than what a unsgined4 could hold. And down the lane, it ended up coming out as a junk number that made no sense. Here is an example of how this would cause issues that depend on datapattern library for profiling the data. if you run a code like this: unsigned4 test := 4449418834; test; you would get 154451538 !!!
Discussed this need with Dan and creating this ticket.