search

universam1 / iSpindel

electronic Hydrometer
http://www.ispindel.de
Other
827 stars 324 forks source link

Calibration using Excel spreadsheet #475

Closed KabweMichael closed 3 years ago

KabweMichael commented 3 years ago

I'm submitting a ...

Did you follow the general troubleshooting steps first:

Report

Please replace this line with a brief summary of your issue

Console Logs

Steps to Reproduce (for bugs)

1. 2. 3.

Context

Your Environment

Features

Please note by far the quickest way to get a new feature is to file a Pull Request.

KabweMichael commented 3 years ago

Polynomials calculated using the Excel speadsheet for calibration are ridiculously long. My calculated polynomials contained factors with 21 digits and then the multiplication term and "tilt", up to 29 digits including 0 and . The possible accuracy of the calibration method can not possibly justify the use of such extreme factor lengths. Trying to use the whole of my calculated tertiary polynomial resulted in my first brew being reported as having an s.g. of about 0.6, clearly impossible. On checking my configuration I find that the firmware can not contain the whole of the calculated expression, despite it apparently having been entered correctly, and the expression is simply truncated when the capacity of the firmware is reached. Restricting the polynomial to 6 significant digits is more than sufficient for the possible accuracy of the method, and it fits into the firmware.

KabweMichael commented 3 years ago

Make your calibration polynomial a sensible length.

ErikdBr commented 3 years ago

You know that the iSpindel is calculating the density in degrees Plato? Because when you calculate your outcome of 0.6 to SG according to the Plato to SG conversion chart, you have a perfect FG. So I suspect that is the case and you'll have to let the receiving service know that is the case. image

thegreatgunbantoad commented 3 years ago

Look harder: https://github.com/universam1/iSpindel/blob/master/docs/ManualPolyGenerator.xlsx

Edit: I don't think this sheet is mentioned in the docs though. There is a sister sheet that estimates a poly based on a single data pair, based on two known curves: https://github.com/universam1/iSpindel/blob/master/docs/iSpindelPolyEstimator.xlsx Bear in mind there is a limitation of excel that means these sheets don't like the European system of 1,00 rather than 1.00

KabweMichael commented 3 years ago

iSpindel will calculate either specific gravity or degrees Plato depending on the manner in which it was calibrated. The whole point of my article was to examine the calibration process and call into question the ridiculous number of significant digits which it spews out. The Excel spreadsheet will produce a best fit polynomial between two sets of data points, either tilt and s.g., or tilt and degrees Plato, I choose s.g. because it is a well defined physical entity which I understand and know how to use to establish a final value of % alcohol in my brew. Degrees Plato is an artificial construct invented by brewers, and not used by a large number of professionals. The excessive number of significant digits produced by the spreadsheet best fit can not be accommodated in the firmware, particularly if you choose to use the tertiary polynomial. It appears to enter in full but, as I found out after first use, the excess number of digits is truncated leaving out, in my case, the tertiary terms in the equation. This is the reason why I got stupid s.g. values calculated. Because Ubidots also saves the tilt variable, I was able, in retrospect, to hand calculate my s.g. values and verify that my iSpindel was working correctly and the reason for the miscalculation. The message is – do not try and use the huge expressions produced by the polynomial fitting spreadsheet application, 6 significant digits is greatly in excess of the accuracy which any home brewer can achieve in a calibration process such as this and will produce totally satisfactory results.

From: Erik Sent: Monday, March 1, 2021 8:07 PM To: universam1/iSpindel Cc: KabweMichael ; State change Subject: Re: [universam1/iSpindel] Calibration using Excel spreadsheet (#475)

You know that the iSpindel is calculating the density in degrees Plato? Because when you calculate your outcome of 0.6 to SG according to the Plato to SG conversion chart, you have a perfect FG. So I suspect that is the case and you'll have to let the receiving service know that is the case.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub, or unsubscribe.

ErikdBr commented 3 years ago

I am clearly using an older spreadsheet :-) image

KabweMichael commented 3 years ago

Yes, I think your spreadsheet is older than mine, and you are using only a first degree correlation. Your expression well illustrates my point that it gives the user far too many significant digits for the possible accuracy of the method. You could abbreviate your equation to 0.8854*tilt-21.872 with no loss of accuracy.

I am using a third degree correlation and the equation that the spreadsheet produced is 0.882069122723967+0.009080162334059323tilt-0.00022232894925724674tilttilt+0.000001889419468219906tilttilttilt. This can not be accommodated in the firmware. I have reduced all of the numerical factors to 4 significant digits and have lost no accuracy, and it fits into the firmware.

From: Erik Sent: Wednesday, March 3, 2021 9:34 PM To: universam1/iSpindel Cc: KabweMichael ; State change Subject: Re: [universam1/iSpindel] Calibration using Excel spreadsheet (#475)

I am clearly using an older spreadsheet :-)

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub, or unsubscribe.

ErikdBr commented 3 years ago

...the spreadsheet produced is 0.882069122723967+0.009080162334059323*tilt-0.00022232894925724674*tilt*tilt+0.000001889419468219906*tilt*tilt*tilt. This can not be accommodated in the firmware...

I was still intreged by this because your formula is 116 characters long, As of version 6.5.1 of the firmware the polynominal can hold up to 250 characters.

KabweMichael commented 3 years ago

Hi Erik,

I tried it twice and on both occasions the polynomial was truncated, with severe effects of course. As I stated before, this length of polynomial is unnecessary and unjustified. It comes from a calibration process which is based on taking tilt readings and corresponding density readings. No matter how accurately the accelerometer can read the tilt, the density readings will be, at best, no more accurate than 1%. To expect the polynomial to project 15 digit accuracy from a 1% calibration is a staggering leap of faith.

From: Erik Sent: Thursday, March 18, 2021 10:26 PM To: universam1/iSpindel Cc: KabweMichael ; State change Subject: Re: [universam1/iSpindel] Calibration using Excel spreadsheet (#475)

...the spreadsheet produced is 0.882069122723967+0.009080162334059323tilt-0.00022232894925724674tilttilt+0.000001889419468219906tilttilttilt. This can not be accommodated in the firmware...

I was still intreged by this because your formula is 116 characters long, As of version 6.5.1 of the firmware the polynominal can hold up to 250 characters.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub, or unsubscribe.

thegreatgunbantoad commented 3 years ago

@KabweMichael did https://github.com/universam1/iSpindel/blob/master/docs/ManualPolyGenerator.xlsx not help you out? Any feedback from using that one is welcomed and I can update as needed. Do need to find a way to stop the decimal notation style of differing countries from causing problems though.

If you take enough samples then you can certainly project a higher accuracy than the readings you get are too, but yes 15 s.f. is a bit of a stretch.

KabweMichael commented 3 years ago

I am happy that I have a resolution that works, just eliminate several of the over long string of significant digits. I was hoping to help others who may have fallen into the same trap. I agree that the more samples you use in the calibration process the greater the accuracy, but let us not delude ourselves that 15 significant digits is either achievable or useful.

Sorry, I have no solution to your problem of decimal notation. The world has had an internationally agreed system of notation for, to my knowledge, 50 years – the ISO system, but we also have a world full of non scientific users who will not relinquish historical usage. (God knows when we will ever get the USA to go metric).

From: thegreatgunbantoad Sent: Friday, March 19, 2021 10:35 AM To: universam1/iSpindel Cc: KabweMichael ; Mention Subject: Re: [universam1/iSpindel] Calibration using Excel spreadsheet (#475)

@KabweMichael did https://github.com/universam1/iSpindel/blob/master/docs/ManualPolyGenerator.xlsx not help you out? Any feed back from using that one is welcomed and I can update as needed. Do need to find a way to stop the decimal notation style of differing countries from causing problems though.

If you take enough samples then you can certainly project a higher accuracy than the readings you get are too, but yes 15 s.f. is a bit of a stretch.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

thegreatgunbantoad commented 3 years ago

@KabweMichael That linked workbook only runs to 5 s.f. so just copy paste from the green cell without faffing about. The whole point of that workbook is to provide a poly you don't have to mess around with.

"The world has had an internationally agreed system of notation for, to my knowledge, 50 years" Not true, no ISO standard defines it (Happy to be corrected though, I'd love there to be an actual definitive standard): "ISO 80000-1 stipulates that "The decimal sign is either a comma or a point on the line." The standard does not stipulate any preference, observing that usage will depend on customary usage in the language concerned, but adds a note that as per ISO/IEC Directives all ISO standards should use the decimal comma." https://en.wikipedia.org/wiki/Decimal_separator#:~:text=ISO%2080000%2D1%20stipulates%20that,should%20use%20the%20decimal%20comma.

UK has a very odd quirk, we use metric everywhere but speed signs, which are still miles per hour!

KabweMichael commented 3 years ago

Refer ISO Standards Handbook 2 Units of Measurement. “The decimal point is a comma on the line EXCEPT in the English language where it is a point.” The preference is clear in this edition. First written in 1979, we have not learned a lot in the intervening years!

From: thegreatgunbantoad Sent: Friday, March 19, 2021 11:33 AM To: universam1/iSpindel Cc: KabweMichael ; Mention Subject: Re: [universam1/iSpindel] Calibration using Excel spreadsheet (#475)

@KabweMichael That linked workbook only runs to 5 s.f. so just copy paste from the green cell without faffing about. The whole point of that workbook is to provide a poly you don't have to mess around with.

"The world has had an internationally agreed system of notation for, to my knowledge, 50 years" Not true, no ISO standard defines it (Happy to be corrected though, I'd love there to be an actual definitive standard): "ISO 80000-1 stipulates that "The decimal sign is either a comma or a point on the line." The standard does not stipulate any preference, observing that usage will depend on customary usage in the language concerned, but adds a note that as per ISO/IEC Directives all ISO standards should use the decimal comma." https://en.wikipedia.org/wiki/Decimal_separator#:~:text=ISO%2080000%2D1%20stipulates%20that,should%20use%20the%20decimal%20comma.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

thegreatgunbantoad commented 3 years ago

So yes it remains internationally undefined, seems us Brits made it awkward. It's all a right pain in the bum either way.

thegreatgunbantoad commented 3 years ago

Lets all use parts of the body as measurement, thumb first joint is the inch, typical foot is a foot, what's a yard though? We had cubit which is the forearm. All very scientific of course. I imagine a mile is based on how far a horse can run on half a standard radish assuming no herbs or sauces.