Move from floating point to decimal

[xoudini]

Double-precision floating point values are currently used in three locations, although a more exact decimal representation would be better suited for the purpose. Additionally, the front-end doesn't strip any insignificant decimals, leaving floating point inaccuracies visible to the user, as can be seen here:

Suggested fix:

Change all instances of DataTypes.DOUBLE to DataTypes.DECIMAL, and migrate the respective columns in the database. A precision of two decimals should suffice, so e.g. DataTypes.DECIMAL(3, 2) would probably be enough for all intents and purposes.

For further information, see sequelize docs on data types, as well as PostgreSQL docs on numeric types.

Affected files:

• courseinstance.js
• week.js
• codereview.js

[qzuw]

Just a note, @Chamion did something to partially address this, but this bug is still around at least on listing page.

[ollikehy]

The floating point error still exist, atleast in the teacher view sum column. A possible fix to make the frontend look decent is to force the value into fixed two decimals with for example: toFixed(2).replace(/[.,]00$/, "") The regex is so that whole numbers are left as they are.

Related Table Cell