Floating Error Causes Failing to Meet "Number by Number"

[I-am-Future]

Hi TAs, When I am doing task1, I found that my result is all matched with the given one, except two places error by 0.0001. I believe this is caused by floating errors (e.g., may because operation order, e.g., theta -= grad lr / batch and theta -= grad / batch lr may result in slightly different in floating calculation). But I don't know the operation order in your case. So is that little and occational difference tolerated?

My result: | Epoch | Train Loss | Train Err | Test Loss | Test Err | | 0 | 0.35134 | 0.10182 | 0.33588 | 0.09400 | | 1 | 0.32142 | 0.09268 | 0.31085 | 0.08730 | | 2 | 0.30802 | 0.08795 | 0.30097 | 0.08550 | | 3 | 0.29987 | 0.08532 | 0.29558 | 0.08370 | | 4 | 0.29415 | 0.08323 | 0.29215 | 0.08230 | | 5 | 0.28980 | 0.08182 | 0.28973 | 0.08090 | | 6 | 0.28633 | 0.08085 | 0.28793 | 0.08080 | | 7 | 0.28345 | 0.07997 | 0.28651 | 0.08040 | | 8 | 0.28099 | 0.07923 | 0.28537 | 0.08010 | | 9 | 0.27886 | 0.07847 | 0.28442 | 0.07970 |

Your instruction's result: | Epoch | Train Loss | Train Err | Test Loss | Test Err | | 0 | 0.35134 | 0.10182 | 0.33588 | 0.09400 | | 1 | 0.32142 | 0.09268 | 0.31086 | 0.08730 | | 2 | 0.30802 | 0.08795 | 0.30097 | 0.08550 | | 3 | 0.29987 | 0.08532 | 0.29558 | 0.08370 | | 4 | 0.29415 | 0.08323 | 0.29215 | 0.08230 | | 5 | 0.28981 | 0.08182 | 0.28973 | 0.08090 | | 6 | 0.28633 | 0.08085 | 0.28793 | 0.08080 | | 7 | 0.28345 | 0.07997 | 0.28651 | 0.08040 | | 8 | 0.28100 | 0.07923 | 0.28537 | 0.08010 | | 9 | 0.27887 | 0.07847 | 0.28442 | 0.07970 |

[EnderturtleOrz]

Same

[tonyyxliu]

I think this is completely acceptable and I will make an announcement on BB for it.

[tonyyxliu]

I have just made an announcement on BB about this problem. Such slight difference will be tolerated and please worry about it.