Nov 20th 2012, 17:03:06
Maybe some of those ROUND()s should be TRUNC()s. Simple explanation.
Or just simple computer 32-bit floating point errors. 32-bit floats are only accurate to 6 significant digits, so if you have billions of cash, it can be off by 1 when you do division if the 64-bit integer cash is converted into a 32-bit float before the division happens. (Solution, use 64-bit doubles)
For example, 11,234,567,890 dollars is converted to 1.123457+e10 and you lose 7890 dollars in the NW calculation.
I'll only be concerned if the formula is off by more than 2.
Or just simple computer 32-bit floating point errors. 32-bit floats are only accurate to 6 significant digits, so if you have billions of cash, it can be off by 1 when you do division if the 64-bit integer cash is converted into a 32-bit float before the division happens. (Solution, use 64-bit doubles)
For example, 11,234,567,890 dollars is converted to 1.123457+e10 and you lose 7890 dollars in the NW calculation.
I'll only be concerned if the formula is off by more than 2.