in

Dé specialist in .NET trainingen en consultancy

Erik van Appeldoorn

Double trouble

Simple question if you divide 2 by 3 what do you get?

Well a zero, followed by loads of sixes and at the end a… seven.  Right!  When you check this in Visual Studio it’s not so obvious though.

DoubleError

Huh!, what the f**k is going on here , there is no spoon were the first reactions between me and my colleague.

As you see when you convert the value to a string you get the value you expect. But debug visualizer, locals, autos and immediate window all show the wrong value. Then we checked some calculations and couldn’t find any errors.

Other facts:

  • This phenomenon only comes up with doubles. Floats or decimals don’t give this strange behavior.
  • This error shows up in both VS2008 and VS2010

At the bottomline it seems just a presentation error, internal calculations are all ok.

NB. For optimal accuracy in .NET use decimals in stead of doubles. Just keep in mind: memory consumption will double and performance of calculations will drop (I read about factor 40 for a simple multiply).

Comments

No Comments