Agner`s CPU blog

INF is generated by overflow and division by zero. The NAN code is generated by calculations that have no meaningful result, such as 0/0 and INF-INF, as well as calculations that would require complex numbers where only real numbers are available, such as sqrt(-1) and log(-1).

An instruction with INF or NAN input will generate INF or NAN output. This means that if there is an error somewhere in a sequence of calculations then the end result will be INF or NAN in most cases. If you print out the result, you will simply see 'inf' or 'nan' instead of a valid number. This is a convenient way of detecting floating point errors, and it is more efficient than using traps (software interrupts) for detecting numerical errors. Traps are particularly troublesome if the code uses vector instructions (SIMD).

The NAN code has additional bits which can contain information about the kind of error that generated the NAN. This extra information is called a NAN payload. The NAN payload can be very useful for error codes from mathematical function libraries. Unfortunately, the official standard for floating point representation (IEEE 754) is incomplete with respect to the propagation of NAN payloads. The standard does not specify exactly what should happen when two NANs with different payloads are combined (e.g. NAN1 + NAN2) and when converting between single and double precision.

I have discussed these problems with the working group behind the IEEE 754 floating point standard, but they do not want to make any modifications in a forthcoming revision of the standard because NAN payloads are rarely used today and it is difficult to predict future needs (http://754r.ucbtest.org/background/nan-propagation.pdf)

I have written a paper describing the details of NAN propagation, including recommendations on how to use it and which compiler optimization options to use. You can find the paper here: agner.org/optimize/nan_propagation.pdf