Generally, “floating point error” refers to when a number cannot
be stored in the IEEE floating point representation exactly. For example, saving
1.234567899 as a floating point value is stored as 1.234567881. In fact, this discrepancy
is not due to rounding, but rather because floats are stored as approximate
values.
Integers are stored with the right-most bit being 1, and each
bit to the left being double that (2,4,8,...). It's easy to see that this can
store any integer up to 2^n, where n is the number of bits.
The mantissa (decimal part) of a floating point number is stored
in a similar way, but moving left to right, and each successive bit being half
of the value of the previous one.
Thus, numbers like 0.5 (1/2) are easy to store, but not every
number <1 can be created by adding a fixed number of fractions of the form
1/2, 1/4, 1/8, ...
A really simple example is 0.1, or 1/10. This can be done with
an infinite series (which is beyond the scope of this discussion), but whenever
a computer stores 0.1, it's not exactly this number that is stored.