Generally, “floating point error” refers to when a number cannot be stored in the IEEE floating point representation.
Integers are stored with the right-most bit being 1, and each bit to the left being double that (2,4,8,...). It's easy to see that this can store any integer up to 2^n, where n is the number of bits.
The mantissa (decimal part) of a floating point number is stored in a similar way, but moving left to right, and each successive bit being half of the value of the previous one.
Thus, numbers like 0.5 (1/2) are easy to store, but not every number <1 can be created by adding a fixed number of fractions of the form 1/2, 1/4, 1/8, ...
A really simple example is 0.1, or 1/10. This can be done with an infinite series, but whenever a computer stores 0.1, it's not exactly this number that is stored.