Shannon , in 'A Mathematical Theory of Communication
', proposed a linear schematic model of communications, defining
the fundamental problem of communication as the task of reproducing
at one point in space a message created at another point.
He worked out how such a message could be reliably sent, the theoretical
limit of the amount of information it could contain, and contributed
the notion of negentropy as a measure of information, thereby
creating 'information theory.' The word 'bit,' short for binary
digit, and credited to John Tukey , was used in print for the
first time.