Information Theory

Prerequisite: CIT 604. This course covers mathematical models for channels and sources. The basic concepts of entropy, relative entropy, and mutual information are defined, and their connections to channel capacity, coding, and data compression are presented. Limits for error-free communication, information theory also presents limits for data compression, information, and data compression, Topics also include channel capacity, Shannon's theorems and rate-distortion theory.