The course: The course is intended as an introduction to information and coding theory with emphasis on the mathematical aspects. It is suitable for advanced undergraduate and graduate students in mathematics, applied mathematics, statistics, physics, computer science and electrical engineering.
Course content: Information measures - entropy, relative entropy, mutual information, and their properties. Typical sets and sequences, asymptotic equipartition property, data compression. Channel coding and capacity: channel coding theorem. Differential entropy, Gaussian channel, Shannon-Nyquist theorem. Information theory inequalities (400 level). Additional topics, which may include compressed sensing and/or elements of quantum information theory.
Material to be covered: Chapters 2, 3, 5, 7 and parts of chapters 8, 9 from the Cover&Thomas book (material from other chapters may be touched on lightly, but not covered systematically) plus some additional topics as indicated under "Course Content."
Grades: 394 level: homework 20%, midterm 25%, quizzes/attendance/class participation 15%, final 40%
494 level: homework 20%, midterm 25%, quizzes/attendance 10%, presentation/report 15%, final 30%
Exam Dates: Midterm: TBA (early to mid October); Final: Mon, Dec 18, 8:30-11AM
Students with special needs should contact
Division of Student Affairs/Disability Resources.
Please keep in mind that accommodations are not retroactive.
Assigments etc: Homework will be assigned weekly to bi-weekly. Some assignments may require Mathematica or Matlab, but no expert knowledge of these will be required. There will be a midterm, a final exam, and occasional quizzes. Additionally, students registered for the 494 level will be required to make a presentation and/or submit a written report on an approved topic, and to attend presentations made by fellow students. Presentation/report topics may involve information theory inequalities (chapter 17 from Cover&Thomas), an article from an information theory journal, or some mathematical topic underlying or relevant to material covered in the course. A very tentative schedule of assignments can be found here. Finalized assignments, solutions, and other documents will be progressively posted on Canvas.
Integrity: It is OK (and indeed encouraged) to discuss homework assignments with fellow students. However, any submitted work must be your own. Merely copying someone else's work is unethical, a waste of time, and may be penalized. Any substantive collaboration and/or usage of sources or AI tools has to be acknowledged. See CWRU academic integrity policy and Academic Integrity Board