Everyday objects around us are becoming intelligent. Phones, watches, glasses, and even clothing now sense, interpret, and respond to human behavior. But as we move toward an AI-powered world, the challenge is no longer just building smarter devices; it is designing sensing systems that understand human context while preserving privacy, supporting user agency, and earning trust.
This course explores how sensing and AI come together to create interactive, context-aware systems that responsibly perceive and act on human activity. Students will learn the principles and practice of designing end-to-end sensing systems: from data capture and signal processing to interpretation and user interaction, through the lens of human–AI collaboration. We will discuss how fields such as human–computer interaction, embedded computing, computer vision, distributed systems, machine learning, and security intersect to create systems that are not only powerful but also transparent, ethical, and user-centered.
The course emphasizes a hands-on approach: students will prototype, program, and evaluate sensing systems in domains such as activity recognition, health and wellness, environmental awareness, and gestural interaction. Along the way, we will examine questions such as:
Students will gain practical experience in:
The class will combine lectures, tutorials, and discussions with project-based learning. Assessment will include three mini-projects, weekly readings and reflections, and a final project that integrates technical and human-centered design principles.
Instructors: Yuvraj Agarwal, Mayank Goel (Office hours: Fridays 1:30pm - 2:30 pm, TCS Hall 235)
TA: Prasoon Patidar (Office hours: 1-2PM Wednesdays TCS Hall 235)
Riku Arakawa (Office hours: 4-5PM Tuesdays, TCS Hall 235)
Location: Posner Hall 146
Time: Mondays and Wednesdays, 11:00 AM -12:20 PM
Canvas: https://canvas.cmu.edu/courses/52576
| Date | Theme | Topic | Lecturer | Assignments | Projects | Resources |
|---|---|---|---|---|---|---|
| Mon 1/12 | Introduction | Intro to Human-AI Interaction & Sensing Systems. | Mayank | Lecture 1 - Logistics and Introduction.pdf | ||
| Wed 1/14 | Sensing and Machine Learning | Signal Processing and Feature Extraction | Mayank | Lecture 2 and 3- Signal Processing and Feature Extraction.pdf | ||
| Wed 1/21 | Sensing and Machine Learning | Machine Learning for Sensing Data - 1 | Mayank | A1 Release | Lecture 2 and 3- Signal Processing and Feature Extraction.pdf |
Recofit Jupyter Notebook | | Mon 1/26 | Sensing and Machine Learning | Machine Learning for Sensing Data - 2 | Mayank | | | Lecture Video (1)
Lecture Video (2) | | Wed 1/28 | Sensing and Machine Learning | Embedded Computing - 1 | Yuvraj | A2 Release | | Lecture Video | | Thurs 1/29 | | | | A1 Due | | | | Mon 2/2 | Sensing and Machine Learning | Embedded Computing - 2 | Yuvraj | | | Lecture Video | | Wed 2/4 | Sensing and Machine Learning | Indirect Sensing | Mayank | | | Lecture Slides | | Mon 2/9 | Sensing and Machine Learning | Case Study: End-to-End Sensing Stacks -- Mites | Yuvraj | | Initial Project Ideas Due | | | Wed 2/11 | Human-AI Interaction | Introduction to Human-AI Interaction | Mayank | A2 Due, A3 Release | | Initial Project Ideas
Lecture Slides | | Mon 2/16 | Human-AI Interaction | Coding Assistants for Sensing Tasks - 1 | Prasoon | | | Lecture Slides | | Wed 2/18 | Human-AI Interaction | Coding Assistants for Sensing Tasks - 2 | Prasoon | A3 Due, A4 Release | Final Project Proposals Due | Lecture Slides | | Mon 2/23 | Human-AI Interaction | Human-in-the-loop (1) | Riku | | | Lecture slides | | Wed 2/25 | Human-AI Interaction | Human-in-the-loop (2) | Riku | | | Lecture slides | | Mon 3/2, Wed 3/4 | | | | Spring Break - No classes | | | | Mon 3/9 | Security and Privacy | ML on the Edge | Prasoon | | | | | Wed 3/11 | Security and Privacy | Security | Yuvraj | A4 Due | | | | Mon 3/16 | Security and Privacy | Privacy | Yuvraj | | | | | Wed 3/18 | Security and Privacy | Privacy-Aware Sensing | Yuvraj | | | | | Mon 3/23 | Human-AI Interaction | Case Study: Human-AI Interaction | Riku | | | | | Wed 3/25 | Human-AI Interaction | Mixed Reality: A new gateway for AI | Guest Lecture: Hyunsung Cho | | | | | Mon 3/30 | Projects | Project Progress Discussion with Instructors | Yuvraj, Mayank | | Progress Feedback | | | Wed 4/1 | Ethics | Sustainability | Yuvraj | | | | | Mon 4/6 | Ethics | Responsible AI | Mayank | | | | | Wed 4/8 | Ethics | Bias | Mayank | | | | | Mon 4/13 | Human-AI Interaction | Conversation Design | Guest Speaker: Jill Lehman | | Project Progress Checkpoint | | | Wed 4/15 | Design | Device Design | Mayank | | | | | Mon 4/20 | | Projects Troubleshooting | Yuvraj, Mayank | | | | | Wed 4/22 | | Projects Troubleshooting | Yuvraj, Mayank | | | | | 4/27-5/6 | | Final Project Presentations / Demos | | | Final Project due | |