Mobile Health

#

Our lab has been active in the mobile health research community since 2012 with the Amulet Project (of which a post-mortem summary can be found here written by Prof. David Kotz at Dartmouth). This has since turned into collaboration with Prof. Nabil Alshurafa in the Department of Preventive Medicine at Northwestern, and Prof Hester’s courtesy appointment in that department. Our goal is to reduce the effect of structural, societal, monetary, or mental barriers to receiving healthcare treatment. This has come in the form of lowering the burden of long term monitoring and interventions through novel wearable design, building tools for clinical researchers to conduct better clinical trials with strong groundtruth and high adherence, and bootstrapping clinical processes.

Early work in this space was in the design and development of the Amulet platform and software toolchain, published in SenSys’16, USENIX ATC’18, and MobiCom’19 as full papers. Amulet was the first multi-tenant and ultra low power smart watch for mobile health, allowing for a nine month battery lifetime.

Eating Detection with Wearables

#

Wearable Visual Systems

#

Extending from this we have explored how wearable visual systems can be used to understand behavior, establish ground truth, or serve as interactive devices. Our first foray into this space was investigating how privacy norms prevent the gathering of strong ground truth (for example with video), reducing adherence in medical trials that use wearables, as described in our IMWUT/UbiComp’19 paper, “To Mask or Not to Mask? Balancing Privacy with Visual Confirmation Utility in Activity-Oriented Wearable Cameras”.

We have continued this work with an IEEE Percom 2022 paper titled “ActiSight: Wearer Foreground Extraction using a Practical RGB-Thermal Wearable” which explores how low resolution thermal imaging can be used as an attentino mechanism in wearable, egocentric video collection, to greatly reduce the amount of processing power and time required to gather useful inferences from video streams. We also explored how thermal alone is a powerful mechanism to infer gestures and human behaviors, in an ISWC'21 paper titled “HeatSight: Wearable Low-power Omni Thermal Sensing.”

Battery-free Body Area Intelligence

#

With support from the NSF (CNS-2107400), together with collaborators at UMass Amherst, we have been exploring body area networks for health monitoring. An exciting new paradigm for wearable health sensors is one where sensors and batteries need not be integrated within the same device. This project seeks to realize this vision by utilizing human tissue as a medium for transferring power and data between different device pairs. The project objective is to develop hardware and software mechanisms that allow heterogeneous networks of sensors that are largely battery-free. The key working principle is the design of a system where energy transmitters that include batteries, provide power to relatively many battery-free sensors while simultaneously serving as an endpoint for data collection from the sensors. The project has three thrusts: 1) quantifying the performance and safety issues of the use of human tissue as a propagation medium for power and communications; 2) the design of hardware and software primitives needed to facilitate energy sharing between networks of battery-free devices; 3) the operating system, run-time support, and tools needed to coordinate devices that in turn enable higher level health applications.

Low-cost, Batteryless Smart Personal Protective Equipment (PPE) Tackling the COVID-19 Pandemic

#

We have been funded by the NSF (via the RAPID mechanism CNS-2032408) to embed personal protective equipment (PPE) like masks, and face shields, with ultra low cost and maintenance free sensing devices to help battle the COVID-19 pandemic. These smart PPE devices will enable healthcare workers, caregivers, and the general population to stay safe and informed.

AI Systems for Human Well-being

#

AI-Mediated Social Co-Presence

#

CompanionCast (Under Review), developed in collaboration with Dolby Laboratories, asks a simple but underexplored question: what does it feel like to watch something alone, together? The system uses multi-agent conversational AI and spatial audio to simulate the social texture of co-viewing—filling the room with the subtle presence of others—for people watching solo. Beyond entertainment, this raises a broader design question relevant to health: can technology restore a felt sense of social connection for people who are isolated by illness, recovery, or distance?

Multi-Agent Socio-Collaborative Companions

#

MASCOT (Under Review) is a multi-agent framework for building socio-collaborative AI companions that can sustain rich group-like support conversations. Rather than a single chatbot, MASCOT coordinates a team of agents—each with a distinct persona (e.g., an empathetic listener, a perspective-shifter, a positivity anchor)—under a Meta-Agent director that keeps the conversation coherent and non-redundant. A two-phase training process first aligns each agent’s individual persona using RLAIF, then optimizes collective dialogue quality to prevent agents from echoing each other or drifting into generic assistant behavior. In emotional support settings, this means a user experiences something closer to a supportive group than a single chatbot—relevant to mental health contexts where diverse social perspectives matter.

Implantables

#

Cancer Fighting Implants

We develop implantable electronics that sense the tumour micro-environment and trigger living call based therapy to treat ovarian cancer.

Circadian Computing

We develop wearable devices that can sense circadian phase and activity, then deliver interventions (via implantable) to entrain new circadian rythyms.