As part of the Internet of Things, smart speaker devices, and voice assistants, such as Amazon Alexa or Google Home are becoming increasingly popular.  Our group is exploring the “next-generation” of smart speakers that use AI algorithms and innovative hardware and sensors to perform health assessment and therapy.  The video above demonstrates a simple questionnaire interaction with machine learning analysis and prediction that our group developed in 2018 to diagnose pulmonary diseases.   We have also developed chatbot algorithms for cardiovascular disease, diabetes, and mental health.  These “questionnaire chatbots” are considered first-generation health AI tools.

We are currently working on the next-generation of AI chatbots, using multi-modal sensors, such as thermal cameras and wireless sensors, coupled with audio analysis and environmental sensors, to enable more natural and continuous sensing that does not require long questionnaires.  Future AI assistants will be able to infer your health status and screen for diseases automatically and in the background as part of your natural lifestyle and natural interaction with the IoT devices in your home, car or work.

Students: Subha Nawer Pushpita, Mahmoud Sobier, Alison Fang

Collaborators: Karin Hodges, PsyD.