Systems and methods for predicting real-time behavioral risks using everyday images

Unmet Need

Daily environments can significantly impact health outcomes, influencing factors such as stress, sleep, nutrition, and harmful behaviors. Fall risk is a major concern, particularly for older adults, with environmental factors like poor lighting or clutter increasing the likelihood of falls. Similarly, creating a safe environment for infants and toddlers—baby-proofing—is essential for preventing accidents in the home. With over 90% of the US population owning a smartphone, mobile technologies offer a low-cost, accessible way to assess risks and provide recommendations to reduce them. Further, unlike traditional safety approaches, these technologies can monitor environments in real-time to support just-in-time adaptive interventions (JITAIs), which provide timely, personalized recommendations to decrease risks and promote overall wellness. There is a growing need for mobile device-integrated computer vision systems that can not only identify environmental hazards and deliver immediate, personalized interventions for risks such as falls and child safety, but also monitor and enhance overall wellbeing.

Technology

Duke inventors have developed a computer-vision system for real-time behavioral risk prediction. This system is designed to assist healthcare providers, behavioral scientists, and individuals in monitoring environments and assessing behavioral risks using everyday image data. Specifically, the system includes a camera and a computing device that work together to analyze environmental images, using a trained convolutional neural network to identify specific objects or settings associated with increased safety risks. The system can issue real-time alerts when a high-risk environment is detected, enabling immediate interventions to help users avoid risky situations. This system has been validated through a large study of environmental influences on smoking that included 2,457 high-risk smoking environment images and 2,445 low-risk smoking environment images. The system successfully identified risk-related environments with an AUC of 0.827, which was non-inferior to three of four smoking cessation experts.

Other Applications

This technology could also be applied to other high-risk behaviors and physiological events, such as predicting and preventing anxiety attacks, alcohol consumption, overeating, or even asthma attacks and allergic reactions by identifying environmental triggers. It could also inform design and architectural recommendations to improve environments for better sleep and concentration.

Advantages

• Real-time, automated detection of environmental risks and high-risk environments using image data, enabling timely interventions
• More affordable, accessible, and immediate compared to standard care interventions
• Adaptable to a wide range of target behaviors
• Can provide continuous monitoring via optional wearable cameras, eliminating the need for user input
• Potential for seamless integration with other wearable devices and sensors