Leveraging IoE and AI for Continuous Observation of Human Social Dynamics

Abstract

The proliferation of sensors and smart devices has led to the emergence of the Internet of Everything (IoE), where real-world objects and events can be digitally measured and analyzed. This ubiquity of data presents new opportunities to study and quantify human social dynamics and behaviors. Recent advances in artificial intelligence, especially machine learning techniques for sequential data, provide additional analytical capabilities to model the complexity of social systems and conversations as they unfold in real-time. This paper explores the potential for leveraging IoE and AI to enable continuous, ethical observation of social interactions for discovery and support of positive behaviors. Specifically, we discuss the development of an IoE architecture using environmental and wearable sensors to capture conversational dynamics at a small house party among friends. Audio, motion, and physiological data is securely transmitted to an edge computing hub. Various preprocessing and feature extraction techniques distill social signals like turn-taking, excitement levels, mirroring of expressions. These signals then feed into a specialized recurrent neural network designed to track the evolution of conversations, as well as dynamically update a social graph linking party attendees. The trained model can produce real-time visualizations of conversation dynamics over the course of the informal event. Researchers can then conduct extensive post-hoc analyses into factors driving successful interactions and information exchange within the observed friend group. Our experiments highlight promising applications in understanding small group formation and bonding for those struggling with social situations. However, we also discuss substantial privacy risks and the limitation of current sensors in capturing complete social contextual information. Extensive data safeguards and consent processes are reviewed to uphold ethical standards. Overall, this research highlights the future potential for IoE and AI to subtly augment our awareness of human and social patterns all around us. With care, such ambient intelligence could support self-understanding, empathy, and psychological wellness. We conclude with proposals for oversight procedures and considerations for additional in-context studies across more diverse demographics and public settings.