Face2Statistics: User-Friendly Alternatives to In-Vehicle Multi-modal Bio-Sensors

Abstract

We present Face2Statistics, a comprehensive roadmap to deliver user-friendly, low-cost and effective alternatives for extracting drivers' statistics. Face2Statistics is motivated by the growing importance of multi-modal statistics for Human-Vehicle Interaction, but existing approaches are user-unfriendly, impractical and cost-ineffective. To this end, we leverage Face2Statistics to build a series of Deep-Neural-Network-driven predictors of multi-modal statistics, by taking facial expressions as input only. We address two outstanding issues of the current design, and then (1) leverage HSV color space; and (2) Conditional Random Field to improve the robustness of Face2Statistics in terms of prediction accuracy and degree of customization. Our evaluations show that, Face2Statistics can be effective alternatives to sensors/monitors for Heart Rate, Skin Conductivity and Vehicle Speed. We also perform the breakdown analysis to justify the effectiveness of our optimizations. Both source codes and trained models of Face2Statistics are online at https://github.com/unnc-ucc/Face2Statistics.

Publication
In Proceeding of the 24th International Conference on Human-Computer Interactions (HCII'22)
Zeyu Xiong
Zeyu Xiong
Incoming PhD Student

My research interests include Human-Computer Interaction, Human-AI Collaboration, Accessibility, Ubiquitous Computing and Computer Music.