People Faculty Central Organization
搜索人机交互，纳米材料，集群机器人...
Xinyi Fu
Associate Professor
Natural HCI, Affective computing, Cultural habitat preservation

Introduction

Dr. Xinyi Fu received her Ph.D. in Design from the Academy of Arts & Design, Tsinghua University, in 2018, and her Bachelor’s degree from the Department of Computer Science and Technology, Tsinghua University, in 2012. After completing her postdoctoral fellowship at the Academy of Arts & Design (2018–2022), she joined The Future Laboratory at Tsinghua University, where she currently serves as an Research Associate Professor.

Dr. Fu’s research focuses on multimodal sensing, computing, and interaction in living spaces. Her work emphasizes the sensing and modeling of human behavior and psychology through multi-modal sensing to construct human-centric intelligent systems. These efforts aim to enhance the efficiency and quality of human-machine collaboration and deepen the machine's understanding of human cognition. Her team’s research spans several key domains:

  • Human Activity Recognition (HAR) and Affective Computing

  • Smart Home systems and Embodied Intelligence

  • Virtual and Augmented Reality (VR/AR)

  •  Digital Cultural Heritage

Dr. Fu is a Senior Member of the CCF (China Computer Federation) and CSIG (China Society of Image and Graphics). She holds several leadership and advisory roles, including:

  • Executive Member of the CCF Pervasive Computing and CCF Human-Computer Interaction (HCI) Technical Committees.

  • Board Member of ICACHI (International Chinese Association of Computer-Human Interaction).

  • Youth Researcher for the Beijing Association for Science and Technology’s expert team on interdisciplinary design and innovation.

  • Conference Service: Organizing or Program Committee member for top-tier international conferences, including ACM CHI, IEEE ISMAR, SIGGRAPH Asia, and ICHEC.

Dr. Fu has led her team in publishing over 40 papers in leading HCI and AI journals and conferences, and has filed more than 20 invention patents. Her research has received prestigious accolades, including: Best Poster Award at IMWUT/Ubicomp (CCF-A, top 2.9%), Special Contribution Award from the ICACHI.

Her work has garnered significant media attention, notably from CCTV. Her team's achievements in smart home technology were featured on the CCTV-9 program "The Future of China" as a landmark case study for the development of future domestic intelligence, achieving widespread social impact.

Dr. Xinyi Fu has led or participated in several national-level and corporate-sponsored research projects:

  • Research on Rapid Identification and Linked Response Mechanisms for Psychological Anomalies in the Elderly within Community-Based Elderly Care Systems (PI)

  • Key Technologies for Privacy-Preserving Proactive Human Identification and Intelligent Sensing Based on Large Models (PI)

  • Interdisciplinary Frontier Exploration and Collaborative Exchange Project (PI)

  • Research on Long-term Behavioral and Psychological Monitoring and Anomaly Early Warning for the Elderly Based on Fusion Sensing (PI)

  • Top-level Design for Smart Home Interaction Experience (PI)

  • Field Evaluation of Performance for Acoustic and Radio Frequency (RF) Wireless Sensing Technologies (PI)

  • Research on Digital Science Popularization, Exhibition, and Broadcasting of Cultural Heritage Based on Virtual Reality (PI)

  • Research on New Modes of Innovation and Social Development under the Context of New-Type Artificial Intelligence (PI)

  • Research on High-Quality Development Paths for the Sci-Fi Industry Supported by 5G, AR, and VR Technologies (PI)

  • Research on the Construction and Design of Smart Spaces for Future Personalized Living (PI)

  • Theories and Methods of Multimodal Affective Computing in Living Scenarios (PI)

  • Development of Algorithm and Software Systems for Touch Functions of Control Boards (CO-PI)

  • Theories and Methods of Multi-source Immersive Sensing and Multimodal Real-time Generation for the Metaverse (CO-PI)

  • Theories and Methods of Multimodal Natural Human-Computer Interaction for Smart Homes (CO-PI)

Selected Publications：

  • He Zhang and Xinyi Fu*. 2025. Zero-shot Emotion Annotation in Facial Images Using Large Multimodal Models: Benchmarking and Prospects for Multi-Class, Multi-Frame Approaches. In Proceedings of the 3rd International Workshop on Multimodal and Responsible Affective Computing (MRAC ’25), October 27–31, 2025, Dublin, Ireland. ACM, New York, NY, USA, 10 pages. https://doi.org/10.1145/3746270.3760233

  • Xinyi Fu#, Meng Li#, Xiaomeng Li, Wen Chen, Lening Yu, Zixin Chen, Shuting Wen, Yilin Li, Jiachen Du, Yun Wang*, Yingqing Xu*, and Yunbing Chen*. 2025. Emerging Computing Technology for Digital Culture Heritage Preservation and Inheritance: A Literature Review. IEEE Transactions on Computational Social Systems (2025), 1–38. DOI:https://doi.org/10.1109/TCSS.2025.3589324

  • He Zhang, Xinyang Li, Xingyu Zhou and Xinyi Fu*. VR Calm+: Furry Stress Ball as a Haptic Modality for Relaxation in Virtual Reality. In 2025 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct). IEEE, Daejeon, Korea, Republic of, 632–633. DOI: 10.1109/ISMAR-Adjunct68609.2025.00128.

  • Jiachen Du, Xinkai Zou, Liwen He, Shuzi Yin, Bingjie Gao, Wenjing Deng, Xinyi Fu, and Haipeng Mi*. 2026. Emotional Support Originating from Dialogue: Empowering Older Adults' Self-Tracking of Sleep-Related Daytime Factors. In Companion of the 2025 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp Companion '25). Association for Computing Machinery, New York, NY, USA, 70–74. https://doi.org/10.1145/3714394.3754418

  • Jiachen Du, Bingjie Gao, Shuzi Yin, Xinkai Zou, Wenjing Deng, and Xinyi Fu*. 2026. TaskWise: Enhancing Cognitive Performance in Task Management through Interactive Detail Enrichment. In Companion of the 2025 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp Companion '25). Association for Computing Machinery, New York, NY, USA, 271–275. https://doi.org/10.1145/3714394.3754433

  • Jiachen Du, Hanyu Huang, Xinkai Zou, Shuzi Yin, Bingjie Gao, and Xinyi Fu*. 2025. The Social Dynamics of Voice Cloning: Trust, Privacy, and Ethical Tensions When Sharing Your AI Voice Replica. In Companion Publication of the 2025 Conference on Computer-Supported Cooperative Work and Social Computing (CSCW Companion '25). Association for Computing Machinery, New York, NY, USA, 307–311. https://doi.org/10.1145/3715070.3749244

  • Cheng Xue, Minghui Liu, Yuxiang Zhai, Jiachen Du, Xinyi Fu*. (2025). Physiological and Behavioral Analysis Based Multimodal Evaluation of Older Adults’ Experience with Kitchen Appliances. In: Gao, Q., Zhou, J. (eds) Human Aspects of IT for the Aged Population. HCII 2025. Lecture Notes in Computer Science, vol 15810. Springer, Cham. https://doi.org/10.1007/978-3-031-92710-2_11

  • Jiachen Du, Tongtong Jin, Ruowen Niu, Yuxiang Zhai, and Xinyi Fu*. 2025. Enhancing Older Adults' Lives with Conversational Agents: A Systematic Review of Contexts, Capabilities, and User-Centered Design Strategies. In Proceedings of the Twelfth International Symposium of Chinese CHI (CHCHI '24). Association for Computing Machinery, New York, NY, USA, 258–268. https://doi.org/10.1145/3758871.3758891

  • Xiao Xue, Shuang He, Keqi Chen, Ruoxin Qiao, Weitai Xu, Enyang Wang, Yaxuan Zhao, and Xinyi Fu*. 2025. When My Car Whispers Secrets to My Home: Envisioning the Opportunities and Challenges of Human-Vehicle-Home Integrated Ecosystems through Probes. In Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems (CHI EA '25). Association for Computing Machinery, New York, NY, USA, Article 598, 1–12. https://doi.org/10.1145/3706599.3720252

  • Shuang He, Da Gao, Xingyu Zhou, Haoyu Dai, Shengyi Wang, Yingqing Xu, and Xinyi Fu*. 2025. BakeEase: Redefining Baking Experiences with AI-Powered Multimodal Systems. In Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems (CHI EA '25). Association for Computing Machinery, New York, NY, USA, Article 90, 1–8. https://doi.org/10.1145/3706599.3720180

  • Cheng Xue, Zonglin Zuo, Xinran Jiang, and Xinyi Fu*. 2024. DogChat: A Pet-centered Smart Collar Prototype based on Large Language Models and Wechat. In Companion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp '24). Association for Computing Machinery, New York, NY, USA, 162–166. https://doi.org/10.1145/3675094.3677606

  • He Zhang, Xinyang Li, Xinyi Fu*, Christine Qiu, Jiyuan Zhang, and John M. Carroll. 2024. Understanding Fear Responses and Coping Mechanisms in VR Horror Gaming: Insights From Semistructured Interviews. IEEE Transactions on Games 16, 4 (2024), 868–881. DOI:https://doi.org/10.1109/TG.2024.3403768

  • Xiao Xue, Xinyang Li, Boyang Jia, Jiachen Du, and Xinyi Fu*. 2024. Who Should Hold Control? Rethinking Empowerment in Home Automation among Cohabitants through the Lens of Co-Design. In Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI '24). Association for Computing Machinery, New York, NY, USA, Article 854, 1–19. https://doi.org/10.1145/3613904.3642866

  • Mengyu Chen, Shuang He, and Xinyi Fu*. 2024. A Case Study Exploring the Applicability of Heuristic Evaluation in Smart Home Systems. In Extended Abstracts of the 2024 CHI Conference on Human Factors in Computing Systems (CHI EA '24). Association for Computing Machinery, New York, NY, USA, Article 498, 1–7. https://doi.org/10.1145/3613905.3637131

  • He Zhang, Xinyang Li, Yuanxi Sun, Xinyi Fu*, Christine Qiu, and John M. Carroll. 2024. VRMN-bD: A Multi-modal Natural Behavior Dataset of Immersive Human Fear Responses in VR Stand-up Interactive Games. In 2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR), 320–330. DOI:https://doi.org/10.1109/VR58804.2024.00054

  • Zhuying Li, Xipei Ren, Chengyu Liu, Ding Ding, and Xinyi Fu. 2024. Editorial: Body-centric computing for health and wellbeing. Frontiers in Computer Science Volume 6-2024, (2024). DOI:https://doi.org/10.3389/fcomp.2024.1393102

  • Tongxin Sun, Tongtong Jin, Yuru Huang, Meng Li, Yun Wang, Zhe Jia & Xinyi Fu*. (2023). Restoring Dunhuang Murals: Crafting Cultural Heritage Preservation Knowledge into Immersive Virtual Reality Experience Design. International Journal of Human–Computer Interaction, 40(8), 2019–2040. https://doi.org/10.1080/10447318.2023.2232976

  • 肖虹,唐健凯,丘雨涵,付心仪*.隐私友好的步态数据采集与情绪识别方法[J].计算机辅助设计与图形学学报,2023,35(02):203-212. （Xiao Hong, Tang Jiankai, Qiu Christine, Fu Xinyi*. A Method of Privacy-Friendly Gait Date Acquisition and Emotion Recognition[J]. Journal of Computer-Aided Design & Computer Graphics, 2023, 35(2): 203-212. DOI: 10.3724/SP.J.1089.2023.20042）

  • Jie Liu, Dan Luo, Xinyi Fu, Qi Lu, and Karen Yixin Kang. 2022. Design Strategy of Multimodal Perception System for Smart Environment. In Internet of Things for Smart Environments. Springer, 93–115. https://doi.org/10.1007/978-3-031-09729-4_6

  • Xinyi Fu, Cheng Xue, Qiuyi Yin, Yu Jiang, Ye Li, Yichen Cai and Weilin Sun. Gesture based fear recognition using non-performance dataset from VR horror games [C]//2021 9th International Conference on Affective Computing and Intelligent Interaction (ACII). IEEE, 2021: 1-8

  • Xinyi Wang, Cheng Xue, Tongxin Sun, Jingyi Liu, and Xinyi Fu*. 2021. Mobile Screen-based User Interface Design Guideline for Panoramic VR in Shopping Scene. In Proceedings of the 23rd International Conference on Mobile Human-Computer Interaction (MobileHCI '21). Association for Computing Machinery, New York, NY, USA, Article 34, 1–14. DOI:https://doi.org/10.1145/3447526.3472055

  • 付心仪,李岩,孙志军,杜鹃,王凤平,徐迎庆*.敦煌莫高窟烟熏壁画的数字化色彩复原研究[J].敦煌研究,2021(01):137-147.（FU Xinyi，LI Yan, SUN Zhijun, DU Juan, WANG Fengping, XU Yingqing. Digital Color Restoration of Soot Covered Murals in the Mogao Grottoes at Dunhuang [J]. Dunhuang Research, 2021(01):137-147.）

  • 付心仪,蔡天阳,薛程,张宇翔,徐迎庆*.基于BGRU-FUS-NN神经网络的姿态情感计算方法研究[J].计算机辅助设计与图形学学报,2020,32(07):1070-1079. （Fu, X., Cai, T., Xue, C., Zhang, Y., & Xu, Y. (2020). Research on body-gesture affective computing method based on BGRU-FUS-NN neural network. Journal of Computer-Aided Design and Computer Graphics, 32(7), 1070-1079.）

  • 付心仪,薛程,李希,张玥泽,蔡天阳.基于姿态的情感计算综述[J].计算机辅助设计与图形学学报,2020,32(07):1052-1061. （Xinyi F, Cheng X, Xi L, et al. A Review of Body Gesture Based Affective Computing[J]. Journal of Computer-Aided Design & Computer Graphics, 2020, 32(7): 1052-1061.）

  • Xinyi Fu, Yaxin Zhu, Zhijing Xiao, Yingqing Xu, and Xiaojuan Ma*. RestoreVR: Generating Embodied Knowledge and Situated Experience of Dunhuang Mural Conservation via Interactive Virtual Reality [C]. //In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI ’20). Association for Computing Machinery, New York, NY, USA, 2020:1–13.

  • 付心仪,麻晓娟*,孙志军.破损壁画的数字化复原研究——以敦煌壁画为例[J].装饰,2019(01):21-27.(Xinyi Fu, Xiaojuan Ma*, Zhijun Sun. Digital Restoration of Damaged Murals: Based on Dunhuang Murals [J]. ZHUANGSHI, 2019(01):21-27.)

  • 付心仪,刘世霞,徐迎庆*. 信息可视化的发展与思考[J]. 装饰, 2017(04):16-19. (Xinyi Fu, Shixia Liu, Yingqing Xu*. The Development and Reflection of Information Visualization [J]. ZHUANGSHI, 2017(04):16-19.)

Selected Patents：

  • Xinyi Fu, Xinyang Li, Yun Wang, Cheng Xue. Smart Home Chair (Design Patent). China, Patent No.: ZL 202330124880.3. Granted: Sep 26, 2023.

  • Yang Jiao, Xinyi Fu, Cheng Xue. Method and Device for Controlling Virtual Reality Camera Equipment. China, Patent No.: ZL 202211735771.6. Granted: Oct 28, 2025.

  • Jie Liu, Cheng Xue, Xinyi Fu. Chair for Affective Sensing (Design Patent). China, Patent No.: ZL 202230557005.X. Granted: Dec 27, 2022.

  • Jie Liu, Cheng Xue, Jihong Jeung, Mufei He, Xinyi Fu. Method, System, and Storage Medium for Affective Sensing. China, Patent No.: ZL 202211020906.0. Granted: Nov 5, 2024.

  • Xinyi Fu, Ying-Qing Xu, He Zhang, Cheng Xue, Shuang He, Zhe Sun, Yingting Gao. Integrated Experimental System for Smart Home and Data Processing Method. China, Patent No.: ZL 202210863171.1. Granted: Aug 26, 2024.

  • Ying-Qing Xu, Zhe Sun, Xinyi Fu. Image Processing System, Method, Equipment, Readable Storage Medium, and Program Product. China, Patent No.: ZL 202110762134.7. Granted: Sep 20, 2024.

  • Xinyi Fu, Yaxin Zhu, Ying-Qing Xu. Posture-Based Affective Recognition Method and Device. China, Patent No.: ZL 202010183739.6. Granted: May 23, 2023.

  • Xinyi Fu, Yi Han, Wen Wei, Ying-Qing Xu. Method and System for Restoring Damaged Murals. China, Patent No.: ZL 201710199268.6. Granted: Oct 30, 2020.

  • Gongming Wang, Ying-Qing Xu, Xinyi Fu, Wen Wei, Xian Yan, Yingxue Zhang. Method and Device for Nearest Color Search Based on NPsim Matrix. China, Patent No.: ZL 201510108710.0. Granted: Apr 10, 2018.

  • Gongming Wang, Ying-Qing Xu, Feng Xian, Xinyi Fu, Wen Wei, Xian Yan. Simulation Method for the Dynamic Formation Process of Smoked Murals. China, Patent No.: ZL 201510063876.5. Granted: Jul 7, 2017.

  • Gongming Wang, Ying-Qing Xu, Wen Wei, Xinyi Fu. Realistic Simulation Method for Smoking Effects. China, Patent No.: ZL 201410331612.9. Granted: Feb 15, 2017.


Part of published papers

More

Part of patents

More
Xinyi Fu
Associate Professor
Natural HCI, Affective computing, Cultural habitat preservation

Contact information

fuxy@mail.tsinghua.edu.cn