Welcome to Xingjian's Website!

Xingjian Diao

About Me



I am currently a Ph.D. student at Dartmouth College. I focus my research on Multimodal Learning as it has numerous practical applications that can benefit society in various ways. I am excited to carry out advanced research focused on training sophisticated, trustworthy, and high-quality machine learning models that can better interpret and comprehend the audio-visual world.


Prior to Dartmouth, I earned a Master's degree in Computer Science from Northwestern University (2021), advised by Prof. Nabil Alshurafa (Thank you, Nabil!), and a B.S. degree in Computer Science from the University of Pittsburgh (2020).



Interests

  • Musical Representation Learning
  • Natural Language and Speech Processing
  • Audio-Visual Question Answering
  • Video-Language Understanding

Education

  • Ph.D. in Computer Science, -Present
    Dartmouth College
  • M.S. in Computer Science, 2021
    Northwestern University
  • B.S. in Computer Science, 2020
    University of Pittsburgh
Publications

* indicates equal contribution

FT2TF: First-Person Statement Text-To-Talking Face Generation
Xingjian Diao, Ming Cheng, Wayner Barrios, SouYoung Jin
Winter Conference on Applications of Computer Vision (WACV) , 2025

Learning Musical Representations for Music Performance Question Answering
Xingjian Diao, Chunhui Zhang, Tingxuan Wu, Ming Cheng, Zhongyu Ouyang, Weiyi Wu, Jiang Gui
Findings of the Association for Computational Linguistics: Empirical Methods in Natural Language Processing (Findings of EMNLP), 2024

AlphaExpert: Assigning LoRA Experts Based on Layer Training Quality
Peijun Qing, Chongyang Gao, Yefan Zhou, Xingjian Diao, Yaoqing Yang, Soroush Vosoughi
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024

Analytical free vibration solutions of rectangular thin plates subjected to three edges rotationally-restrained and one edge free
Jinghui Zhang, Pin Gao, Xingjian Diao, Salamat Ullah, Jiapeng Li, Yuwei Zhang, Wenyue Qi
International Journal of Structural Stability and Dynamics (IJSSD), 2024

SAIC: Integration of Speech Anonymization and Identity Classification
Ming Cheng*, Xingjian Diao*, Shitong Cheng, Wenjun Liu
AI for Health Equity and Fairness: Leveraging AI to Address Social Determinants of Health, Cham: Springer Nature Switzerland, 2024

GluMarker: A Novel Predictive Modeling of Glycemic Control Through Digital Biomarkers
Ziyi Zhou*, Ming Cheng*, Xingjian Diao*, Yanjun Cui, Xiangling Li
Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), 2024
🎖 IEEE EMBC NextGen Scholar Award

Efflex: Efficient and Flexible Pipeline for Spatio-Temporal Trajectory Graph Modeling and Representation Learning
Ming Cheng*, Ziyi Zhou*, Bowen Zhang*, Ziyu Wang, Jiaqi Gan, Ziang Ren, Weiqi Feng, Yi Lyu, Hefan Zhang, Xingjian Diao
Conference on Computer Vision and Pattern Recognition (CVPR) Workshop: SG2RL2024, 2024

AV-MaskEnhancer: Enhancing Video Representations through Audio-Visual Masked Autoencoder
Xingjian Diao*, Ming Cheng*, Shitong Cheng
International Conference on Tools with Artificial Intelligence (ICTAI), 2023

An End-to-End Energy-Efficient Approach for Intake Detection With Low Inference Time Using Wrist-Worn Sensor
Boyang Wei, Shibo Zhang, Xingjian Diao, Qiuyang Xu, Yang Gao, Nabil Alshurafa
IEEE Journal of Biomedical and Health Informatics (JBHI), 2023

Building a Cloud-based Energy Storage System through Digital Transformation of Distributed Backup Batteries in Mobile Base Stations
Song Ci, Yanglin Zhou, Yuan Xu, Xingjian Diao, Junwei Wang
China Communications, 2020

Projects
Selected Projects
Intake Detection Tool with Multiple Classifiers

An Android application for wrist-worn devices to detect feeding patterns with low energy consumption and fast inference times. It applied template-based multi-centroid classifier which could provide an end-to-end battery-efficient approach for feeding detection.

Interactive Active Learning Annotation Tool

An interactive annotation software that utilizes active learning to reduce data labeling time and cost. The front-end was created with PyQt5 and pyqtgraph, offering features such as time synchronization and video frame-by-frame rewinding. The back-end, utilizing cv2, sklearn and xgboost, performed data processing, K-means clustering, and clustered entropy active learning.

iPADshiny

iPADshiny (integrated Protein Array Data management,analysis and visualization tools) is a desktop application that simplifies protein analysis for biologists. It integrates multiple algorithms, including the auto-antibody Profiling Analysis, and utilizes state-of-the-art computational methods for efficient and effective analysis.

Online Drawing Management System

An Online Drawing Management System with B/S structure and Windows OS, including features such as notice announcement, navigation menu, user and role management, flexible authorization, and online management and preview of large drawing documents. It automatically loads existing document storage structures, eliminating the need for manual entry of basic information. (Copyright: 2018SR071476)

Remote Voting System

A remote voting system that uses SMS texts to count unique votes while recording phone numbers to prevent repetitive voting, offering an accessible and transparent solution for remote voting scenarios.

Introvert

An inclusive online chat environment for introverted students, utilizing JavaScript, Python, and Google Cloud platform to implement anonymous chatting and user-friendly direct messaging features, aimed at promoting engagement and improving the chat experience for introverted individuals.

Teaching

TA indicates Teaching Assistant

Contact