Project Experience
Publication
Where Fact Ends and Fairness Begins: Redefining AI Bias Evaluation through Cognitive Biases
Jen-tse Huang, Yuhang Yan, Linqi Liu, Yixin Wan, Wenxuan Wang, Kai-Wei Chang, Michael R. Lyu
| EMNLP Findings 2025 | DOI: 10.48550/arXiv.2502.05849 | code |
Selected Projects
- Can AI Agent Fit in Human Society? [Slides 1] [Slides 2] Apr. 2024 - present
Supervisor: Prof. Michael R. Lyu & Dr. Jen-tse Huang (ARISE@CUHK) Hong Kong SAR- Analyzed gender and racial biases in generative models using demographic data, uncovering issues of over-balance in content generation.
- Developed open-source fairness benchmark based on 3 cognitive errors and 19 statistics, enhancing bias detection in model outputs. [1]
- Assessed reliability and factual accuracy of LLMs when using web search for real-world and real-time information retrieval.
- Evaluation on the Vulnerability of Current Generative Models [Slides] Feb. 2024 - Jun. 2024
Supervisor: Prof. Sabine Süsstrunk & Dr. Daichi Zhang (IVRL@EPFL) Lausanne, Switzerland- Tested defense mechanisms of 7 generative models by replicating 8 “jailbreak” attacks methods, exposing ongoing security vulnerabilities.
- Studied gender and racial biases in images generated by 3 diffusion models, highlighting the need for more diverse training data.
- Uncovered demographic gaps in the accuracy of 2 fake image detectors, emphasizing the importance of robustness beyond bias mitigation.
- Efficient Video Analytics [Poster] Jun. 2023 - Sep. 2023
Supervisor: Prof. Eric Lo (CPII@CUHK) Hong Kong SAR- Won the Best Project Award 2023 among 58 undergraduate projects.
- Developed a multimodal AI system using CLIP and OWL-ViT models for text or image based lost-and-found queries at Hong Kong Airport.
- Created custom datasets and evaluation protocols to identify the most accurate and efficient Zero-Shot Object Detection and NLP models.
- Optimized frame detection with designing ran algorithm that improved fault tolerance and achieved 90%+ recall within minutes.