Large language models (LLMs) have shown high-quality semantic comprehension ability and extensive external knowledge, which has been incorporated into recommendation systems as multiple functions. However, existing bias evaluation pipelines designed for conventional recommendation systems are not fully applicable to recommendation systems via LLM (RecLLM) and most bias mitigation methods are limited to a single intervention stage, rendering them inadequate for addressing the overall bias of the complex RecLLMs. Xinye will introduce a comprehensive evaluation framework designed to assess the biases within RecLLMs and their constituent sub-modules (Wanyan et al., 2025; Wanyan et al., 2026; Wanyan et al., 2026). In addition, a calibrated synthetic benchmark dataset, leveraging LLMs, will be developed to facilitate the bias evaluation and mitigation experiments.
Xinye is a scholarship recipient of the ARC Centre for Automated Decision-Making and Society (ADM+S) and is supervised by Dr. Chenglong Ma, Prof. Jeffrey Chan and Dr. Danula Hettiachchi.
References
CIKM
Temporal-Aware User Behaviour Simulation with Large Language Models for Recommender Systems
Large Language Models (LLMs) demonstrate human-like capabilities in language understanding, reasoning, and generation, driving interest in using LLM-based agents to simulate human feedback in recommender systems. However, most existing approaches rely on static user profiling, neglecting the temporal and dynamic nature of user interests. This limitation stems from a disconnect between language modelling and behaviour modelling, which constrains the capacity of agents to represent sequential patterns. To address this challenge, we propose a Dynamic Temporal-aware Agent-based simulator for Recommender Systems, DyTA4Rec, which enables agents to model and utilise evolving user behaviour based on historical interactions. DyTA4Rec features a dynamic updater for real-time profile refinement, temporal-enhanced prompting for sequential context, and self-adaptive aggregation for coherent feedback. Experimental results at group and individual levels show that DyTA4Rec significantly improves the alignment between simulated and actual user behaviour by modelling dynamic characteristics and enhancing temporal awareness in LLM-based agents.
@inproceedings{Wanyan2025-ta,title={Temporal-Aware User Behaviour Simulation with Large Language Models for Recommender Systems},author={Wanyan, Xinye and Hettiachchi, Danula and Ma, Chenglong and Xu, Ziqi and Chan, Jeffrey},booktitle={Proceedings of the 34th ACM International Conference on Information and Knowledge Management},series={CIKM '25},year={2025},publisher={ACM},doi={10.1145/3746252.3760878},}
SIGIR
Task-Aware Automated User Profile Generation for Recommendation Simulation Using Large Language Models
Large Language Model (LLM)-based agent simulation has emerged as a promising approach to meet the increasing demand for real-time and rigorous evaluation in modern recommender systems. A typical LLM-driven simulation framework comprises three essential components: the profile module, memory module, and action module. However, existing studies have primarily concentrated on enhancing the memory and action modules, with limited attention to profile generation, which plays a pivotal role in ensuring realistic agent behaviours and aligning simulated interactions with real user dynamics. Moreover, the scarcity of datasets specifically designed for recommendation simulations has led to heavy reliance on manually crafted profiles, significantly limiting the scalability and generalisability of simulation frameworks across different datasets. To address these challenges, this work proposes an Automated Profile Generation Framework for Recommendation Simulation, APG4RecSim, that constructs realistic, coherent, and robust user profiles with minimal supervision. Extensive experiments on three benchmark datasets demonstrate that APG4RecSim achieves the best overall performance on discrimination, ranking, and rating tasks, improving ranking quality by up to 7% in nDCG@10 and reducing rating distribution divergence by 8% in JSD compared to existing profile-generation baselines. Beyond overall performance gains, our results show that profiles generated by APG4RecSim are resilient to popularity- and position-induced biases and maintain stable performance across datasets and different LLMs.
@inproceedings{Wanyan2026-ta,title={Task-Aware Automated User Profile Generation for Recommendation Simulation Using Large Language Models},author={Wanyan, Xinye and Ma, Chenglong and Hettiachchi, Danula and Xu, Ziqi and Chan, Jeffrey},booktitle={Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval},series={SIGIR '26},year={2026},publisher={ACM},doi={10.1145/3805712.3809707},}
CIKM
Beyond Static Profiles: Intent-Aware Behavioural Control for LLM-based Recommendation Simulation
Large language models (LLMs) are increasingly used in recommender systems for user modelling, ranking, and simulation. Among these applications, LLM-based user simulators provide a flexible way to evaluate recommender systems without relying on live user feedback. However, existing simulators typically reduce user behaviour to static preference matching based on fixed user profiles, overlooking the fact that real decisions may also reflect recent preference shifts and exploratory behaviour. To address this limitation, we propose IntRecSim (Intent-aware Recommendation Simulation), a plug-in behavioural control framework for LLM-based user simulation. IntRecSim introduces a lightweight intent mechanism that guides whether the simulated decision should follow long-term preferences, recent behavioural evidence, or exploratory behaviour. Since standard recommendation datasets lack explicit intent annotations, we construct pseudo-intent supervision from historical interactions using hybrid category and semantic preference signals. The inferred intent is injected into the LLM simulator as a dynamic control signal, making the framework compatible with different profile-generation backbones. Experiments on three benchmark datasets, covering discrimination and ranking tasks show that IntRecSim improves simulation performance in most evaluated settings. Further analysis highlights the importance of explicit behavioural control for modelling realistic user decisions in LLM-based recommender simulation.
@inproceedings{Wanyan2026-bs,author={Wanyan, Xinye and Ma, Chenglong and Hettiachchi, Danula and Xu, Ziqi and Chan, Jeffrey},title={Beyond Static Profiles: Intent-Aware Behavioural Control for LLM-based Recommendation Simulation},year={2026},isbn={9798400725395},publisher={ACM},doi={10.1145/3799682.3840925},booktitle={Proceedings of the 35th ACM International Conference on Information and Knowledge Management},pages={5074–5085},numpages={12},keywords={recommender system, generative ai, user behaviour simulation},location={Rome, Italy},series={CIKM '26},}