School of Computing, Ulster University
Artificial Intelligence (AI) holds significant potential for health innovation, but safety-critical healthcare demands more than just accurate predictions. Clinicians and health systems need AI solutions that are transparent, handle real-world uncertainty, and provide reasoning that can be examined and accepted as credible.
Researchers in the School of Computing at Ulster University, led by Professor Jun Liu, are tackling this challenge in collaboration with Southwest Jiaotong University. Together, they are developing a new generation of trustworthy AI designed specifically for complex medical decision-making.
Standard AI models often act as “black boxes” by delivering an answer without revealing how they arrived at it or confirmation of data validity. In healthcare, where data is frequently incomplete, this opacity creates significant risks. The collaborative research reported here addresses this by merging two powerful approaches.
Firstly, offering interpretable and uncertainty-aware decision reasoning, through Belief Rule-Based (BRB) and Extended Belief Rule-Based (EBRB) systems and their recent integration with large language models (LLMs).
Secondly, principle-based self-credible AI reasoning, through the Automated Theorem Generation (ATG) framework and LLM pipeline developed in collaboration with Professor Yang Xu and his team at Southwest Jiaotong University in China.
The work combines belief rule-based systems, uncertainty modelling, data-driven AI, automated theorem generation and LLMs. Together, these technologies enable AI to use both data and expert knowledge, manage uncertain information, and provide reasoning that is more transparent and credible.
A prime example of this technology in action is in smart-homes and wearable monitoring for early signs of health deterioration. In smart home-care settings, sensor data is not perfect whereby a wearable might drop signal, or environmental sensors might produce conflicting signals. The Ulster-SWJTU framework processes this uncertain data transparently, alerting healthcare teams to subtle declines in a patient’s health while clearly stating the uncertainty level of the data and demonstrating that the recommendation follows sound logical principles.
Instead of presenting an answer that merely looks convincing, the AI provides clinicians with the tools to inspect, assess, and challenge its conclusions.
Translating trustworthy AI into clinical practice requires overcoming real-world hurdles. Human oversight into decision making is critical to use human experience and expertise but this is time consuming and often creates blockages in workflows in clinical settings; AI must enhance clinical judgment rather than replace it or add unnecessary administrative friction.
The natural resource consumption to fuel AI has been featured increasingly in the media, where land is being used to build data centres, and high volumes of water usage highlighted. As LLMs grow, so too do their computational and environmental footprints. Developing greener, resource-efficient AI models remains a primary focus of this research.
Patient privacy, ethics and data regulation remains an ongoing challenge. AI developments require full alignment with GDPR, patient privacy, and safety standards as essential. Policy and Strategy changes are creating conditions to protect individual’s data compliance and safety; the Universities continue to support these strategies and feed into local working groups and policy creation to offer ethical AI solutions.
EU AI regulatory sandboxes could also help bridge the gap between research and safe clinical adoption by providing controlled environments to test innovative AI systems under regulatory supervision.
The ultimate goal is an integrated, trustworthy AI ecosystem where data-driven learning, clinical knowledge, and self-checking logic work in harmony. By ensuring AI-supported recommendations are explainable, uncertainty-aware, and credible; this collaborative research at Ulster University and Southwest Jiaotong University aims to build the foundation of trust required for large-scale AI adoption in global health systems.