In March, at the Healthcare Information and Management Systems Society Global Health Conference, a consortium of healthcare leaders announced the organisation of the Trustworthy & Responsible AI Network (TRAIN). TRAIN intends to improve the use of artificial intelligence (AI) in health, making it safer and more reliable.

AI has the potential to profoundly alter the way we provide healthcare and share health information, but it has to be trustworthy. If the information isn’t reliable, the cost could be human lives, which is why trustworthiness has to stay at the forefront when AI is used for international development in general, and health work in particular.

But, retrieval-augmented generation (RAG) could start to shift the needle towards increased reliability. Simply put, RAG is a method for enhancing the outputs of large language models (LLMs), a kind of generative AI.

LLMs like ChatGPT are prone to errors called hallucinations, which occur when the model may not know the exact or complete answer to a question, so will fill in the gaps with incorrect information or make things up entirely. Additionally, responses lack traceability. In other words, even if an LLM produces an accurate response, users are left without the ability to know what information was used to do so—and very often, we need to know.

Unlike Chat GPT and other stand-alone LLMs that are trained on data taken from the internet, RAG models are trained on knowledge bases filled with documents and data that the user provides, “so you know that these are documents that are authoritative and are high quality,” says Palladium Data Science Director Jonathan Friedman. In Kenya, RAG is changing the way patients access information about HIV using an AI-based chatbot—part of the “Nishauri” application implemented by the Kenya Health Management Information Systems project (KeHMIS).

Nishauri is available to all patients living with HIV and allows them to see clinical histories, laboratory results, and look at upcoming appointments, among other functions. The application has always had a chatbot function, but implementing RAG has made it possible to provide patients with HIV-related content based on reputable sources. For KeHMIS, the application now allows patients to access reliable information on their own schedule and gives them the opportunity to ask sensitive questions they might not feel as comfortable bringing up face-to-face. Ultimately, the goal is for health information and data to become more accessible to a larger audience—especially a non-technical one who could benefit from it the most.

Friedman explains that the app’s previous chatbot leveraged a model that matched user questions, written in their own language, to a set of pre-written questions and answers. However, this version was limited in terms of content and frequently misinterpreted user intent which led to frustration and a lack of uptake. RAG has the potential to limit, although not completely erase, these kinds of issues. “With RAG, patients are no longer limited to prewritten questions and answers, and can ask the questions any way they want, whether in English or Swahili.”

More Accessible AI

RAG frameworks not only have the ability to improve the patient experience but can change the way organisations and programs do knowledge management.

USAID-funded Data for Implementation (Data.FI) is also implementing RAG into their work in collaboration with USAID to improve decision-making by making community-led monitoring reports more accessible. Their model is trained on internal reports produced by country programs. “We can code the model in such a way that it’s able to take user queries and questions, and then retrieve information from our internal corpus of documents,” says Anubhuti Mishra, Palladium Senior Technical Advisor, Data Science.

For an individual, combing through massive amounts of project-related data and reports often isn’t a good use of time or resources, but the RAG framework “can really help because of its ability to process huge numbers of documents. It presents a very good opportunity for knowledge management, being able to extract insights from a wealth of documents.”

Into the Unknown

“There’s a lot of interest” in the application of machine learning models in Kenya, says Benedette Otieno, a data scientist who worked on KeHMIS’ new chatbot. But there was a time when machine learning was considered the new kid on the block. “It took a lot of time to gain trust from the stakeholders,” Otieno explains, but the team is excited about this and future applications of the technology.

From the Data.FI perspective, Mishra reflects similarly, and adds that there is still much unproven in terms of the value addition and cost effectiveness of new AI-based programs. “There is a lot of reluctance to engage with these technologies because they’re new,” says Mishra, but at the same time, “there is a lot of enthusiasm to try these technologies because everyone seems to be using them.”

For Friedman, “it’s still early days” in terms of how generative AI has the potential to change the way information is shared and managed. “We don’t yet know everything that’s possible because there are computer scientists who are working now to develop new technologies and new tools that are going to create possibilities in years to come that we can’t even think about right now.”

Advances like RAG that make generative AI more reliable will continue to be discovered, tested, and released, and their future implementation in the healthcare space could make a world of difference in patients’ lives.