Readability of published vs. artificial intelligence (AI)-generated plain-language summaries (PLS)

This page contains a poster presented at the 20th Annual Meeting of ISMPP (International Society for Medical Publication Professionals), on April 28-May 1, 2024.

Authors and affiliations

Authors: Shobana Ganesan, Nilima Vyas, Swapna Ganduri, Charvin D’souza, and Namita Bose

Affiliation: Cactus Life Sciences, Cactus Communications, Mumbai, India

Abstract

Objective: PLS play an important role in increasing the accessibility of medical publications to a wide audience. We assessed the readability of PLS in published literature vs. that of PLS developed using generative AI (genAI).

Research design and methods: We used the following search string to identify journal articles with PLS indexed on PubMed (2018-2023): ((plain language summar*) OR PLS OR (plain-language summar*) OR (lay summar*)). Only PLS with >50 words were analyzed. For these PLS, we generated corresponding AI PLS using the published abstracts as prompts on ChatGPT 3.5 (OpenAI). The readability of both PLS was determined using Flesch-Kincaid scores (Microsoft Word) and compared using a t-test (Microsoft Excel). Additionally, we surveyed medical publication professionals regarding the use of readability indices and genAI for developing PLS.

Results: Of 295 search results, 211 had an associated PLS and 180 were analyzed. For published PLS, the mean (SD) Flesch-Kincaid reading-ease score was 20.8 (14.5) and the grade level was 15.9 (2.8). For AI PLS, a significant improvement in mean reading-ease score (35.5 [13.3]) and grade level (12.9 [2.3]) was observed (both P<0.05) without any meaning changes. Of 63 survey respondents, 46 (73%) were involved in developing PLS. Of these, 80% were familiar with and 50% reported using readability indices, but 20% did not consider any specific reading grade level while developing PLS. Moreover, 65% had not considered using genAI to assist with developing PLS, with data confidentiality concerns being a common barrier.

Conclusions: Readability scores of published PLS tend to be much higher than recommended levels. GenAI tools could help achieve the desired readability levels provided barriers to their use are addressed.

Keywords: Plain Language Summaries (PLS), Generative AI, Survey

Access more details of this study here.

Cactus Life Sciences

Cactus Life Sciences is a medical communication company that provides scientific strategy and content across the healthcare continuum, anywhere in the world ─ with a focus on science, innovation, and efficiency. We work alongside leading healthcare companies to establish the optimal role of medicines and encourage positive behaviors (physician and patient) that improve patient outcomes.