大语言模型的中文文本简化能力研究

杨尔弘,朱君辉,朱浩楠,宗绪泉,杨麟儿

语言战略研究 ›› 2024, Vol. 9 ›› Issue (5) : 34-47.

欢迎访问《语言战略研究》官方网站!今天是
语言战略研究 ›› 2024, Vol. 9 ›› Issue (5) : 34-47. DOI: 10.19689/j.cnki.cn10-1361/h.20240503
语言战略研究

大语言模型的中文文本简化能力研究

  • 杨尔弘,朱君辉,朱浩楠,宗绪泉,杨麟儿
作者信息 +

A Study on the Evaluation of Large Language Models’ Capabilities in Chinese Text Simplification

  • Yang Erhong, Zhu Junhui, Zhu Haonan, Zong Xuquan and Yang Lin’er
Author information +
文章历史 +

摘要

大语言模型为自动文本简化提供了新思路。为了探究大语言模型的中文文本简化能力,本研究构建了中文篇章级文本简化数据集,对其中的平行文本对进行了特征分析;在此基础上,设计大语言模型自动文本简化实验,采用零样本、少样本、少样本+词表和少样本+规则这4种提示策略,综合已有的和本研究特有的语言特征评估指标,测评了6种国内外常用大语言模型在不同提示策略下的中文文本简化能力。研究发现,少样本提示策略在文本特征上表现最佳,显著提高了信息保存度;在提示中加入外部词表,有助于大语言模型使用相对简单的词语;在提示中融入简化规则,能使大语言模型使用更简洁的句法结构。不同的大语言模型在难度控制和语义保留程度上各有优势与局限,但在语篇衔接与连贯和段落划分上与人类专家存在明显差距,且均出现了不同程度的幻觉现象。未来仍需构建较大规模的高质量中文简化数据集,多角度诱导语言大模型的文本简化能力。

Abstract

Large language models (LLMs) offer new approaches for automatic text simplification. To explore the capabilities of LLMs in simplifying Chinese texts, this study constructed a Chinese passage-level text simplification dataset and conducted a feature analysis of the parallel text pairs within it. Based on this, an experiment was designed to assess the automatic text simplification performance of LLMs using four prompting strategies: zero-shot, few-shot, few-shot with lexicon, and few-shot with rules. The study evaluated the performance of six commonly used domestic and international LLMs in Chinese text simplification under different prompting strategies, utilizing a combination of existing and study-specific linguistic feature evaluation metrics. The findings revealed that the few-shot prompting strategy performed best in terms of text features, significantly enhancing information retention. Incorporating external lexicons in the prompts helped the LLMs use relatively simpler words, while integrating simplification rules enabled the LLMs to employ more concise syntactic structures. Different LLMs exhibited distinct strengths and limitations in controlling complexity and preserving semantics, but all showed a noticeable gap compared to human experts in discourse cohesion, coherence, and paragraph segmentation, with varying degrees of hallucination also observed. Future research should focus on constructing larger-scale, high-quality Chinese simplification datasets and exploring multi-faceted approaches to enhance the text simplification capabilities of LLMs.

关键词

中文文本简化 / 大语言模型 / 语言特征分析

Key words

automatic text simplification / large language models (LLMs) / linguistic profiling

引用本文

导出引用
杨尔弘,朱君辉,朱浩楠,宗绪泉,杨麟儿. 大语言模型的中文文本简化能力研究[J]. 语言战略研究. 2024, 9(5): 34-47 https://doi.org/10.19689/j.cnki.cn10-1361/h.20240503
Yang Erhong, Zhu Junhui, Zhu Haonan, Zong Xuquan and Yang Lin’er. A Study on the Evaluation of Large Language Models’ Capabilities in Chinese Text Simplification[J]. Chinese Journal of Language Policy and Planning. 2024, 9(5): 34-47 https://doi.org/10.19689/j.cnki.cn10-1361/h.20240503

基金

国家语委重大科研项目“大语言模型的评测技术和方法研究”(ZDA145-17)。

Accesses

Citation

Detail

段落导航
相关文章

/