• 中文核心期刊
  • 中国科技核心期刊
  • RCCSE中国核心学术期刊
  • Scopus, DOAJ, CA, AJ, JST收录期刊
高级检索

面向煤矿安全隐患文本的预训练语言模型构建

Construction of pre-training language model for coal mine safety hidden danger text

  • 摘要: 煤矿各类安全管理信息化平台积累的大量非结构化文本数据目前并没有得到充分利用。为充分挖掘煤矿安全隐患文本知识,提出一种基于领域术语词掩码语言建模(DP-MLM)和句子顺序预测建模(SOP)学习机制的煤矿安全领域预训练语言模型(CoalBERT)。利用收集到的110万余条煤矿隐患排查记录数据和自构建的1 328个领域术语词词典进行模型训练,并在煤矿安全隐患文本分类和命名实体识别2个任务上分别进行对比实验。研究结果表明:在文本分类实验中,CoalBERT模型总体结果的精准率、召回率和F1值较双向编码器表征法预训练模型(BERT)分别提高0.34%、0.21%、0.27%;在命名实体识别实验中,CoalBERT模型的精准率和F1值较BERT模型分别提高3.84%、2.13%。CoalBERT模型能够有效提升煤矿安全隐患文本语义理解能力,可为煤矿安全领域文本挖掘相关任务场景提供基础参考。

     

    Abstract: At present, a large amount of unstructured text data accumulated by various safety management information platforms in coal mines has not been fully utilized. In order to fully explore the text knowledge of coal mine safety hidden danger,a pre-training language model (CoalBERT) based on the learning mechanism of domain term word-mask language modeling (DP-MLM) and sentence order predictive modeling (SOP) was proposed. The model was trained by the collected data of more than 1. 1 million records of coal mine hidden danger investigation and the self -constructed dictionary of 1 328 domain terms,and comparative experiments were conducted respectively on the two tasks of coal mine safety hidden danger text classification and named entity recognition. The research results show that in the text classification experiment,the accuracy rate, recall rate and F1 value of the overall results of the CoalBERT model are increased by 0. 34%,0. 21% and 0. 27% respectively compared with the pre-training model of the bidirectional encoder representation from transformers (BERT). In the named entity recognition experiments,the accuracy rate and F1 value of CoalBERT model are 3. 84% and 2. 13% higher than BERT model, respectively. The CoalBERT model can effectively enhance the text semantic understanding ability of coal mine safety hidden danger text and can provide a basic reference for text mining related task scenarios in the field of coal mine safety.

     

/

返回文章
返回