An Enhanced Hybrid Framework for Intelligent Research Paper Plagiarism Detection: Integrating TF-IDF Lexical Vectorization, SBERT Deep Semantic Embeddings, and Explainable Risk Analytics

Varun K S; Varun R Shivannanavar

Journal of Information, Data, and Security Trends

An Enhanced Hybrid Framework for Intelligent Research Paper Plagiarism Detection: Integrating TF-IDF Lexical Vectorization, SBERT Deep Semantic Embeddings, and Explainable Risk Analytics

Volume 2 |
Issue 1 |
Pages 1-11 |
January 2026

Abstract

Traditional plagiarism checkers are built around verbatim string and n-gram matching, which do not account for paraphrasing and other forms of semantic rewriting, making research integrity screening susceptible to bypassing. This paper introduces an enhanced research paper plagiarism detection system implemented in Python, Flask, PyMuPDF, Scikit-learn, and Sentence Transformers. The tool is designed to address the mentioned limitations by utilising a two-step approach that integrates a Lexical Similarity Engine (TF-IDF and cosine similarity) and a Deep Semantic Engine (SBERT paraphrase-MiniLM-L3-v2, with 384-dimensional embeddings). By combining the outputs of both engines, the system calculates a composite plagiarism score, a Research Integrity Index score, and implements a tiered risk classification. Additionally, the tool offers four key features that distinguish it from commercial alternatives, including section-wise analytics, a sentence highlighter, a citation-aware filtering module, and an institutional-level analytics dashboard. An instrumented end-to-end pipeline utilised a 42-paper corpus to demonstrate the system’s applicability. The results showed that all six pipeline stages took a median of 260 ms to process per document. Testing on identical, heavily paraphrased, and unrelated documents revealed that the hybrid approach maintained an approximate detection capability of 68% for semantically rewritten texts compared to 6% for the TF-IDF baseline, while citation-aware filtering decreased the similarity score of an appropriately cited excerpt from 54.9% to 11.6%. SQLite and MySQL compatibility and light deployment requirements allow the system to operate on a standard server without requiring extensive optimisation.

References

N. Reimers and I. Gurevych, “Sentence-BERT: Sentence embeddings using Siamese BERT-networks,” in Proc. EMNLP-IJCNLP, 2019, pp. 3982–3992. doi: 10.18653/v1/D19-1410.
A. Vaswani et al., “Attention is all you need,” in Proc. NeurIPS, 2017, pp. 5998–6008.

G. Salton, A. Wong, and C. S. Yang, “A vector space model for automatic indexing,” Commun. ACM, vol. 18, no. 11, pp. 613–620, 1975. doi: 10.1145/361219.361220.

F. Pedregosa et al., “Scikit-learn: Machine learning in Python,” J. Mach. Learn. Res., vol. 12, pp. 2825–2830, 2011.

M. Grinberg, Flask Web Development. O'Reilly Media, 2018.

J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” in Proc. NAACL-HLT, 2019, pp. 4171–4186. doi: 10.18653/v1/N19-1423.

E. Loper and S. Bird, “NLTK: The natural language toolkit,” in Proc. ACL Workshop on Effective Tools and Methodologies for Teaching NLP, 2002.

A. Barrón-Cedeño and P. Rosso, “On automatic plagiarism detection based on n-grams comparison,” in Proc. ECIR, 2009, pp. 696–700. doi: 10.1007/978-3-642-00958-7_69.

J. Ramos, “Using TF-IDF to determine word relevance in document queries,” in Proc. First Instructional Conf. Machine Learning, 2003.

T. Foltýnek, N. Meuschke, and B. Gipp, “Academic plagiarism detection: A systematic literature review,” ACM Comput. Surv., vol. 52, no. 6, pp. 1–42, 2020. doi: 10.1145/3345317.

S. M. Alzahrani, N. Salim, and A. Abraham, “Understanding plagiarism linguistic patterns, textual features, and detection methods,” IEEE Trans. Syst., Man, Cybern. C, vol. 42, no. 2, pp. 133–149, 2012. doi: 10.1109/TSMCC.2011.2134847.

M. Potthast, T. Gollub, M. Hagen, and B. Stein, “Overview of the 5th international competition on plagiarism detection,” in CLEF, 2013.

M. Chong, L. Specia, and R. Mitkov, “Using natural language processing for automatic detection of plagiarism,” in Proc. 4th Int. Plagiarism Conf., 2010.

T. Vrbanec and A. Meštrović, “Corpus-based paraphrase detection experiments and review,” Information, vol. 11, no. 5, p. 241, 2020. doi: 10.3390/info11050241.

Z. Lan, M. Chen, S. Goodman, K. Gimpel, P. Sharma, and R. Soricut, “ALBERT: A lite BERT for self-supervised learning of language representations,” in Proc. ICLR, 2020.

Y. Liu et al., “RoBERTa: A robustly optimized BERT pretraining approach,” arXiv:1907.11692, 2019.

C. Raffel et al., “Exploring the limits of transfer learning with a unified text-to-text transformer,” J. Mach. Learn. Res., vol. 21, no. 140, pp. 1–67, 2020.

T. Brown et al., “Language models are few-shot learners,” in Proc. NeurIPS, 2020, pp. 1877–1901.

T. Gao, X. Yao, and D. Chen, “SimCSE: Simple contrastive learning of sentence embeddings,” in Proc. EMNLP, 2021, pp. 6894–6910. doi: 10.18653/v1/2021.emnlp-main.552.

PyMuPDF / Artifex Software, “PyMuPDF Documentation,” 2024.

W. McKinney, “Data structures for statistical computing in Python,” in Proc. SciPy, 2010, pp. 56–61. doi: 10.25080/Majora-92bf1922-00a.

Pallets Projects, “Werkzeug Security Utilities Documentation,” 2024.

Docker Inc., “Docker Documentation: Containerization Best Practices,” 2024.

J. Johnson, M. Douze, and H. Jégou, “Billion-scale similarity search with GPUs,” IEEE Trans. Big Data, vol. 7, no. 3, pp. 535–547, 2021. doi: 10.1109/TBDATA.2019.2921572.

R. Smith, “An overview of the Tesseract OCR engine,” in Proc. ICDAR, 2007, pp. 629–633. doi: 10.1109/ICDAR.2007.4376991.

A. Conneau et al., “Unsupervised cross-lingual representation learning at scale,” in Proc. ACL, 2020, pp. 8440–8451. doi: 10.18653/v1/2020.acl-main.747.

Celery Project, “Celery: Distributed Task Queue Documentation,” 2024.

SQLAlchemy Authors, “SQLAlchemy ORM Documentation,” 2024.

E. Bertino and R. Sandhu, “Database security—concepts, approaches, and challenges,” IEEE Trans. Dependable Secure Comput., vol. 2, no. 1, pp. 2–19, 2005. doi: 10.1109/TDSC.2005.9.

Journal of Information, Data, and Security Trends

An Enhanced Hybrid Framework for Intelligent Research Paper Plagiarism Detection: Integrating TF-IDF Lexical Vectorization, SBERT Deep Semantic Embeddings, and Explainable Risk Analytics

Abstract

References

Access options

Author information

Affiliations

Publisher Information

Plagiarism Detection; Semantic Textual Similarity; Sentence-BERT Embeddings; TF-IDF Cosine Similarity; Citation-Aware Filtering; Explainable Risk Analytics

Publisher : Skilldux Publisher