FST Graduate Research Topic


Research Topic: Authorship Forensics - An Open-Source Moodle Plug-in
Suggested By: Maiga Chang
Last Update: March 19th, 2025


===============================


 


Route: This research topic can be taken by essay-route, project-route and thesis-route student. 


 


NOTE: if project-route student take this topic, a simple usability analysis will need to be done to assess how users perceived the usability of the Moodle plugin with a small group of test users (12 to 15 users).


 


NOTE: if thesis-route student wants to take this topic, then besides the usability analysis, an experiment in using the Moodle plugin in a course needs to be done and teacher's thoughts and feedback needs to be collected (via online interview) and analyzed (with qualitative data analysis approach).


 


===============================


 


In this evolving landscape of text generation, distinguishing between human-written and ChatGPT-generated content has become increasingly important. Our research group has analyzed and developed a novel approach to authorship attribution, leveraging both Statistical Natural Language Processing (SNLP) and Convolutional Neural Networks (CNN) techniques to differentiate between documents written by humans and ChatGPTs. The research uses 212 abstracts of academic papers written and published by the research group as the human-written set and asks both ChatGPT 3.5 and 4 to generate correspondent abstracts based on paper titles as AI-written set. Models are trained to classify human and AI-written abstract texts in 2-class (i.e., human and ChatGPT) and 3-class (i.e., human, ChatGPT 3.5, and ChatGPT 4) based on their part-of-speech tag frequency distribution patterns. The 2-class model is well-trained in less than ONE minute (i.e., 56.82 seconds) and the 3-class model is well-trained in 7 minutes and 26.076 seconds. The results demonstrate a significant ability of the models to distinguish between human and AI-written text, with precision 0.9682 (F0.5 score 0.95) for the 2-class (human and ChatGPT) testing subset and precision 0.9806 (F0.5 score 0.96) in the 3-class (human, ChatGPT 3.5, and ChatGPT 4) testing subset. The proposed 3-stage Authorship Forensics approach has also been implemented as an open access web application to allow teachers and users to either training their own models or using the existing trained model to get some advice on how the model considers a piece of given text is written by human or AI.


 


The results show that the computer can be trained quickly with acceptable amount of data like existing research use and the trained model can effectively differentiate texts from an unseen subset in terms of human or ChatGPT-written. Since the trained model’s performances have outperformed almost all existing research, this research aims to design and develop 


 


(A) the correspondent functions that can 


 




  1. assigns license key for a registered user as well as when the user registers an account at Authorship Forensic Portal;



  2. has a separate PHP that provides a list of available public models as well as the models belong to a license key (if provided) in JSON;



  3. has a separate PHP that receives the recognition job request package in JSON format;


    1. interprets the request package;



    2. checks the content integrity;



    3. validates the license key included in the package;



    4. provides Authorship Forensic the text and chosen model to do the recognition; and,



    5. prepares the result package in JSON format and return it to the client.






 


B) an open-source Moodle plugin that 


 




  1. Allows Moodle site administator to enter the license key for the plugin to include in the recognition job request package. Teachers will not be able to see the license key.





  2. Allows teacher to setup it automatically requesting recognition result from Authorship Forensic service provider for one or more learning activities (e.g., the discussion posts, assignment submissions, etc.). 





  3. Prepares the request package and send it to Authorship Forensic service provider and record the jobs and their status. 





  4. Interpret the result package to update job status and store results.





  5. Provides teachers a status list of the reconigition jobs and allows them to filter and shorten the list by choosing dates or date range, checking one or more learning activity discussion, selecting specific forum(s) and/or assignment, choosing Moodle users, and using keywords in free text.





  6. Shows teachers a particular job's details like the post content or a link to the uploaded assignment with the received recognition result when they click a job whose status is "complete".