The use of AI in academic work has increased significantly in a relatively short time. Researchers and students are using AI to explore ideas, find literature, summarise papers and support their writing.
These tools can be helpful, but their outputs are not always reliable. One particular problem is the generation of references that look convincing but simply do not exist. An AI-generated citation may include plausible authors, a journal title, a publication year and even a DOI. In other cases, the publication exists but some of the details are incorrect.
These errors are not always easy to identify, particularly when reviewing a long reference list. I have therefore developed GhostCite, a small vibe coded open-source tool intended to support an initial check of references in academic documents.
What GhostCite does
GhostCite runs locally on a computer and accepts PDF or Word documents. It attempts to:
- identify the reference or bibliography section;
- separate the text into individual references;
- search for each reference in Crossref;
- classify the result as verified, partial match or not found;
- display a possible matching title and DOI; and
- produce a PDF summary for further review.
Crossref is a public database containing publication information supplied by publishers and other organisations. Checking against Crossref can help establish whether a reference corresponds with a known publication record.
The purpose of GhostCite is to support a first review. It may help identify references that would benefit from closer manual checking, but it does not replace that checking.
Running the tool
GhostCite currently runs through Docker. It can be downloaded and started using:
git clone https://github.com/DrDanL/GhostCite.git
cd GhostCite
./start.sh
The interface is then available at http://localhost:8000. A user can upload a PDF or .docx file, review the results and download a summary report.
A note on privacy
Documents are processed locally. The full PDF or Word file is not sent to another service, although individual reference strings are submitted to Crossref to search for possible matches.
Interpreting the results
A verified result means that GhostCite found a sufficiently strong match in Crossref. A partial match means that some details appear to align, but there may be differences in information such as the title or authors. A not found result means that the tool could not identify a suitable match.
“Not found” should not be interpreted as evidence that a reference is fabricated.
Crossref does not contain every legitimate source. Books, reports, websites, grey literature, preprints and some non-English publications may not be included. The formatting of a reference or problems extracting text from a PDF can also affect the result.
GhostCite checks whether a reference can be matched with a publication record. It does not establish whether the source supports the claim for which it has been cited. That assessment still requires the original source to be read and considered.
The results should therefore be treated as prompts for further checking rather than judgements about a document or its author.
An early-stage project
GhostCite is at an early stage of development and was built with AI-assisted coding. It will not handle every document or referencing style correctly, and its matching approach will need further testing and refinement.
I am making the code available on GitHub in case it is useful to others working with academic documents. Feedback and contributions are welcome, particularly where the tool fails to extract or match references correctly.
As AI becomes more common in research and education, reference checking is likely to become an increasingly important part of maintaining transparency and confidence in academic work. GhostCite is one exploratory attempt to support that process.