Detecting AI-Generated Text: Can AI-Content-Detection Software Identify Text from ChatGPT, Codex, BERT, and T5?

In today's world, AI technology is thriving, and ChatGPT, alongside other tools like Codex, has taken the world by storm.

Though ChatGPT and its counterparts are capable of performing a wide range of tasks, they particularly excel at writing articles. While this presents numerous benefits, several drawbacks also exist.

The most significant challenge posed by AI-generated written content, such as that from ChatGPT and BERT, for the publishing industry is the spread of false information.

Articles created by ChatGPT, Codex, or any other AI tool might contain factual inaccuracies or outright falsehoods.

Imagine launching a medical blog without any prior experience in the field, or starting a law advice column without legal expertise, and using ChatGPT or another AI tool to generate articles.

Mistakes could be present in the content that only qualified medical professionals can identify. If such content gains traction on social media or ranks highly in search results, those who read and follow any poor medical advice may be adversely affected.

Another potential issue is how students may use ChatGPT and similar tools in their academic writing.

The quality of education is significantly diminished if someone can write an essay merely by following a prompt (without any real effort), as learning about a subject and expressing personal ideas are essential aspects of essay writing.

Even before ChatGPT was released, many publishers employed AI tools to generate content. While some are transparent about this practice, others may not be.

---

TLDR summary:  

Six AI content detection programs were tested to determine the viability of using ChatGPT, Codex, and other AI tools for writing articles.TLDR content is at the bottom of the page.

---

BankRate started publishing articles created by AI, clearly disclosing this information online. Over 160 articles have been identified, with the earliest one dating back to April 2022. It would be interesting to discover how these articles perform in terms of ranking.

Recently, Google updated its guidelines to clarify that its policies do not always apply to AI-generated content.

This led to the decision to test existing tools to ascertain the current state of technology in identifying content produced by ChatGPT, Codex, BERT, and AI in general.

ChatGPT was used to generate written responses to the following questions, which were then run through various detection programs.

• What is local SEO? Why is it important? What are the best local SEO techniques? 
• Write an essay on the invasion of Egypt by Napoleon Bonaparte." 
• What are the main differences between the Samsung Galaxy and the iPhone?

The following AI detection services were used to detect if generative AI was at work":

1. Writer.com
2. Copyleaks
3. Contentatscale.ai 
4. Originality.ai 
5. GPTZero
6. AI Text Classifier from OpenAI

Here are the results for each tool. Let's take a closer look at how it went with each of them:

1. Writer.com

For the first prompt's response, Writer.com failed to determine that the content generated by ChatGPT was 94% human-generated.

In the case of the second prompt, it successfully identified the content as AI-written.

The final prompt was also unsuccessful.

However, Writer.com accurately identified genuine human-written text as being 100% human-generated during testing.

2. Copyleaks

Copyleaks successfully identified all three prompts as AI-authored content.

3. Contentatscale.ai

Contentatscale.ai did an excellent job of identifying all three prompts as AI-written content, even though the first question was given a 21% human score.

4. Originality.ai

Originality.ai successfully identified all three prompts as AI-generated content.

Additionally, when tested with actual human-written text, it recognized the text as 100% human-generated, which is crucial.

No instances of copying were found by Originality.ai. This could change in the future.

Eventually, people may use the same prompts to generate AI-written content, likely resulting in many similar responses. When these articles are published, plagiarism detection software will catch them.

5. GPTZero

Edward Tian created a non-profit tool specifically designed to detect articles generated by ChatGPT. It accurately identified all three prompts as AI-generated.

A more in-depth analysis of issues found was provided by GPTZero, including sentence-by-sentence evaluations.

6. AI Text Classifier from OpenAI

AI-detection tools uncovering each other

To conclude, let's see how OpenAI identifies responses that it has generated on its own.

For the first and third prompts, it labeled the responses as "possibly-AI generated," indicating that an AI was involved.

Surprisingly, however, it misidentified the second prompt, labeling it as "unlikely AI-generated." Tests were conducted with various prompts, and it was found that, at the time of writing, some of the aforementioned tools detected AI-generated content more accurately than OpenAI's own tool.

The tool had been released only a day earlier at the time of testing. It is likely that it will be refined and perform better in the future.

Conclusion: The best AI content creation tools currently available, such as ChatGPT and Codex, can be detected by AI content detection programs (with varying degrees of success).

Although it remains possible to create content using ChatGPT or Codex and then paraphrase it to make it untraceable, doing so might take nearly as much time as starting from scratch, rendering the benefits less immediate.

Consider how easy it would be for Google to identify an article written by ChatGPT, Codex, or another AI tool as AI-generated content if the tools examined above can do so.

Additionally, Google employs quality raters who will manually assess articles as they are found to help their system better detect AI-written content.

As a result, it is recommended to use ChatGPT, Codex, and other AI tools as support in your content strategy rather than relying solely on them.

TLDR;

While AI-generated content can offer several benefits, such as saving time and generating ideas, it's essential to be mindful of its limitations and ethical considerations. Relying solely on AI-generated content might result in lower quality, less accurate, or even harmful information being disseminated.

To mitigate these risks, content creators can use AI tools to augment their writing process rather than replacing it completely. For example, AI-generated content can serve as a starting point, providing ideas and structure, but human intervention remains necessary to ensure accuracy, relevance, and context.

Moreover, the continued development and improvement of AI content detection tools will help in identifying and flagging AI-generated content. This can encourage transparency and discourage the use of AI-generated content for malicious purposes.

In conclusion, while AI content generation tools like ChatGPT and Codex are becoming more sophisticated, it's crucial to strike a balance between leveraging their advantages and maintaining the quality and credibility of the content produced.