In 2023, two New York lawyers got fined $5,000 for submitting a legal brief full of court cases that did not exist. ChatGPT had invented them - complete with case names, judges, and citation numbers that looked exactly like real legal references. The lawyers did not check. Nobody caught it until opposing counsel went looking for the cases and found nothing.
That was not a one-off. A database tracking these incidents has now logged over 1,900 court cases worldwide where AI-fabricated citations showed up in legal filings. Judges have sanctioned lawyers repeatedly, including after they were explicitly warned. A Stanford study found general chatbots hallucinated on legal questions between 58% and 82% of the time.
If a room full of lawyers with law degrees and a professional obligation to double-check everything can get caught out by this, it is worth asking what the same failure looks like when the question is not "does this court case exist" but "what does AS/NZS 3000 actually require."
The short answer
General AI chatbots like ChatGPT generate answers from patterns learned during training, not from the actual standard open in front of them - which is exactly why they can confidently invent a clause number that sounds completely real. This is a well-documented problem, extensively so in the legal profession. Smart Assistant works differently: it searches a library of real Australian Standards and manufacturer documentation, and shows you where the answer came from, so you're not taking a plausible-sounding guess on faith.
1. Why this happens, and why it isn't a glitch
A general AI chatbot learned to write by processing enormous amounts of text and learning to predict what a good answer looks like. It has absorbed a huge amount of real information along the way - which is exactly the problem, because it also learned what a citation is supposed to look like: a document name, a colon, a clause number, maybe a subsection. It can produce something in that precise shape without the number actually corresponding to anything real, because generating correctly-formatted text was never the same task as checking a document.
Reuters reported that US courts questioned or disciplined lawyers over fabricated AI citations in at least seven separate cases within two years - and that was before the practice became widespread enough to need a dedicated tracking database. One judge put it plainly: using AI-generated citations without checking them is "incompetence, pure and simple." The same standard applies whether the citation is a court case or a clause of the Wiring Rules.
2. What a real AS/NZS 3000 clause actually looks like
For context on what genuine specificity looks like: RCD protection under AS/NZS 3000:2018 is not one blanket rule. Socket outlet circuits rated up to 20A are covered under Clause 2.6.3.2. Lighting circuits in domestic installations are covered separately under 2.6.3.3 - and this was a significant change from the 2007 edition, which did not require RCD protection on lighting circuits at all. Bathrooms and other damp-area circuits are covered under Section 6. Altering an existing circuit versus replacing a whole switchboard triggers different obligations again, spelled out specifically in Clause 2.6.3.2.5.
That is a genuinely intricate structure, and it is exactly the kind of thing a general chatbot will happily flatten into one confident, wrong-in-the-details sentence - because producing a clean, simple-sounding answer is what it is optimised to do, not preserving the actual complexity of the source document.
3. What Smart Assistant does differently
Smart Assistant is built around a specific library, not the open internet. That library includes:
- Australian Standards relevant to trade work - AS/NZS 3000 for electrical installations, AS/NZS 3500 for plumbing and drainage, and other standards tradies reference regularly, depending on your trade.
- Manufacturer installation manuals and technical documentation - the actual specs for the products you're installing, not a generic summary of how that category of product usually behaves.
When you ask a question, Smart Assistant searches that library for the relevant section and builds its answer from what is actually there - not from a general impression of what standards typically say. And because it is reading from a real document rather than generating from memory, it can point back to where the answer came from, so you can check it yourself if the job warrants the extra step.
This is the same principle behind why the SWMS Generator and Invoice Generator in Smart Tools are built around structured, checkable output - the goal is speeding up real work, not producing something that merely reads as finished.
4. A test you can run on any AI tool
Next time you use an AI tool for a standards or compliance question, ask it directly: "which specific clause is that from, and can you show me the source?"
A tool actually searching real documents will point to something specific - a standard number, a clause, a section you could go check. A tool generating from memory will either restate the same claim more confidently, dodge the question entirely, or produce a second citation that still cannot be verified against anything real. It is the exact same test that would have caught the fabricated cases in Mata v. Avianca before they ever reached a judge.
5. Where general chatbots are still genuinely fine to use
None of this makes ChatGPT and similar tools useless for a tradie. They are genuinely good at drafting a message, explaining a broad concept in plain language, or thinking through a problem out loud where being slightly imprecise costs nothing.
The line is specificity and stakes. "Explain roughly how an RCD works" is a perfectly fine question for a general chatbot. "What's the exact clause I need to reference for this specific altered circuit" is the kind of question where a confidently wrong answer can end up in a document with your name on it. The more specific and the higher the stakes, the more it matters whether the tool answering you is actually checking a real source.
Why this is worth getting right
A wrong social caption costs nothing. A wrong clause cited in a SWMS, or repeated confidently to a client who repeats it to their builder, is a different category of mistake - the same category that has now landed lawyers in front of federal judges nearly two thousand times and counting. Smart Assistant exists specifically for that second category: questions where the answer needs to come from an actual standard or manual, not a plausible guess.
For the wider picture of where AI genuinely helps across a trade business and where it doesn't, see AI tools for tradies: what actually works in 2026. And for how this connects directly to safety documentation, see how to write a SWMS.
Frequently asked questions
What standards and documents does Smart Assistant actually search?
A library of Australian Standards relevant to trade work - AS/NZS 3000 for electrical, AS/NZS 3500 for plumbing and drainage, and other standards tradies use regularly - alongside manufacturer installation manuals and technical documentation. It searches that actual library when you ask a question, built specifically around what a working tradie needs to check, not the entire internet.
Has AI actually gotten citations wrong in real situations, not just standards?
Yes, extensively and publicly. A database has documented over 1,900 court cases worldwide involving AI-fabricated citations, and lawyers have been fined and sanctioned repeatedly - even after being warned. A Stanford study found general chatbots hallucinated on legal queries 58 to 82% of the time. The same underlying failure applies to technical standards, just with less public visibility because nobody's built a tracking database for it the way courts have.
Why does ChatGPT sometimes invent a clause number that doesn't exist?
General AI chatbots generate answers by predicting plausible-sounding text based on training patterns - they don't have the standard open while answering. Asked for a specific clause, they can produce something in the exact shape of a real citation, with a number that looks entirely legitimate, without it existing anywhere in the document. It's not a rare glitch. It's the tool doing exactly what it was built to do.
Is it worth switching from ChatGPT to a trade-specific AI tool for standards questions?
For general questions, drafting, or brainstorming, a general chatbot works fine. For anything tied to a specific standard, requirement, or manufacturer spec - the kind of thing you'd need to defend if asked to justify it - a tool built to search real trade documentation is a meaningfully different level of reliability. They're not really solving the same problem.