Lesson 1 of 5, 11 min with the exercise
What the model is really doing
Understand why AI can be useful and wrong in the same sentence.
A language model predicts the next piece of text based on patterns from its training. It isn't looking your question up in a database unless the product has added search or your files.
That explains most of what you'll see. It writes well because it has seen a lot of writing. It can state a wrong date with total confidence because a date that fits the sentence is what it was built to produce.
What this means at work
- Good jobs: drafting, summarising, rewriting, sorting messy text, pulling fields out of a document you supplied.
- Risky jobs: exact facts, legal citations, recent events and sums, unless it's working from a source or a tool you can check.
- It can give different answers to the same question. That's normal, and it's why you test on more than one example.
See it in practice
In 2023, two New York lawyers filed a brief citing court cases that ChatGPT had invented, and a judge fined them $5,000 (Mata v. Avianca). ChatGPT had also told them the cases were real and could be found in legal databases. Asking the same tool to confirm its own answer isn't a check. Open the original source.
Common mistake: Asking the model "are you sure?" and taking the answer as proof. It may apologise and change a correct answer, or confidently repeat a wrong one.
Try it
Ask your AI tool about something you know very well, such as your own company or industry. Mark every sentence that's wrong or made up. That's your calibration for how far to trust it.
You're done when
- You can name two tasks you'd give it and two you wouldn't
- You've seen it get something wrong in your own field
Sources: Mata v. Avianca, summary of the case