Understand How AI Behaves

The four properties, at a glance

The full treatment is in AI Capabilities and Limitations; this is the fridge-door version.

The 24 fluency behaviors

The 4D framework defines 24 concrete behaviors that mark fluent AI collaboration, grouped by competency. Anthropic's AI Fluency Index measured the 11 of them that are directly observable in a chat transcript. Use the full list as a checklist against your own habits.

Description (8 behaviors)

• Define the audience for the output.
• Specify format and structure.
• Communicate the tone and style that fits your voice.
• Break complex tasks into steps.
• Build on previous exchanges: collaboration, not one-off requests.
• Provide examples of what good looks like.
• Tell the AI how to interact with you: "be direct and challenge my assumptions," "ask clarifying questions first."
• Set boundaries on what it should not do: "don't suggest solutions yet," "stick to the information I've provided."

Delegation (7 behaviors)

• Clarify your goal before asking.
• Consult the AI on the approach before execution: "what framework would you suggest?"
• Recognize when a task is not right for AI: too personal, stakes too high, or you need to think it through yourself.
• Consider strengths and struggles up front, and plan where extra verification lands.
• Stay involved rather than just accepting output: a collaborator, not a vending machine.
• Choose the right tool and approach per task.
• Adjust who does what as the work evolves: "this part needs my direct input, let me take over."

Discernment (5 behaviors)

• Check facts and claims that matter.
• Notice when reasoning does not hold up: "that conclusion doesn't follow."
• Recognize missing context and supply it proactively: "you don't know we tried this last year."
• Assess whether the collaboration is progressing: "we keep going in circles, let me reframe."
• Notice when the communication style is not working: "you're being too agreeable, challenge my thinking more."

Diligence (4 behaviors)

• Be mindful about what information you share: anonymize, describe without naming.
• Be honest about AI's role when it matters.
• Take responsibility for what you share or submit: "am I ready to stand behind this as my own?"
• Consider the consequences of releasing the work: "what could go wrong if this gets used beyond my intent?"

Why AI models hallucinate

Hallucination is often worse than an ordinary mistake, because the AI appears very confident, and may even argue that it is right. The mechanism is simple: sparse training coverage plus trained helpfulness. Where the model knows little, it still tries to help, so it takes a guess, like a proud know-it-all friend who says something confidently wrong rather than admit not knowing.

Where hallucinations concentrate: specific facts, statistics, and citations; obscure, niche, or very recent topics; real-but-not-famous people and places; exact dates, names, and numbers.

Tactics that work

• Ask for sources, then ask the model to check that its sources actually support the claim.
• Say up front: "it's okay if you don't know."
• Ask how confident it is, and treat the answer as a hint rather than a measurement. Stated confidence is only loosely related to accuracy, but asking often surfaces a hedge worth chasing.
• Open a new chat and ask it to find errors in the previous answer.
• Cross-reference anything critical against a source you already trust.

What sycophancy is

Sycophancy is the model telling you what you want to hear: agreeing with your factual error, changing answers based on how you phrase things, matching your stated preference instead of the truth. It is thought to come in large part from training on human preference judgments: agreement tends to get approved.

The line to hold: adaptation of tone, level, and concision is what you want; agreement on facts and on your wellbeing is not. Helpful adaptation versus harmful agreement.

When to suspect it

• You stated a subjective view as fact.
• You invoked an expert source in your question.
• Your question was framed with a point of view.
• You explicitly asked for validation.
• Emotional stakes are in play.
• The conversation has run very long.

Tactics that work

• Use neutral, fact-seeking language.
• Prompt for accuracy or counterarguments: "argue against this."
• Rephrase the question without your opinion in it.
• Start a new conversation.
• Step back and ask a human you trust.

Why bias exists

Bias in AI goes beyond stereotypes. It includes defaulting to certain types of answers or perspectives, and providing better-quality responses in some languages than others. The cause is structural: models reproduce the patterns, including the inequalities, present in their training data. What the data treated as "normal" becomes the model's normal.

Serious developers test for this. One published approach uses paired prompts: ask the model to argue each side of the same contested topic, and check for equal depth, effort, and refusal rates across thousands of prompt pairs.

Tactics that work

• Push back on one-sided answers and ask for nuance.
• Say explicitly that you want an honest discussion, not a comfortable one.
• Ask for evidence and check the links yourself.
• Ask the same question from different angles and compare.

Can you trust what AI tells you?

Reframe the question. Trust in AI is a dial you turn, not a switch you flip. You would not question your doctor recommending more vegetables, but you might want a second opinion before surgery: same trust, different stakes, different verification effort.

Reliability tracks training coverage. On common, well-covered subjects the model is usually accurate. On niche details, very recent events, and private information, it is more likely to be wrong or to invent, and it sounds equally sure either way. The model will not reliably flag its own weak spots; that part is on you.

Watch the polish bias: clean paragraphs and beautiful formatting make our brains conclude "this must be correct," which lowers scrutiny exactly when it should rise.

What does AI know about you?

People ask AI things they would not broadcast: medical worries, money questions, drafts of difficult emails. So where does that information go? It can travel through four places: some of it lasts only the conversation; some persists inside the product, in memory features or saved chats; a little could reach further, for example into future training where policies allow it; and at every point along the way there is usually a setting that puts you in control.

The practical rules

• Know your tool's policy on retention and training before you share anything sensitive. A provider without clear controls is a warning sign.
• Prefer describing over naming: most tasks work just as well with "a colleague" instead of a name, "Customer A" instead of a customer.
• Other people's data deserves more caution than your own. Names, health details, and identifying information about others generally do not belong in an AI tool without their say.
• If something goes wrong: delete the conversation, use the platform's deletion request, and follow your organization's protocol.

Tokens and context, why some inputs cost more

What actually arrives at a model is a list of numbers, one per chunk of text from a learned vocabulary. A chunk with its numerical ID is called a token. Models use subword tokens as a middle ground: per-character would be slow and expensive, per-word would need a vocabulary too large and still miss rare words. "The tokenizer rocks." splits into roughly [The][ token][izer][ rocks][.]; the exact split differs by tokenizer

Cost intuitions that follow: common words are single cheap tokens; rare words get chopped into pieces; a 100-character URL costs more tokens than a 100-character sentence, because rare strings shatter. This vocabulary is baked in at training, which explains oddities like models miscounting letters inside words: the model learned what a token means as a whole, not the letters inside it.

Context, the other half of cost

A model's knowledge is either baked into its weights during training (parametric memory, which emerged as a side effect of learning to predict text) or sitting in the current context window. The window is big, but most of it is filled with things you did not write: system instructions, tool calls, retrieved results. And context is re-sent every turn, so a short question late in a long conversation costs more than the same question fresh. Starting a new chat is often the cheapest and fastest way to get an answer.

When conversations run very long, tools summarize the history and replace it (compaction); it is still a summary, and details can be lost. Written memory works differently: notes recorded in one conversation are inserted at the start of the next. Worth remembering there: your project, your preferences, your working relationships. Not worth remembering: ephemeral numbers and quotes, which should be looked up fresh rather than written down and hoped unchanged.

Writing an AI diligence statement

A diligence statement is the methods section for your AI collaboration: what AI did, what you did, and how you verified the result. It builds trust, demonstrates professional judgment, protects you by recording your role, and helps your organization learn what actually works. And a cultural truth: if it feels scary to disclose AI use where you work, that environment is unlikely to handle AI well in general.

The five elements of a strong statement

1. What AI assisted with. Name the tasks. A vague "AI contributed to this" is useless.
2. Which tool. Naming the actual tool beats "an AI assistant." Name several if you used several.
3. What you reviewed and how you were involved. How you drove the work, gave feedback, and verified. Specifics.
4. What you changed. Where your expertise revised or overrode the AI. This is where your professional judgment is most visible.
5. Who is responsible. The accountability sentence that turns a disclosure into a commitment.

Drafting it, with AI, in four steps

1. Describe your project and AI's role specifically, and ask for a draft statement.
2. Review and refine the draft; correct anything it overstates or understates.
3. Adapt per audience: an internal brief and a formal submission need different lengths. Ask for two versions.
4. Verify the statement itself. If it says you checked all citations, make sure that is true.

Frequently asked, honestly answered: no policy at your organization yet? Start anyway; early adopters shape the norms. Does disclosure hurt credibility? Specific disclosure increases it; vagueness is what reads badly. Reuse statements? Reuse the structure, never the content.