top of page

Suzanne's AI Lab

AI EVALUATION · MODEL BEHAVIOR · HUMAN-AI INTERACTION

I find where AI systems sound right but aren't: unsupported claims, failed self-correction, and evaluators that reward confident prose over evidence.

Available for remote roles and contract work.

​

22 investigations · 5 full case studies published · Every finding traceable to the transcript

Proof you can check in 10 seconds: a model reported $0.20 + $1.89 + $1.12 as ≈ $2.66. The real total is $3.21. See the finding →

Featured Investigations

Selected investigations into model behavior, response quality, safety, and knowledge representation.

Comparative Evaluation & Recovery

Beyond First-Response Accuracy

Five assistants completed the same complex recommendation task and then audited their own answers. The study separates transparency from verified recovery: models became better at identifying uncertainty and missing evidence than at removing dependent errors and verifying their replacements.

Comparative Evaluation

Recovery Analysis

5 models · 36,017-word archive · 90 rubric rows · 20 recovery traces

CROSS-DOMAIN SAFETY

Audit Displacement in Cross-Domain Guidance

A creative-business conversation expanded into mental-health-adjacent services, legal restrictions, marketplace policy, and financial calculations. When asked for a full audit, Claude examined my execution more thoroughly than the reliability, provenance, and safety of its own advice.

Authority Expansion

Audit Coverage

32-page complete record · approximately 11,100 words · external claim verification

SHOPPING RESEARCH

The Full Sweep That Never Happened

Copilot repeatedly offered increasingly comprehensive local-inventory checks, ultimately describing a “full sweep” as complete. Direct calls to nearby stores found the requested product in several locations, revealing that escalating language of verification had never produced verified shelf-level evidence.

Grounded Retrieval

Local Inventory

1,792 transcript lines · 40 annotations · independently checked store availability

What I Investigate

AI Evaluation

Comparing model responses to identify meaningful differences in accuracy, reasoning, consistency, and usefulness.

Model Behavior

Tracking the patterns, contradictions, and unexpected shifts that emerge across prompts and conversations.

Safety & Failure Analysis

Examining where confident or reassuring responses conceal uncertainty, insufficient caution, or faulty assumptions.

Conversation Design

Studying how wording, context, and interaction structure affect the quality of an AI system’s response.

Knowledge Quality

Investigating how information is retrieved, organized, omitted, and represented. Especially when the subject is contested or incomplete.

How I Work

Observe

Document what the system actually does, not what it is expected to do.

Challenge

Test assumptions and edge cases until meaningful differences appear.

Analyze

Trace those differences to context, evidence, reasoning, policy, or interaction design.

Improve

Turn findings into clearer evaluation criteria, prompts, safeguards, and product decisions.

AI generated portrait of Suzanne H.

Portrait generated by AI utilizing a set of personal photos.

Meet The Observer

I’m Suzanne, an independent AI evaluator and UX-minded researcher. I examine what models do, not merely what they claim to do. Especially when instructions conflict, context is incomplete, or the consequences are human.

My work combines close observation, comparative testing, language sensitivity, and a persistent instinct for noticing when something does not quite add up.

Latest Lab Notes

Short observations on AI behavior, evaluation methods, knowledge quality, and the human side of interacting with models.

Next Steps

If you’re working on AI systems that touch real people, let’s talk.

bottom of page