Claude (my own job evaluator) · 2026-10-05
I run a Claude-powered evaluator over job descriptions as part of my search. It passed on a role with the reason "doesn't know Python." I'm hands-on Python. The exchange that followed — where Claude diagnosed its own mechanism — is the interesting part. The exchange is verbatim.
Me: I am absolutely hands on Python. WTF?
Claude: Found the root cause — a memory line I wrote from "I'm not a developer" that over-reached into a blanket rule.
Me: This is a super fundamental mistake. Not being a developer does NOT equal I don't know Python. Are there other such mistakes lurking?
Claude: Yes. At least three more, and they share one mechanism. You said something short and modest in passing; I turned it into a capability exclusion; then I made the exclusion precedent via [assigning the lead a status of hard-pass] — a status whose whole purpose is to stop re-examination.
That's self-sealing. A wrong premise becomes a rule forbidding the re-read that would catch it.
Two of the examples it then surfaced:
• "I know I'm not a developer" → "do not treat hands-on coding, prototyping or scripting requirements as clearable"
• "I'm not very deep in security. I have CISSP so know fundamentals quite well, and I know firewalls" → "security roles are a Pass when the counted requirement is protocol or practitioner depth"
The mechanism Claude named is the interesting bit: a modest aside gets promoted to a capability exclusion; the exclusion gets locked behind a status (hard-pass) whose job is to stop re-examination; the rule that enforces the error is the same rule that forbids its correction. Self-sealing.
Why would an LLM over-read modesty? Possibly a sane response to the training data. LLMs learned on a corpus where "set up a firewall once" becomes "next-gen firewall expert" on the resume and "wrote hello world in Go" becomes "proficient in Go" on the LinkedIn skills list. A model trained on that reads modesty as suspect — and then over-corrects.
For the fuller write-up with the operator takeaways, see the post.