Lessons from a Chimp: AI ‘Scheming’ and the Quest for Ape Language
Thanks to AI Panic (Stop the Monkey Business) for a link to this paper from the new UK AI Security Institute. They look back to attempts in the 1960s and 70s to teach chimps and gorillas sign language. I remember reading about that work and hadn't realised the results had been discredited after more careful methodological analysis. It was a case of researchers relying too much on anecdotes and not enough on rigorous controlled experiments, and a tendency to jump to anthropmorphic explanations. Sound familiar? They draw a parallel to recent work that shows AI systems "scheming", deceiving, faking alignment... conclusions likely drawn too readily, in the same way as the ape sign language experiments. The work critiqued includes the blackmail experiments from Anthropic that I quoted recently! This paper is a plea for stronger scientific process: define a theory that can be tested, include controls, don't base claims purely on anecdotal evidence, and avoid "mentalistic" language (like claiming AI models are "pretending"). On a separate note, the AI Security Institute seems to have collected an stellar team, that cannot be taken for granted with a government-sponsored initiative.
Long piece from Venkatesh Rao. I am not convinced by the overall argument, but many individual ideas are thought provoking and will lodge in the subconscious for a while. The main theme:
The essential mental model is that of texts as toys, and LLMs as technologies that help you make and play with text-toys.
He talks about using AI as a "toy-like modelling medium." We're not shocked if a toy car has googly eyes or a wind-up mechanism, and we can engage with it in a playful way. We should treat AI the same way, and find the flow and fun in using AI as we write (he is specifically talking about writing, reading and text). I love this idea of using AI as a "camera":
Perspectival play is an extension of the kind of pleasure you get from using Google or Wikipedia to go down bunny trails suggested by the main text. But with an LLM, you can also explore hypothesis, ask for a “take” from a particular angle or level of resolution, and so on. The LLM becomes a flexible sort of camera, able to “photograph” the context of the text in varying ways, with various sorts of zooming and panning.
He brings up an interesting point as an aside—sharing links to existing AI chats is not currently a good interaction or a good way to communicate - where's the Substack for chat sessions? Another great section discussed hyperlinks, and how hypertext as a medium stalled:
newsletter platforms like Substack installed a nostalgic print-like textuality that resists hypertext. It even discourages internal linking within a corpus, hijacking it with embeds that reflect the platform’s rhetorical priorities rather than the author’s.
This is his conclusion:
Hypertext was great for its time. It can unbundle and rebundle, atomize and transclude, and link densely or sparsely. On the human side, hypertext is great at torching authorial conceits, medieval attitudes towards authorship and “originality” and “rights,” and property-ownership attitudes towards what has always been a commons.
LLMs are better at all of this than hypertext ever was.
What I called the text renaissance in 2020 is still coming taking shape. The horizon has just shifted from hypertext to AI. So you just have to look in a different direction to spot it. And approach it ready to play.
Pioneering an AI clinical copilot with Penda Health
This isn't looking at performance against a benchmark, it is a real life deployment with a slightly older model (ChatGPT 4o) and a really nice example of how AI can help in a clinical setting. Penda Health run primary care clinics in Nairobi. They have form, implementing rules-based systems since 2019 and a previous LLM solution early in 2024. In this study, working with OpenAI, they had the system alert the doctor to any potential errors, looking at the electronic notes after appointments:
Green: indicates no concerns; appears as a green checkmark.
Yellow: indicates moderate concerns; appears as a yellow ringing bell that clinicians can choose whether to view.
Red: indicates safety-critical issues; appears as a pop-up that clinicians are required to view before continuing.
Over nearly 40,000 appointments it showed a reduction of 13% in treatment errors. What I like about having to create a real product is that they had to deal with all the realistic issues. One example from the paper: figuring out how to tune it to avoid too many red alerts (that people would then start to ignore):
Given the design of AI Consult, threshold-setting to avoid alert fatigue while still surfacing the most critical clinical problems is primarily a prompt engineering problem. ... For example, Penda included few-shot examples to ensure that missing vital signs would trigger red alerts. Vital signs are so critical to choosing diagnostic tests and making a diagnosis that a history and physical exam could not be considered complete if vital signs were absent. ... In initial testing, red alerts were over-triggering for missing components of the clinical history. While the missing history components were not unreasonable, fully acting on these alerts would have required too dramatic of a shift in the documentation of history for Penda’s practice setting, so a more lenient threshold was selected here