SSP Forum: Avery Louis, Junyi Tao, and Chelsea Zou (M.S. Candidates)
The
Symbolic Systems Forum
(community sessions of SYMSYS 280 - Symbolic Systems Research Seminar)
presents
On the Shape of Progress in Science: Evolutionary (Dis)analogies, Coherence & Paradigm Shifts
Avery Louis (M.S. Candidate)
Symbolic Systems Program
and
Explaining Model Generalization Through Internal Causal Mechanisms
Junyi Tao (M.S. Candidate)
Symbolic Systems Program
and
CalBench: Evaluating Coordination–Privacy Trade-offs in Multi-Agent LLMs
Chelsea Zou (M.S. Candidate)
Symbolic Systems Program
Monday, May 11, 2026
12;30-1:45 pm PT (note the later than usual ending time)
Computing and Data Science Building (CoDa), Room E160
In-person event, not recorded
Note: Lunch is provided, if pre-ordered, only for members of SYMSYS 280, but others are welcome to bring a lunch and eat during the presentation.
Abstracts:
Avery Louis, "On the Shape of Progress in Science: Evolutionary (Dis)analogies, Coherence & Paradigm Shifts" (primary advisor: Maneesh Agrawala, Computer Science; second reader: Marina Dubova, Santa Fe Institute)
How does science make progress? It's tempting to answer this with the evolutionary analogy: science can be thought of as a population of ideas under selection, where good theories outcompete bad ones and progress is the cumulative work of variation and retention. This picture is rather seductive! It aligns something quite nebulous (the idea of progress in science) with something fairly well-understood, and it sells the cumulative work of science as a process matching the genius of natural evolution. But the analogy breaks in revealing ways. Selection in biology operates on degenerate genotype-phenotype mappings, where many configurations realise similar functions — not so for science. In this talk I’ll outline some revealing differences between scientific progress and evolution using a genetic algorithm-inspired model of theory-building. In particular, I’ll focus on the ways in which scientists engage in epistemic niche construction by favouring new theories that cohere with old paradigms. There are some interesting upshots of that behaviour, and much to learn from the disparities between natural selection and scientific selection.
Junyi Tao, Explaining Model Generalization Through Internal Causal Mechanisms (primary advisor: Thomas Icard, Philosophy; second reader: Christopher Potts, Linguistics)
(Joint work with Jing Huang) When a model exhibits a pattern of behavior, what is the best way to understand it? One answer is purely behavioral: we can simply list all the input-output mappings. A perhaps more helpful explanation describes what abstract, algorithmic solutions the model implements internally to produce those behaviors. This has been the focus of causal interpretability research. Our work further extends this approach by exploring a novel direction: can such internal mechanism-based explanations give us some understanding (and even predictive power) of behavioral patterns that go beyond what we have already observed?
We find positive evidence that they can. Our results show that we can robustly predict the correctness of the model’s generalization behavior precisely by identifying the algorithmic solutions it implements. This approach even significantly outperforms existing correctness-prediction baselines (confidence scores and probing) under distribution shifts across a diverse set of five tasks.
At the end, I would like to discuss some preliminary results on compositional generalization and future research directions. While the work above concerns a model’s inference-time behavior, the question I am currently most excited about concerns learning. The abstract algorithmic mechanisms we use to explain behavior have to come into existence in the first place. How should we explain their development, and how do they contribute to loss minimization? One possibility is that these algorithmic solutions, as a form of compression, are exactly what support a model’s ability to generalize.
Chelsea Zou, "CalBench: Evaluating Coordination–Privacy Trade-offs in Multi-Agent LLMs" (primary advisor: Noah Goodman, Psychology and Computer Science)
(Joint work with Yiheng Yao, Selena She, and Robert Hawkins) We introduce CalBench, a controlled evaluation environment for evaluating multi-agent coordination through calendar scheduling. In CalBench, N agents each manage a private calendar containing pre-existing commitments and must coordinate to schedule a stream of M incoming meetings while minimizing disruption costs. Because agents observe only their own calendars, successful scheduling requires communication across private information boundaries. Furthermore, commitments are costly to move, so success requires more than finding any feasible slot. Each scenario is generated with an oracle solution, enabling precise measurement of coordination quality via realized-to-optimal cost, as well as a Distributed Constraint Optimization (DCOP) baseline to provide a fair comparison under the same private-information constraints. The CalBench benchmark enables precise verification of task success, communication efficiency, and fairness (distribution of disruption costs) in a multi-agent setup. Furthermore, our environment also presents affordances to study privacy-preserving coordination. CalBench augments calendar entries with private semantic contexts of varying sensitivity and measures whether agents reveal task-irrelevant private information during negotiation. Unlike multi-agent benchmarks where a single capable agent can often substitute for the group, CalBench is inherently decentralized: no agent has access to another agent’s private calendar, yet agents must still reach mutually consistent decisions over shared meeting scheduling. CalBench therefore provides a practical and verifiable setting for studying coordination protocols, communication efficiency, negotiation strategies, fairness, and privacy leakage in multi-agent systems.