OfflineEvaluations
A system’s isolated score answers the smallest question. The harder question is what changes when machine behavior returns to offline life.
How do artificial systems behave once they enter human contexts?
Core question
How does deployment change machine behavior?
Offline Evaluations study how artificial systems behave once they enter human life.
How do artificial systems behave inside human institutions?
Which behaviors emerge only after people adapt around a system?
How should evaluations represent real social context without flattening it?
When does a machine system begin changing the environment it is measured inside?
Offline Evaluations study how artificial systems behave after deployment becomes social.
Human context
How system behavior changes inside human settings.
Offline effects
How machine outputs return through everyday practice.
Evaluation design
How to study artificial systems without stripping away the world they act within.
Represent context
Design evaluations that preserve the human setting around the system.
Observe adaptation
Study how human behavior changes around the system being evaluated.
Measure return paths
Track how system outputs come back into offline decisions and institutions.