Beyond Natural-Language Interfaces: Decisions, Scores, and Predictive Distributions

Abstract

Language models increasingly supply the predictions on which other systems make decisions. For these systems, an interface defines the predictive information passed to a consumer, yet output format alone does not establish its statistical meaning or computational cost. This survey connects predictive interfaces across classification, reward and quality scoring, numerical regression, and probabilistic forecasting. We organize the literature around the target being reported, the transformations applied to a report, and the computation required to produce it. Drawing on statistical foundations and comparisons between concrete method configurations, we explain why accurate ranking need not provide calibrated probabilities, structural validity need not improve task performance, and compact outputs can conceal extensive generation. We also distinguish uncertainty about observed outcomes from uncertainty in a teacher’s generation process. These distinctions connect interface design to the information a downstream decision actually requires: an appropriate statistic may suffice for a fixed objective, while changing objectives can require richer retained reports or renewed validation. The resulting taxonomy and selection guide link predictive uses to feasible production mechanisms and the evidence needed to compare them. We propose evaluation protocols for changes in losses, constraints, and teacher processes. The unifying principle is to design interfaces around the decisions their reports must support, and to make explicit what must be revalidated when the intended use changes.