Recent contributions have used combinatorial algorithms to determine the
likelihood of particular social choice violations in rank sum scoring. Given the broad
importance of rank sum scoring (e.g., in non-parametric statistical testing, sporting competition,
and mathematical competition), it is important to establish the level of ambiguity
generated by this aggregation rule. Combinatorial likelihoods are naı¨ve, however, in that
they assume each possible outcome sequence for an event to be equally likely. We develop
a computational algorithm to extend upon previous combinatorial results as to the likelihood
of a violation of transitivity or independence in rank sum scoring. We use a similar
computational scoring approach to analyze the empirically-observed likelihood of each
such violation across fourteen NCAA Cross Country Championships. Within the data, rank
sum scoring fails to specify a robust winning team (i.e., one that also rank sum wins against
each possible subset of opponents) in 4 of 14 cases. Overall, we find that empirical
likelihoods of social choice violations are consistently (significantly) overestimated by
combinatorial expectations. In the NCAA data, we find correlated ability (quality) levels
within team (group) and discuss this as a cause of lower empirical likelihoods. Combinatorial
analysis proves reliable in predicting the order of empirical likelihoods across
violation type and event setting.

• The likelihood of social choice violations in rank sum scoring: algorithms and evidence from NCAA cross country running