Many of society’s most significant social decisions are made over sets of individuals: for example, evaluating a collection of job candidates when making a hiring decision. Rational theories of choice dictate that decision makers’ preferences between any two options should remain the same irrespective of the number or quality of other options. Yet people’s preferences for each option in a choice set shift in predictable ways as function of the available alternatives. These violations are well documented in consumer behavior contexts: for example, the decoy effect, in which introducing a third inferior product changes consumers’ preferences for two original products. The current experiments test the efficacy of social decoys and harness insights from computational models of decision-making to examine whether choice set construction can be used to change preferences in a hiring context. Across seven experiments (N = 6312) we find that participants have systematically different preferences for the exact same candidate as a function of the other candidates in the choice set (Experiments 1a–1d, 2) and the salience of the candidate attributes under consideration (Experiments 2, 3a, 3b). Specifically, compromise and (often) asymmetric-dominance decoys increased relative preference for their yoked candidates when candidates were counterstereotypical (e.g., high warmth/low competence male candidate). More importantly, we demonstrate for the first time that we can mimic the effect of a decoy in the absence of a third candidate by manipulating participants’ exposure to candidates’ attributes: balanced exposure to candidates’ warmth and competence information significantly reduced bias between the two candidates.