Individuals and organizations often rely on numerical metrics to make decisions and form judgments. Numbers can be difficult to process, leading to under-utilization, but are also uniquely suited to making comparisons. Do we decide differently when some dimensions of a choice are quantified and others are not? We explore this question across six pre-registered experiments (n=7,000) involving managerial, policy, and consumer decisions. Participants face choices that involve trade-offs (e.g., choosing between employees, one of whom has higher potential but less commitment); we randomize which dimension of each tradeoff is presented numerically and which is presented qualitatively (using verbal estimates, discrete visualizations, or continuous visualizations). People systematically shift their preferences towards options that dominate on trade-off dimensions conveyed numerically. We identify one mechanism that underlies this quantification fixation: greater fluency of quantified information. Our findings suggest that when we count, we change what counts.