Reconciling the measurable with the meaningful

In 1970, Italian political scientist Giovanni Sartori wrote a paper arguing that social scientists, in determining the scope of their research, face a trade-off between two desirable yet inversely related qualities: explanatory power and empirical validity. This trade-off can be illustrated by a theoretical framework called the ladder of abstraction.

Toward the top of the ladder are increasingly abstract and broad categories; toward the bottom, increasingly concrete and narrow ones. For example, in a study of political institutions, "regimes" might occupy the top rung of the ladder of abstraction, "democracies" the middle rung, and "parliamentary democracies" the bottom rung. For a study of desserts, it might go "desserts," then "pies," then "cherry pies."

A ladder of abstraction: The level of abstraction increases as each rung on the ladder is ascended (image source)

Operating at the top of the ladder of abstraction means dealing with highly general categories and talking in highly general terms, while operating at the bottom of the ladder means dealing with highly specific categories and talking in highly specific terms. Sartori argues that it is problematic to operate at either of the ladder's extremes. "The more we climb toward high-flown universals, the more tenuous the link with the empirical evidence," he writes. Conversely, the lower we descend down the ladder, the less "explanatory power" we have, and the more we begin merely to describe observations rather than synthesize them into useful insights.

Because meaningful questions are associated with the top of the ladder of abstraction while accurate answers are associated with the bottom, Sartori urges social scientists to attempt to reconcile the two by conducting their research at a medium level of abstraction, thereby "bring[ing] together assimilation and differentiation, a relatively high explanatory power and a relatively precise descriptive content, macro-theory and empirical testing.” Through this golden mean sort of approach, Sartori writes, researchers will be able to avoid producing either grandiose theories lacking in evidence or mundane descriptions of reality lacking in insight.

Sartori argues that a nuanced understanding of the proper and interdependent roles of both theory and methodology in the process of discerning the truth is the hallmark of a "conscious thinker." On the contrary, "unconscious thinkers" lack an appreciation for logic, while "overconscious thinkers" get hung up by pedantic rules and overly ambitious standards of precision. Sartori encourages "the conscious thinker to steer a middle course between crude logical mishandling on the one hand, and logical perfectionism (and paralysis) on the other."

Sartori makes a lucid and compelling case for why we shouldn't fly too close to the sun, glibly spouting off grand theories about how everything works, but also for why we shouldn't spend all of our time down in the dirt, rigorously investigating metaphorical grains of soil and coming up with incredibly precise findings about things that just don't matter all that much. His call to reconcile explanatory power with empirical validity—or, in other words, to reconcile what's meaningful with what's measurable—struck a personal chord with me.

In high school, my history teacher introduced our class to the idea of the Hegelian dialectic, in which an idea, the thesis, generates an oppositional reaction, the antithesis, which are then both reconciled as a synthesis.

The Hegelian dialectic: The thesis and the antithesis combine to form the synthesis (image source)

Back in high school, when I was learning about the Hegelian dialectic, I thought I had everything figured out. My view of the world was black-and-white and unambiguous. But then, I went to college, met people from all sorts of different backgrounds, took courses in subjects I never even knew existed, and quickly realized that my beliefs were relatively unexamined and that the world was a lot more complex and ambiguous than I'd previously thought. I had a mild crisis of confidence: intellectually, I went from a youthful blend of conviction and naiveté to a new, unsettling place somewhere in between uncertainty and nihilism.

In Sartori's terms, I'd gone from the glibness of the top of the ladder of abstraction to the skepticism and reluctance at the bottom, from an unconscious thinker to an overconscious one. In terms of the Hegelian dialectic, the thesis of my overconfident worldview had met the antithesis of my new doubtful and insecure one.

This past spring, I wrote a senior essay for my major, political science, about impact evaluations in development aid. Impact evaluations seek to identify causal relationships between inputs and outcomes (i.e., how much did that $40 million in foreign aid reduce cases of malaria?), so before getting into the meat of my essay, I devoted some pages to the question of how to draw causal inferences in the social sciences:

In “Causation and Explanation in Social Science,” Henry E. Brady discusses several approaches to the question of causality. The first, neo-Humean, posits causality when there is evidence of “constant conjunction” and “temporal precedence” between two variables. The neo-Humean approach may be useful in helping to identify interesting relationships between variables, but makes the mistake of equating correlation with causation, failing to establish whether and to what degree one variable may affect the other. The second approach Brady discusses, counterfactuals, determines the existence of a causal relationship between two variables if there is “truth in otherwise similar worlds of ‘if the cause occurs then so does the effect’ and ‘if the cause does not occur then the effect does not occur.’” It improves upon the neo-Humean approach by distinguishing between correlation and causation, but its obvious shortcoming is that it relies upon the existence of multiple “worlds” and is thus purely hypothetical, its utility for thought experiments rather than real-world investigations. The third approach Brady discusses is manipulation, which synthesizes the real-world applicability of the neo-Humean approach with the logical validity of the counterfactual approach. In manipulation, one simulates identical worlds, altering only one variable, so that any change in outcomes between the two worlds can be inferred to be attributable to the treatment variable. Of the three models of causality Brady discusses, manipulation is the only valid mechanism by which to make causal inferences about real-world phenomena.

I think that the three models of causal inference I mentioned—neo-Humean, counterfactuals, and manipulation—characterize the three phases of intellectual development that I, and I'd guess many others as well, went through over the course of high school and college.

The first, neo-Humean, which states that correlation implies causation, is unambiguous yet ignorant, like my worldview in high school.

The second approach, counterfactuals, with its reliance on multiple hypothetical worlds, is logically impeccable yet useless in a practical sense, like the intellectual frustration of my early college years when I sought an impossibly high degree of rigor and certainty before I would be willing to accept an idea.

The third, manipulation, is a sensible integration of the previous two models, combining, as I wrote, "the real-world applicability of the neo-Humean approach with the logical validity of the counterfactual approach." As Sartori advises, manipulation reconciles the explanatory power of counterfactuals with the empirical validity of the neo-Humean approach. As the Hegelian dialectic predicts, it combines the thesis of the neo-Humean approach with the antithesis of the counterfactual approach.

The manipulation model, with its synthesized origins, is a good metaphor for where I think I am now: aware, as Socrates said, that "the only true wisdom is in knowing you know nothing," yet simultaneously cognizant of the fact that, while allowing for a little asterisk at the end of everything that says even objective reality is unproven assumption, we humans can still leverage our ingenuity and technology to develop a progressively fuller and more nuanced understanding of what we believe to be the world and the universe around us.

In the Hegelian dialectic, the synthesis gradually settles into place as the new thesis, where it then generates an antithetical reaction, initiating the process of generating a new synthesis all over again. So, this post is not necessarily to say that I think I've arrived at a final and static destination with regard to my mode of thinking, but rather, simply, to explain how I think I've gotten to where I am now.