Puzzles, Progress and the Scientific Method

In Noah’s reply to my earlier post, he interprets me as saying that purely empirical studies are “worse” than theoretical contributions even if the theories are rejected.  I hope I didn’t give the reader this impression because that certainly wasn’t the point I wanted to make.  If I did, please let me clarify. 

The scientific method goes something like this:

1. Observation
2. Formation of hypotheses
3. Testing/evaluation
4. Repeat

If you can follow these steps then anything (even economics! even macroeconomics!) can be studied scientifically.  When economics is at its best it truly is a science. 

A purely empirical study is a necessary step in the scientific method (it’s step 1).  Indeed, purely empirical studies, by which I mean simple observational studies, are somewhat undervalued in economics.  However, following the scientific method means that after the observation, we have to move on to forming a hypothesis (building a model or otherwise articulating a theory) and then go on to testing and evaluating the hypothesis (this is the hard part, and the part that is often quite unpleasant for many theories).  Step 1 is not “worse” than step 3.  Both are necessary.  I also didn’t say you should try to hypothesize without looking first at the data and I don’t think macroeconomists are typically modelling without a good grasp of the facts. 

Noah makes some other remarks which I also find interesting. 

First, he makes some surprising remarks about “moment matching.”  It’s true that macroeconomists often evaluate their theories by comparing the statistical moments implied by the model with the analogous moments in the data but it’s hard to see this as a problem.  Moment matching is the underpinning of almost all statistics.  Estimating a mean and a variance is a special case of matching moments.  So is OLS estimation (so is IV and basically every other econometric technique other than maximum likelihood estimation).  I’m a little puzzled as to what Noah would suggest that economists should be doing?  In fact, drop the word “moments” and ask yourself this: should we compare our theories with the data?  I would say we should.  Do we want the theories to match up with the data?  I would say we do. 

Second, he questions whether quantification is valuable when often the theory is rejected.  The way I look at it, the parameters are inputs into the models and will likely have relevance in many settings.  Suppose you have a labor supply / labor demand model and you want to use it to analyze an increase in the minimum wage.  The model makes a prediction and you could use this prediction as a basis for anticipating the effect of the policy.  The predicted change in unemployment will be a function of the labor supply elasticity and the labor demand elasticity.  You can certainly estimate these parameters without testing the model.  Suppose you find that there is no change in employment in the short run and a somewhat larger change in employment as time passes.  This would of course be a rejection of the simple model but it would suggest ways of modifying the model to accommodate the new observations.  Moreover, the parameter estimates retain their usefulness even if the model is rejected. 

Noah’s claim that empirical puzzles are easy to come by is simply wrong.  Noah’s (intentionally) facetious example would only be a puzzle if we had some reason to believe the theory a priori.  None of the puzzles in macroeconomics have trivial explanations.  Here’s an example: inventory accumulation is strongly procyclical (a basic observation).  Is this a puzzle for demand driven business cycle theories?  (No – see Bils and Kahn 2000!).  Here is another example: the real relative price of investment is countercyclical (again a simple observation).  Is this a puzzle?  It might be.  If one thinks that fluctuations in investment demand is an important cause of business cycles then you might think that this real price would be pro-cyclical.  It’s not.  Should we retain this observation?  Is it acceptable to highlight this observation as a puzzle?  I think the answer to both questions is ‘yes’.  Of course, one way to avoid puzzles is to simply avoid trying to explain the data.  You can’t lose a race if you don’t run it. 

There is the famous example of Johannes Kepler who tested the heliocentric theory of the universe (proposed by Copernicus).  His model assumed that planets orbited the sun in perfectly circular orbits.  Unfortunately, the data rejected the model.  He later tried different orbital patterns and discovered that the heliocentric theory worked if the planets followed elliptical orbits.  Was the intermediate stage of rejection (and depression, confusion, …) a sign that his efforts were wasted?  Not at all.  The famous Princeton mathematician Andrew Wiles once compared his work to stumbling around in a darkened room.  This is an excellent analogy and I often have to remind myself that confusion and frustration are the norm. 

Caballero is certainly correct about macro/finance being at the periphery rather than at the core of macro.  It used to be that price rigidity was at the core while financial market imperfections were at the periphery.  I suspect this is slowly changing.  If you are going to try to understand the financial crisis, you will need to have a model that has a central role for financial market failures.