More Thoughts on Agent Based Models

My recent post on Agent Based Models (ABMs) generated a few interesting responses and I thought I would briefly reply to a couple of them in this post.  In particular, two responses came from people who actually have direct experience with ABMs.

Rajiv Sethi posts a response on his own blog.  Some excerpts:

Chris House has managed to misrepresent the methodology so completely that his post is likely to do more harm than good.

[Well that doesn’t sound too good …]

Agents can be as sophisticated and forward-looking in their pursuit of self-interest in an ABM as you care to make them; they can even be set up to make choices based on solutions to dynamic programming problems, provided that these are based on private beliefs about the future that change endogenously over time.

What you cannot have in an ABM is the assumption that, from the outset, individual plans are mutually consistent. That is, you cannot simply assume that the economy is tracing out an equilibrium path. The agent-based approach is at heart a model of disequilibrium dynamics, in which the mutual consistency of plans, if it arises at all, has to do so endogenously through a clearly specified adjustment process. This is the key difference between the ABM and DSGE approaches [.]

In a similar vein, in the comments section to the earlier post, Leigh Tesfatsion offered several thoughts many of which fit squarely with Rajiv’s opinion.  Professor Tesfatsion uses ABMs in a multiple settings including economics and climate change – I’m quite sure that she has much more experience with such models that I do (I basically don’t know anything beyond a couple of papers I’ve encountered as a referee here and there).  Here are some excerpts from Leigh’s comments:

Agents in ABMs can be as rational (or irrational) as their real-world counterparts…

The core difference between agent modeling in ABMs and agents in DSGE models is that agents in ABMs are required to be “locally constructive,” meaning they must specify and implement their goals, choice environments, and decision making procedures based on their own local information, beliefs, and attributes. Agent-based modeling rules out “top down” (modeler imposed) global coordination devices (e.g., global market clearing conditions) that do not represent the behavior or activities of any agent actually residing within the model. They do this because they are interested in understanding how real-world economies work.

Second, ABM researchers seek to understand how economic systems might (or might not) attain equilibrium states, with equilibrium thus studied as a testable hypothesis (in conjunction with basins of attraction) rather than as an a priori maintained hypothesis.

I was struck by the similarity between Professor Sethi and Professor Tesfatsion’s comments. The parts of their comments that really strike me are (1) the agents in an ABM can have rational rules; (2) in an ABM, there is no global coordination imposed by the modeler. That is, agents behaviors don’t have to be mutually consistent; and (3) ABMs are focused on explaining disequilibrium in contrast to DSGE models which operate under the assumption of equilibrium at all points.

On the first point (1) I agree with Rajiv and Leigh on the basic principle. Agents in an ABM could be endowed with rational behavioral rules – that is, they could have rules which are derived from an individual optimization problem of some sort. The end result of an economic optimization problem is a rule – a contingency plan that specifies what you intend to do and when you intend to do it. This rule is typically a function of some individual state variable (what position are you in?). In an ABM, the modeler specifies the rule as he or she sees fit and then goes from there. If this rule were identical to the contingency plan from a rational economic actor then the two modelling frameworks would be identical along those dimensions. However, in an ABM there is nothing which requires that these rules adhere to rationality. The models could accommodate rational behavior but they don’t have to. To me this still seems like a significant departure from standard economic models that typically place great emphasis on self-interest as a guiding principle. In fact, the first time I read Rajiv’s post, my initial thought was that an ABM with a rational decision rule would be essentially a DSGE model. All actions in DSGE models are based on private beliefs about the system. Both the system and the beliefs can change over time.  I for one would be very interested if there were any ABMs that fit Rajiv’s description that are in use today.

The second point (2) on mutual consistency is interesting. It is true that in most DSGE models, plans are indirectly coordinated through markets.  Each person in a typical economic model is assumed to be in (constant?) contact with a market and each confronts a common price for each good.  As a result of this common connection, the plans of individuals in economic models are assumed to be consistent in a way that they are not in ABMs.  On the other hand, there are economic models that do not have this type mutual consistency.  Search based models are the most obvious example.  In many search models, individuals meet one-on-one and make isolated bargains about trades.  There are thus many trades and exchanges occurring in such model environments and the equilibria can feature many different prices at any point in time.  This might mean that search / matching models are a half-way point between pure Walrasian theories on the one hand and ABMs on the other.

The last issue (3) that Rajiv and Leigh brought up was the idea that ABMs seek to model “disequilibrium” of some sort. I suspect that this is somewhat more an issue of terminology rather than substance but there may be something more to it.  Leigh’s comment in particular suggests that she is reserving the term “equilibrium” for a classical rest point at which the system is unchanging. I mentioned to her that this doesn’t match up with the term “equilibrium” in economics. In economic models (e.g., DSGE models) equilibria can feature erratic dynamic adjustment over time as prices and markets gradually adjust (e.g., the New Keynesian model) or as unemployment and vacancies are gradually brought into alignment (e.g., the Mortensen Pissarides model) or as capital gradually accumulates over time (e.g., the Ramsey model).  Indeed, the equilibria can be “stochastic” so that they directly incorporate random elements over time. There is no supposition that an equilibrium is a rest point in the sense that (I think) she intends.  When I mentioned this she replied:

As for your definition of equilibrium, equating it with any kind of “solution,” I believe this is so broad as to become meaningless. In my work, “equilibrium” is always used to mean some type of unchanging condition that might (or might not) be attained by a system over time. This unchanging condition could be excess supply = 0, or plans are realized (e.g., no unintended inventories), or expectations are consistent with observations (so updating ceases), or some such condition. Solution already means “solution” — why debase the usual scientific meaning of equilibrium (a system at “rest” in some sense) by equating it with solution?

I suspect that in addition to her background in economics, Professor Tesfatsion also has a strong background in the natural sciences and is somewhat unaccustomed to terminology used in economics and prefers to use the term “equilibrium” as it would be used in say physics.[1] In economics, an outcome which is constant and unchanging would be called a “steady state equilibrium” or a “stationary equilibrium.”  As I mentioned above, there are non-stationary equilibria in economic models as well.  Even though quantities and prices are changing over time, the system is still described as being “in equilibrium.”  The reason most economists use this terminology is subtle.  Even though the observable variables are changing, agents’ decision rules are not – the decision rules or contingency plans are at a rest point even though the observables move over time.

Consider this example. Suppose two people are playing chess. The player with the white pieces is accustomed to playing e4. She correctly anticipates that her opponent will respond with c5 – the Sicilian Defense. White will then respond with the Smith-Morra Gambit to which black with further respond with the Sicilian-Scheveningen variation. Both players have played several times and they are used to the positions they get out of this opening. To an economist, this is an equilibrium.  White is playing the Smith-Morra Gambit and black plays the Sicilian-Scheveningen variation. Both correctly anticipate the opening responses of the other and neither wants to deviate in the early stages of the game. Neither strategy changes over time even though the position of the board changes as they play through the first several moves. (In fact this is common to see in competitive chess – two players who play each other a lot often rapidly fire off 8-10 moves and get to a well-known position.)

In any case, I’m not sure that this means economists are “debasing” the usual scientific meaning of equilibrium or not but that’s how the term is used in the field.

One last point that came up in Rajiv’s post which deserves mention is the following:

A typical (though not universal) feature of agent-based models is an evolutionary process, that allows successful strategies to proliferate over time at the expense of less successful ones.

This is absolutely correct.  I didn’t think to mention this in the earlier post but I clearly should have done so.  Features like this are used often in evolutionary game theory.  In those settings, we gather together many individuals and endow them with different rules of behavior.  Whether a rule survives, dies, proliferates, etc. is governed by how well it succeeds at maximizing an objective.  Rajiv is quite correct that such behavior is common in many ABMs and he is right to point out its similarity with learning in economic models (though it is not exactly the same as learning).

[1] A reader pointed out that Leigh Tesfatsion’s Ph.D. is in economics and so she is well aware of non-stationary equilibria or stochastic equilibria. My original post incorectly suggested that she might unaware of economic terminology (Sorry Leigh). Leigh prefers to reserve the term “equilibrium” for a constant state as it is in many other fields. Her choice for terminology is fine as long as she and I are clear as to what were are each talking about.

Advertisements

9 thoughts on “More Thoughts on Agent Based Models

  1. Perhaps only by accident, or bad luck, ABMs are not now mainstream, so it is its proponents who are stuck with having to show that it is better than the standard approach.

    As an non-ABM-doing economist, it seems that there two issues in particular: 1) as we all know, there are infinitely many ways to be behavioral (not rational), and add to this (2) the extra freedom that comes from the deliberate ejection of any requirement that expectations be fulfilled. (So I wonder: What CAN’T happen in a quantitative ABM model?)

    So it would be useful to know more about just how one uses ABM vs. standard tools to better account for some facts, business cycle facts, say, than some canonical macro model. It’d help (me at least) to know exactly how, in this specific example, how they (ABMs) would be disciplined.

    That sort of head-to-head seems useful if we want to move past a priori arguments for why one is better than the other.

  2. “I suspect that Professor Tesfatsion has a strong background in the natural sciences and is somewhat unaccustomed to terminology used in economics.”

    It pains me to say this Prof. House, because generally I’m a fan of your blog, but this is quite dismissive and condescending. Prof. Tesfatsion has a Ph.D. in economics from the University of Minnesota(!), so I’m pretty sure she understands the concept of equilibrium as it’s used in economics. You could have discovered this, as I did, from an 30-second search of her webpage. Your response to her comments and Rajiv Sethi’s post seems to pretty much concede the points that they made, without saying so explicitly.

    A few years ago Chris Sims gave a speech at an INET conference where he said something along the lines of “DSGE isn’t perfect, but it’s the best we’ve got right now.” I agree with this sentiment which is one of the reasons why I like your blog; I think you do a good job of making that case with respect to many critics of macro modeling. But I think you’ve stretched yourself a bit with your posts on ABM. They may be useful, they may not be. It’s too soon to tell. But your posts have shown that you don’t really know the area well enough to critique it in a meaningful way, and I think it would be better for you to acknowledge that fact rather than be dismissive of its advocates.

    • Hi Steve,

      I’ll rewrite my description of her background… I didn’t mean the post to be dismissive or condecending. Hopefully Leigh will forgive me for not finding out more about her background.

      On your second point, I don’t think the posts are necessarily critiques of ABMs. I’m really just trying to decide / understand for myself what the main differences between ABMs and other models are.

      CH

  3. Chris —

    RE: Response to your September 3 posting on ABM/ACE

    First, I would like to point interested readers (including yourself) to the following new macro working paper, intended to be a bridge between DSGE and Agent-Based Computational Economics (ACE) modeling. It has been posted at the ACE macroeconomics site (among many other works by other researchers), but you can directly access it at the working paper site below:

    ===========================

    ACE Research Area: Agent-Based Macroeconomics
    http://www2.econ.iastate.edu/tesfatsi/amulmark.htm

    Ekaterina Sinitskaya and Leigh Tesfatsion, “Macroeconomies as Constructively Rational Games” (pdf,1.2MB), Working Paper No. 14018, Economics Department, Iowa State University, August 2014.

    http://www2.econ.iastate.edu/tesfatsi/MacroConstructiveRationalityWP.SinitskayaTesfatsion.pdf

    Abstract: “Real-world decision-makers are forced to be locally constructive, in the sense that their actions are constrained by the interaction networks, limited information, and computational capabilities at their disposal. This study poses the following question: Suppose utility-seeking consumers and profit-seeking firms in an otherwise standard dynamic macroeconomic model are required to be locally constructive decision-makers, unaided by the external imposition of global coordination conditions. What combinations of locally constructive decision rules result in good macroeconomic performance relative to a social planner benchmark model, and what are the game-theoretic properties of these decision-rule combinations? We begin our investigation of this question by specifying locally constructive decision rules for the consumers and firms that range from simple reinforcement learning to sophisticated adaptive dynamic programming algorithms. We then use computational experiments to explore macroeconomic performance under alternative decision-rule combinations. A key finding is that simpler rules can outperform more sophisticated rules, but that forward-looking behavior coupled with a relatively long memory permitting past observations to inform current decision-making is critical for good performance.”

    =========================

    As seen in this paper, our purpose is to explore to what degree utility-seeking consumers and profit-seeking firms with intertemporal utility and profit objectives in a structurally-standard macroeconomic model context can learn over time how to survive and even prosper in the economy with having to depend on some external modeler-imposed coordination conditions. Note that our modeled “Dynamic Macroeconomic (DM) Game” features labor and goods markets in every period t that are cleared as competitive auctions by a standard bid-ask matching process. The key difference is that the bids and asks submitted by consumers and firms reflect their own current information, beliefs, and attributes (e.g., money and goods stocks) rather than being externally coordinated by some modeler-imposed optimization or rationality presumptions.

    Here is what I say in an abstract for a different paper (for another day): “If economic modelers truly wish to respect the rationality of decision-makers, they should have the courage of their convictions; they should not be doing for their modeled decision-makers what in reality these decision-makers must do for themselves.”

    IMO we do not yet understand the processes by which real-world macroeconomies manage to sustain themselves over time. We are therefore far from being in a situation where it is scientifically acceptable simply to presume sustainability for such systems.

    A final note on “equilibrium”. I said in my previous post that I use the term “equilibrium” to mean that an economic system has achieved some type of “unchanging condition.” By this I meant some type of persistent pattern in the outcomes of the system. I then gave a number of examples. None of these examples presumes a “stationary state” in the sense of an unchanging state of the world. Indeed, what we typically find in ACE models is that a system can achieve an “unchanging condition” (persistent pattern of outcomes) at some levels even while, underneath, things are still changing. Thus, for example, in my labor market research I saw convergence (sometimes rapid convergence) to persistent interaction networks among firms and workers even though the workers and firms were continuing to evolve their work-site strategies determining their expressed work-site behaviors. That is, the interaction networks stabilized well in advance of stabilization in the expressed worksite behaviors supported by these interaction networks.

    This multi-leveled conception of equilibrium is a far richer (and I believe more realistic) conception than the standard definition(s) of equilibrium I teach in my grad macroeconomics course Econ 502 that is regularly offered each Fall.

    I just had to get that dig in. Yes, not only did I get my PhD from the U of Minnesota in the early years of the rational expectations revolution, I actually regularly teach standard macroeconomics as well as more frontier agent-based macroeconomics to my macro grad students 🙂

    Master’s Level Macroeconomic Theory (Econ 502):
    http://www2.econ.iastate.edu/classes/econ502/tesfatsion/syl502.htm

    Topics, Discussion Questions, and Readings

    Introduction
    Course Overview
    Empirical Characteristics of the U.S. Economy
    Macro Modeling: Alternative Approaches
    Aggregate Macroeconomic Modeling: A Microfoundations Critique
    The Basic IS-LM Model
    Aggregate Supply and Price Adjustment
    Critiques of Aggregate Macroeconomic Modeling: Summary Overview
    Economic Growth: A Microfoundations Approach
    Growth Overview
    Basic Dynamic Economic Modeling Concepts
    Descriptive Growth Models
    Optimal Growth Models
    Overlapping Generations Models
    Treatment of Expectations in Macroeconomic Models
    Rational vs. Adaptive Expectations
    Implications for Policy Choice Over Time
    Dynamic Stochastic General Equilibrium (DSGE) Models and Taylor Rules
    Introduction to DSGE Modeling
    Taylor Rules and Central Bank Monetary Policy
    Macroeconomic Modeling of Endogenous Coordination
    Coordination Issues for Macroeconomies
    Constructive Modeling of Endogenous Coordination:
    Agent-Based Macroeconomics
    Illustrative Application

  4. Pingback: Somewhere else, part 161 | Freakonometrics

  5. Pingback: Introduction to Agent-Based Models with respect to the Future of Macroeconomics | Zachary David's

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s