Dynamic programming:

references, abstracts and comments. Where there is no abstract, an abstract has been written, where abstracts are too long they have been abridged. Abstracts in languages other than English have been translated into English. The comment is personal, it points out errors and possible follow-ups, it is begun: CP: A few lines commenting on fitness, state and patch are quoted from Mangel and Clark 1988, see below.

A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z, Å

Bednekoff, P. A. & A. I. Houston (1994): Dynamic models of mass-dependent predation, risk-sensitive foraging, and premigratory fattening in birds. Ecology 75: 1131-1140.

The effect of mass-dependent predation on risk-sensitive foraging choices of small birds in two ecological contexts was examined using stochastic dynamic models. In the first context, nonmigratory foraging, predation risk dependent on the bird's body mass leads to risk-averse behavior in situations where risk-neutral behavior would otherwise be expected and generally reduces the amount of foraging and the level of reserves carried by foragers. In the second context, premigratory foraging, mass-dependent predation can lead to risk proneness during the buildup of reserves for migration. Risk proneness is favored if birds can reduce mass and hence predation after migration, thereby giving them an advantage to rapid departure. Other advantages of early departure can also lead to premigratory risk proneness but require more specific ecological assumptions. The literature on risk sensitivity in amount of food reward suggests a connection between migration and risk proneness on negative energy budgets but more tests are needed before any conclusions can be drawn.



Caraco, T. & S. L. Lima (1987): Survival, energy budgets, and foraging risk. In Commons, M. L., Kacelnik, A. & S. J. Shettleworth (eds.): Quantitative analyses of behavior, Vol. VI, Foraging; pp. 1 - 21. Lawrence Erlbaum, London.

Clark, C. W. & R. W. Butler (1999): Fitness components of avian migration: a dynamic model of Western Sandpiper migration. Evol. Ecol. Res. 1: 443 - 457.


Farmer, A. H. & A. H. Parent (1997): Effects of the landscape on shorebird movements at spring migration stopovers. Condor 99: 698 - 707.

We monitored the inter-wetland movements of 115 radio-tagged Pectoral Sandpipers (Calidris melanotos) at three migration stopovers in the Great Plains of North America during April and May from 1992 to 1995. While resident at a stopover, individuals were very localized in their movements. Over 40 % of the birds made no inter-wetland movements, and over 90 % of individuals moved less than 10 km from their original release site. Characteristics of wetlands where birds were released did not affect bird movements. However, the structure of the surrounding landscape explained up to 46 % of variation in individual bird movements. As the distance between wetlands decreased, and the proportion of the landscape composed of wetlands increased, individual birds moved between wetlands more frequently and moved longer distances from their release site. These movement patterns indicate that a more connected landscape allows shorebirds to exploit more feeding sites with reduced searchingcosts; a result consistent with foraging theory. We estimate a degree of landscape connectivity at which a wetland complex functions as a single large wetland as measured by sandpiper feeding patterns. Our data provide support for the idea that complexes of small, closely spaced wetlands can be important migration stopovers and may have significant conservation value.

Farmer, A. H. & J.A. Wiens (1998): Optimal migration schedules depend on the landscape and the physical environment: a dunamic programming view. J. Av. Biol. 29: 405 - 415.

We developed a dynamic state variable model of individual migrating shorebirds for use in testing hypotheses about spring migration strategies of the Pectoral Sandpiper Calidris melanotos. We conducted model sensitivity analyses to determine how predicted migration schedules might vary with respect to the landscape and the physical environment.
In landscapes with closely spaced, high-quality stopovers, female Pectoral Sandpipers can vary widely in their migration schedules and still arrive on the breeding grounds early enough and with sufficient energy reserves to achieve maximum reproductive success. Such a population might appear quite variable, and show no stopover patterns, even if all individuals were making optimal decisions. Latitudinal gradients in temperature and photoperiod differentially affect a bird's energy budget as it moves northward in the spring. Stopovers at more northerly locations are associated with higher metabolic rates, lower food abundance in early spring, and longer days for feeding. The optimal migration schedule in these conditions can be quite different from that in a homogeneous environment, and patterns observed in the field can be misinterpreted if the environmental gradients are not considered.
The landscape and the physical environment shape migration schedules and influence one's ability to interpret patterns observed at stopovers. Modeling these factors may lead to new insights about migration adaptations in heterogeneous environments.


Farmer, A. H. & J.A. Wiens (1999): Models and reality: time-energy trade-offs in Pectoral Sandpiper (Calidris melanotos). Ecology 80: 2566 - 2580.

We used a combination of modeling and field studies to determine the spring migration strategy of Pectoral Sandpipers (Calidris melanotos). We developed a dynamic programming model to predict patterns that should be detected along the migratory route if Pectoral Sandpipers use a strategy of early arrival at the breeding grounds (time minimization) or arrival at the breeding grounds with excess energy reserves (energy maximization). The predictions were then compared to data collected at stopover sites in the mid-continent of North America and at the breeding grounds in Alaska over a 5-yr period (1992-1996).
During spring migration to their Arctic breeding-grounds, Pectoral Sandpipers stop periodically to feed. The length-of-stay of such stopovers, for both time minimizers and energy maximizers, was predicted to vary inversely with date and body fat, and to vary directly with invertebrate abundance. We observed that: (1) length-of-stay was negatively correlated with capture date in Missouri and Nebraska, but not in Texas; (2) length-of-stay was not correlated with body fat at any site; and (3) length-of-stay was positively related to invertebrate abundance at the Nebraska and Missouri sites. As the population moves northwards in the spring, three regional patterns are diagnostic of migration strategy. Length-of-stay is predicted to be bimodal (energy maximizer) or constant (time minimizer) with respect to latitude, but neither pattern was observed. The migration window, or period of time during which spring migrants occur, was predicted to decrease with increasing latitude for time minimizers, a pattern that was seen for both males and females. Body fat was predicted to increase with latitude for energy maximizers, a pattern that was seen for females but not for males.
The evidence suggests that males and females differ in their spring migration strategies. Both sexes attempt to arrive in the Arctic as early as possible after ice breakup in the spring. Additionally, females gain significantly higher fat loads than males (up to 60 % body fat for females) during migration, and these energy reserves may later enhance female reproductive success. However, females gained large fat loads only during 1993 and 1995, which had above normal spring precipitation along the migration route. We believe that the correlation between female body fat and precipitation reflects an abundance of high-quality stopover habitat during wet springs. This view is supported by model sensitivity analyses showing that the spacing and quality of stopover habitat can strongly influence observed migration patterns. Our results suggest the need to focus additional research on the landscape-level features of the flyway through which shorebirds migrate.



Houston, A. I., Clark, C. W., McNamara, J. M. & M. Mangel (1988): Dynamic models in behavioural and evolutionary ecology. Nature 332: 29 - 34.

(Conclusions) The technique for analysing behaviour we present here has the following advantages: (i) It takes account of the state of the animal and how that state changes according to the animal's action and the environment; (ii)it provides a common currency for assessing behavioural choices in terms of overall fitness, which can be used to analyse trade-offs between different actions; (iii) it includes constraints on state variables and behaviour. Models based on this technique often include a higher degree of biological realism and lead to predictions and insights not provided by simpler methods. Stochastic dynamic programming provides a method for finding the optimal behaviour (or behaviours)within the dynamic framework. As a computational technique, stochastic dynamic programming has certain limitations, perhaps the most serious of which is the 'curse of dimensionality' in which computational needs grow vastly as the number of state variables increases. Hence, there is an upper limit to the degree of biological complexity that the method can realistically encompass.
We have applied this dynamic approach to many other situations, such as diel vertical migration of aquatic organisms, optimal choice of prey items, diving behaviour of water birds, growth and migration of salmon, sex change in slugs, web locations of spiders, and food hoarding by small birds.

In our main examples, there are no interactions between animals. We have mentioned that under some circumstances evolutionarily stable dynamic strategies can be modelled in a relatively simple way. In general, the development of such models is likely to encounter conceptual and computational difficulties. Nevertheless, we believe that this is a biologically important issue that deserves further theoretical and empirical research, and that the technique of stochastic dynamic programming will be a powerful tool in this enterprise.


Houston, A. I. & J. M. McNamara (1988): A framework for the functional analysis of behaviour. Behav. and Brain Sciences 11: 117 - 154.

Houston, A. I., McNamara, J. M. & J. Hutchinson (1993): General results concerning the trade-offs between gaining energy and avoiding predation. Phil. Trans. R. Sc. Lond. B341: 375-397.

When animals can choose from a range of feeding options, often those options with a higher energetic gain carry a higher risk of predation. This paper analyses the optimal trade-off between food and predation. We are primarily interested in how an animal's decision and its state change over time. Our models are very general. They can be applied to growth decisions, such as choice of habitat, in which case we might consider how the state variable size changes over an animal's lifetime. Equally, our models are applicable to short-term foraging decisions, such as vigilance level, in which case we might consider how energy reserves vary over a day. We concentrate on two cases: (i) the animal must reach a fixed state, its fitness depending on when this is attained; (ii) the animal must survive to a fixed time, its fitness depending on its final state.
In case (i) minimization of mortality per unit increase of state is optimal under certain baseline conditions. In case (ii) behaviour is constant over time under baseline conditions (the 'Risk-spreading Theorem'). We analyse how these patterns are modified by complicating factors, e.g. time penalties, premature termination of the food supply, stochasticity in food supply or in metabolic expenditure, and state-dependence in the ability to obtain food, in metabolic expenditure and in predation risk. From this analysis we obtain a variety of possible explanations for why an animal should reduce its intake rate over time (i.e. show satiation). We show how earlier work can be viewed as special cases of our results.

Houston, A. I. & J. M. McNamara (1999): Models of adaptive behaviour: an approach based on state. Cambridge Un. Press.


Mangel, M. & C. W. Clark (1986): Towards a unified foraging theory. Ecology 332: 29 - 34.

Mangel, M. & C. W. Clark (1988): Dynamic modeling in behavioral ecology. Princeton Un. Press, Princeton.

PATCH (selection as a paradigm): If 'foraging theory,' understood in the usual modern context, has a clear starting-point, it can probably be traced to two papers which appeared together in teh American Naturalist in 1966 (MacArthur and Pianka 1966, and Emlen 1966) on the optimal use of a patchy environment. In these papers the concern is with behavior that in some sense optimizes a rate of energy return to the foraging individual.
We will think of 'patch selection' in a much broader context - although certainly the search for food is a crucial aspect of animal behavior - in which 'patches' correspond to activity choices. These activity choices are characterized by rewards to the individual (e.g. food obtained or eggs laid), costs to the individual (e.g. time spent, metabolic costs), and risks to tghe individual (e.g. the possibility of predation). The problem of 'optimal patch use' thus reduces to the following question: What is the best way to trade off the rewards with the costs and the risks?(...)
STATE: In this section we discuss the patch selection problem using the simplest possible state variable model. We use a single state variable X(t) to characterize the state of the forager at time t. At this point, the exact nature and units of X(t) do not need to be specified, but it may be helpful to think of X(t) as the forager's energy reserves at time t.(...)
The optimization criterion: FITNESS: The principle of evolution by natural selection suggests that the appropriate optimization criterion for behavioral models is lifetime fitness. In this book we have not provided a 'grand' or 'unifying' definition of fitness, but instead have shown how the definition of fitness arises in a natural way in each instance, as a consequence of the basic biology of the problem. Typically, we have adopted a definition of lifetime fitness in terms of survival and reproduction.(...)

McNamara, J. M. & A. I. Houston (1982): Short-term behaviour and life-time fitness. In: McFarland, D. J. (ed.): Functional ontogeny, pp. 60 - 87. Pitman, London.

McNamara, J. M. & A. I. Houston (1986): The common currency for behavioral decisions. Am. Naturalist 127: 358-378.

The study of optimal life histories involves the maximization of lifetime fitness but usually ignores the details of behavioral sequences. In contrast, the study of optimal behavioral sequences usually looks at the details of a sequence in isolation and not as a part of the whole life history of the animal. The currency that is maximized is assumed to be related to lifetime fitness, but this relationship is rarely explored. In this paper we develop a common currency for behavioral decisions that is directly related to lifetime fitness. The common currency makes it possible to compare the benefits of qualitatively different behaviours. We show that many different costs can be used to explain a given behavioral sequence. Most of these costs have no biological interpretation. From these we single out a particular cost, the canonical cost, which measures the reduction in fitness that results from choosing a suboptimal action.
Our general framework is illustrated by the example of a small bird in winter. We quantify the value of energy in terms of fitness and show how this value depends on energy reserves and time of day. As a result of this dependence, optimal foraging decisions depend on energy reserves and time of day, as does the optimal trade-off between foraging and looking around for predators.


McNamara, J. M. & A. I. Houston (1987): A general framework for understanding the effects of variability and interruptions on foraging behavior. Acta Biotheoretica 36: 3-22.

McNamara, J. M., Merad, S. & A. I. Houston (1991): Risk-sensitive foraging for a reproducing animal. Anim. Behav. 41: 787 - 792.

A model in which a foraging animal can reproduce if its energy reserves reach a critical level is presented. The animal has the choice of two foraging options which have the same mean net gain but differ in their variance. The policy that maximizes expected lifetime reproductive success was found. The pattern of risk-sensitive behaviour predicted by this model was compared with that predicted by a similar model in which no reproduction occurs and the optimality criterion is to minimize the probability of death. The two models give different predictions. This serves to emphasize that there is no single model of risk-sensitive foraging. Risk-sensitivity results from a non-linear relationship between fitness and energy reserves. This relationship provides a single principle that is common to all models of risk-sensitive foraging. Not surprisingly, the form of the relationship will depend on the biology of the animal under consideration.

McNamara, J. M. & A. I. Houston (1992): Risk-sensitive foraging-a review of the theory. Bull. Math. Biol. 54: 355-378.


O'Reilly, K. M. & J. C. Wingfield (1995): Spring and autumn migration in Arctic shorebirds: same distance, different strategies. Amer. Zool. 35: 222 - 233.

The Arctic is an extremely inhospitable region for most of the year, but during the summer months it bursts with life. A major proportion of avian species nesting in the Arctic are shorebirds. They migrate thousands of kilometers from their wintering grounds to take advantage of abundant food resources each summer and display a variety of migratory strategies. In an attempt to classify this variation, not only between spring and autumn migration, but within a migration, we present four categories. These relate to the distance a species generally flies between stopovers: short distance bout, intermediate distance bout, long distance bout, and combinations. We then explore further differences between spring and autumn migration. Spring migrants experience poor weather and decreased food availability as they fly north. Many cope with huge flocks, whichserve as protection from predators, but may also reduce foraging efficiency and increase aggression. In contrast, autumn migrants generally encounter favorable weather and ample food. Flock sizes are usually smaller, thus foraging efficiency is higher and aggression lower than during spring migration. Physiologically, spring migrants are preparing for breeding and reproductive hormones are secreted. In the Western Sandpiper (Calidris mauri), luteinizing hormone levels are higher for spring than autumn migrants. Late spring migrants have higher testosterone levels than either early spring migrants or autumn migrants. Corticosterone levels are also higher in spring vs. autumn migrants. Although spring and autumn migrants travel similar distances, their strategies differ behaviorally and physiologically.



Real, L. A & T. Caraco (1986): Risk and foraging in stochastic environments. Ann. Rev. of Ecology and Systematics. 17: 371-390.


Stephens, D. W. & J. R. Krebs (1986): Foraging Theory. Princeton Un. Press.


  • To "Studies of migrating Dunlin Calidris alpina in the Sound area, S. Sweden: Introduction"
  • To "Risk-prone or risk-averse? Dunlin Calidris alpina migrating with and without moult-gaps in the Baltic area"
  • To "Phenology and biometry of Dunlin Calidris alpina migrating by way of the Sound area, S. Sweden"
  • To "Migrating Dunlin Calidris alpina in the Baltic area: the moult issue"
  • To "Wintering and spring staging Dunlin Calidris alpina in the south Baltic area"
  • To "Migratory progress of juvenile and adult Dunlin Calidris alpina from two perspectives: the Baltic and the Waddensea"
  • To "Bill-length distributions in Dunlin Calidris alpina"
  • To the bill length account
  • About "adult buff" coverts
  • To the Meissner scale
  • Back to start page


    Last addition (20 entries) 9.3.06.