CONTENTS

How Economists Conduct Economic Evaluation

Director’s Corner

Next Meeting

NEC-63 Spring 2001 March22-23, 2001 Washington, DC Policy and Distribution Issues in Agricultural Commodity Promotion Programs

How Economists Conduct Economic Evaluation

by Harry M. Kaiser, Cornell University

Over the past several years, the NICPRE Quarterly has published the results of a lot of economic evaluation studies. While the methods used in these studies have always been explained briefly in these articles, there has never been a detailed explanation as to the general paradigm used by economists to investigate the economic impacts of commodity promotion and advertising. Accordingly, we devote the entire issue of this NICPRE Quarterly to discussing how economists evaluate commodity checkoff programs.

This article is organized into five sections. First, a conceptual overview of economic evaluation is presented using the economic paradigm of market supply and demand analysis. Next, the basic determinants of market demand are discussed. Third, the econometric approach to checkoff evaluation is described, followed by a discussion on return on investment estimation. The section concludes with a basic example of milk advertising evaluation.

Conceptual overview economic evaluation

There are four basic questions underlying economic evaluation of commodity checkoff programs. First, does the checkoff program result in increased demand for the basic commodity? To be effective, it is necessary that demand in the marketplace is higher as a result of the program. Second, does the checkoff program result in a higher price to producers paying for the program? It is possible for the program to increase market demand, but not price if the increased demand is equally offset by an increase in quantity supplied by producers. Third, do the industry-wide benefits exceed the total cost of the checkoff program? This is the bottom-line and most important effectiveness criterion for the producers funding the program. The answer to this question depends on a variety of market and institutional characteristics underlying the commodity in question, including: the extent of trade barriers, structure of the market, degree of government intervention (e.g., price supports), and nature of supply response. Finally, what is the optimal allocation of producers’ checkoff? While the benefits may exceed the costs, it is possible that the return on investment can be increased through a better allocation of the budget. The optimal allocation question pertains to a variety of ways checkoff dollars are spent, including money spent by program activity, by product, by type of media, and by market.

To evaluate the economic impacts of checkoff programs on quantity, price, and profits, economists use a market supply-demand framework. The negative slope of this curve illustrates the basic “law of demand,” as price increases, quantity demanded in the market decreases, and as price decreases, quantity demanded in the market increases. Obviously, there other factors besides price that affect market demand and these factors affect the position of the demand curve. Consequently, these factors must be accounted for in any quantitative analysis of market demand (see next section for detail).

Price determination in a market is based on the interaction of market demand and supply. The market supply curve measures how quantity supplied in the market responds to increases and decreases in price. The first possibility is that supply is fixed regardless of price level. Such a situation is most likely to happen only in the very short run when producers do not have time to make adjustments in production in response to a price change. The second possibility is for positive supply response to occur, reflecting the “law of supply.” The last possibility is called an “infinite supply response,” which means any increase in demand is equally offset by an increase in quantity supplied. This possibility is more likely in markets that have few barriers to entry or exit by producers, which generally occurs in markets that do not have barriers to trade, or are perfectly competitive.

The intention of most commodity checkoff programs is to increase the market demand for the commodity. If the program is successful in increasing market demand, the resulting market effects of this depend, in large part, on the nature of the supply response. Figure 1 illustrates the example of no supply response. Initial market “equilibrium” without the checkoff program occurs where supply and demand are equal, and result in a market price and quantity of P1 and Q1, respectively. Suppose that a successful advertising campaign funded by the checkoff program causes the market demand curve to increase from D1 to D2. However, since supply is fixed, the only way to satisfy the increase in demand is for market price to increase from P1 to P2. The benefits to producers in this market from advertising is the gain in industry-wide profit, or “producer surplus,” given by the shaded area in the figure. This gain in producer surplus gives the measure of the advertising campaign’s benefits to producers and should be compared to total advertising costs to determine the profitability of the program.

Figure 2 illustrates a similar case for a positive market supply response. Here the successful advertising campaign again causes the market demand to increase from D1 to D2. Since supply is no longer fixed, there is now a quantity as well as price response to the increase in demand. Price increases from P1 to P2 and quantity increases from Q1 to Q2 as a result. The benefits to producers is the gain in producer surplus depicted by the shaded area, which should be compared against the total cost of advertising.

The final case of infinite supply response is illustrated in Figure 3. In this case, the increase in market demand due to advertising is accompanied by an equal increase in quantity supplied. The net result is that there is no change in price, and therefore no gain in producer surplus. In markets that are characterized by an infinite supply response, it would not be a good investment to increase demand since there are no positive benefits of doing so.

Determinants of market demand

• price of substitutes
• price of compliments
• consumer income
• population
• population demographics
• consumer tastes and preferences
• consumer health concerns
• current and lagged generic (and brand) demand expansion activities
• current and lagged competing product demand expansion activities

Econometric approach

Economists use econometric techniques to empirically estimate market demand and supply functions. This technique uses statistical methods (called regression analysis) along with data (time series or cross-sectional) to correlate changes in various factors with changes in demand or supply. Economists are interested in the “elasticities” that are estimated from the econometric model. An elasticity provides a measure of the percentage change in demand (or supply) given a 1 percent change in one of the demand (or supply) determinants, holding all other determinants constant. For instance, the elasticity of demand with respect to promotion measures the percentage change in market demand given a 1 percent change in promotion.

Most evaluation studies of checkoff programs have been primarily concerned with estimating the elasticity of demand with respect to advertising and/or promotion. The advantage of econometrics is that elasticities can be calculated for all demand (and supply) determinants and can be compared. A comparison of elasticities is useful because this enables one to see what factors have the largest impact on changing demand. On the supply side, the use of econometrics allows one to determine the responsiveness of quantity supplied with respect to price. As previously discussed, this information is critical in determining whether the checkoff program is profitable or not.

Regarding data requirements, econometric studies of commodity checkoff programs have generally used time series data. These studies have used annual, quarterly, monthly, and even weekly time series data. As a general rule, the more frequent the time interval the better, e.g., monthly data are preferred to quarterly. In addition, more observations are preferred to less, and a minimum of 30-40 observations is usually considered to be necessary for statistical validity. The array of time series data must be consistent, i.e., if one wants to use monthly observations over a five year period, then every variable included in the model must have monthly observations over the five year period. Finally, if there are inaccuracies in the data, then the resulting estimates from the econometric model will also be inaccurate.

Return on investment estimation

Most econometric studies also compute the return on investment (ROI) for checkoff activities, which is a benefit-cost ratio for the activity. The ROI calculation is based on the econometric estimates, and there are two types of ROI, an average ROI and a marginal ROI.

Average ROI - an average ROI measures the average benefit-cost ratio of a checkoff activity over a specific period of time by comparing market conditions with and without the activity. The econometric results are used to simulate what market prices, quantities, producer surplus would have been with and without the checkoff program over some specific time period. Then, an average ROI is computed by dividing the benefits of the program (difference in producer surplus) by the costs of the program. The resulting number gives the average return to producers of one dollar invested in the checkoff program for the time period being investigated.

Marginal ROI - a marginal ROI measures the benefit-cost ratio of an extra dollar invested in a checkoff activity. The econometric results are used to simulate market prices, quantities, producer surplus with actual checkoff levels and with checkoff levels increased by a small amount (e.g., 1 percent) over actual levels. Then, a marginal ROI is computed by dividing the increase in producer surplus due to the 1 percent increase in checkoff spending by the increase in costs due to the 1 percent increase in checkoff investment. The resulting number gives the net return to producers of an additional dollar invested in the checkoff program for the time period being investigated.

Example: Impact of milk advertising on milk demand in New York City

To illustrate, consider a previous study by Cornell University on the impact of generic fluid milk advertising in New York City. In this study, it was assumed that per capita fluid milk sales depend on:
- retail price of milk
- non-alcoholic price index
- per capita income
- consumer dietary fat concerns
- competing beverage advertising per capita expenditures
- national plus local generic milk advertising per capita expenditures
- seasonal variables

To capture the carryover effect of advertising, current and lagged values for the advertising variables were included in the model. The milk demand model was estimated with monthly data from January 1986 to June 1995. Table 1 presents the average elasticity values for selected demand determinants. Recall that an elasticity measures the percentage change in per capita milk demand given a 1 percent change in the demand factor. For example, the own price elasticity of demand was estimated to be -0.135 indicating that on average over this period a 1 percent increase in the retail price of milk resulted in a 0.135 percent decline in per capita quantity milk demanded. Based in these results and the following simulation procedures, an average ROI was computed. Monthly per capita milk demand was simulated from 1987 to 1995 under two scenarios: with and without New York advertising. All demand variables, except milk advertising, were set to historic levels. The benefits of generic advertising is the additional revenue to producers of increased milk sales, while the cost of advertising is the total assessment dairy farmers pay for this program.

An average ROI for generic advertising in the New York City market was computed to be 2.77 for this period. That is, for every dollar invested in generic advertising in New York City, dairy farmers received $2.77 back in increased profit.

Because there is possible errors associated with any statistical analysis, it is good practice to conduct sensitivity analysis on the model. In the New York City study, a “90 percent confidence interval” on the ROI was constructed. A 90 percent confidence interval provides an upper and lower bound estimate of the ROI in which one can be confident 90 percent of the time that the true ROI falls within. In this case, the lower bound ROI was 1.20 and the upper bound ROI was 4.50. Since even the lower bound estimate is above 1.0, one can be very confident in the result that the benefits of generic milk advertising in New York City exceed the costs.

[ return to text] [ top ]

Table 1. Estimated average demand elasticities (1986-95) for New York City.

[ return to text] [ top ]

[ return to text] [ top ]

[ return to text] [ top ]