Tag Archives: economic indiciators

CPI Market Basket Determined by only 0.006% of Americans

It seems like every week we discover a new way in which our daily lives are being tracked. Every phone-call, text, e-mail, credit card transaction, even every web-page we ever visit can be intercepted and archived. That’s why it seems like a quaint throwback that the CPI market basket is computed from a self-reported survey of 0.006% of American Households.

The market basket is determined from the Consumer Expenditure (CE) Survey, conducted on the BLS’s behalf by the Census Bureau. This information is supplemented by the Telephone Point of Purchase (TPOP) Survey, an old-fashion telephone survey conducted by the Census. 

Considering the massive liabilities that are tied to the computation of the CPI, using antiquated methods of data collection seems at best reckless, at worst criminally negligent. There are several reasons that the current method is dubious.

Using statistical sampling introduces an amount of error in the quantity you are trying to measure. The logic is that sampling all 316 million US citizens would be prohibitively expensive so pick a smaller sample that is still representative of the overall population. The BLS has chosen a very small sample of 7000 households, making it imperative that their assumptions are correct. Given the monotonicity of thought amongst BLS economists, it would not be surprising if what they thought was representative was in fact biased.

A combination of academic arrogance combined with constrained budgets leads me to question how representative the market basket is of the wider population it is meant to measure. This need not be the case, as spending data is available for purchase from major credit card providers. Taking a census is expensive and inefficient. Additionally, there doesn’t seem to be any thought given to the measurement error inherent in self-reported and phone surveys.

It is unlikely that people record their true purchases; reporting instead an idealized image of what they think they should have purchased. No one is going to accurately report the amount of alcohol, cigarettes, cheese-burgers, candy, soft-drinks, green tea, kale, exercise, travel or whatever else they use to numb the reality of everyday existence. For an everyday example of this phenomenon, look at anyone’s facebook page. Human beings project their own self-image into the data they report to the world (and even themselves), and the CE survey is no different.

Using anonymous spending data filtered from credit card processors would solve several problems at once:

  • It would provide researchers access to a much larger sample of the population and make the CPI market basket much a more accurate representation of individual spending habits.
  • It would eliminate much of the measurement error associated with current survey methods. Your credit card statement is much more objective than a phone survey.

Publicly available pricing information should also be included, and would provide a more nuanced and accurate picture of how inflation is evolving in the broader economy.


Quality Adjusted CPI Saved the Federal Government at least $150 Billion from 1998-2012

Cost-of-living adjustments increase entitlement spending automatically every year. Most COLA’s use all or part of the CPI to calculate inflation. The US further embeds the CPI in the system by indexing a growing portion of its government debt via TIPs. Even welfare benefits like food stamps use applicable indices within the all-items CPI to calculate COLA.

In total almost $3 trillion of federal yearly liability is subject to automatic annual CPI-based increases. This calculation includes:

  • All yearly means-tested welfare benefits subject to a COLA (e.g. SNAP, NSLP, etc…)
  • All yearly social security spending (e.g. SSI, OASI, DI, everything…)
  • All outstanding TIPs balance (every year the principal of an outstanding TIP is adjusted up/down by the inflation rate)
sources: http://budget.house.gov/uploadedfiles/rectortestimony04172012.pdf http://www.ssa.gov/policy/docs/statcomps/supplement/index.html http://www.gpo.gov/fdsys/pkg/ERP-2012/xls/ERP-2012-table87.xls
sources: http://budget.house.gov/uploadedfiles/rectortestimony04172012.pdf

This time series represents the majority of yearly federal obligations that are subject to inflation-based COLA increases. We can thus attempt to calculate how much the government saved each year through methodological changes to the CPI. By the government’s own reckoning:

[The] improvements made by the BLS have reduced the measured
increase in the CPI… The combined effect of the changes made through 1998 has been to lower the CPI inflation rate by 0.44 percentage point per year. Changes to be implemented in 1999 and 2000 will lower CPI inflation by a further 0.20 and 0.04 percentage point per year.

– Economic Report of the President Feb 1999 pg 93

Thats a total of 0.68% a year from 2000 onward. While this might not sound like a lot, given the immense sums of money the government owes the public, this adds up to billions of dollars in savings:

applies 0.0068 deflation rate to total amount calculated above
applies deflation rate implied from report (0.0044 in 1998 up to 0.0068 in 2000) to total obligations calculated above

That adds up to a total savings of $150,147,988,800. Again, this is basing our deflation rate at 0.68% from the economic report referenced above. There is considerable evidence, however, that the real effect of quality adjustments on the CPI is much higher.

Using the more realistic divergence of 1-2% we saw from the BPP data puts the total savings at the $200-$400 billion range. 

In the investigation of any crime, it is important to find motive:

  • When a seemingly trivial change to a statistical index can potentially deprive taxpayers of hundreds of billions of dollars 
  • When an agency keeps raw data hidden from outside inspection (BLS deems raw pricing data as confidential and thus exempt from FOIA)
  • When a government cannot make the unpopular decisions necessary to reign in entitlement spending

Then you are in a time where an executive branch might take actions into its own hands in the name of efficiency. Hiding in econometric obscurity, in an area of research so boring no economist would dare tread, did the government knowingly encourage the adoption of a dubious economic theory that would likely bias inflation downward? If so, it was a good bet.

Estimating the Effect of Hedonic Quality Adjustments on the Consumer Price Index

Having an informed debate over Hedonic Quality Adjustments is difficult due to the lack of comparable consumer price indices. A few exist, however, and today we will look at an index compiled by PriceStats, an off-shoot of MIT’s Billion Prices Project, which scrapes the internet for prices and compiles a daily index that aims to track inflation in real-time.

The time series eschews hedonic and seasonal adjustments and relies on sampling over 5 million products to produce a very different look at inflation (CPI included for comparison):


Since starting calculation of the index in mid-2008, PriceStats inflation series has remained consistently above the CPI as reported by the BLS. Considering the differences in methodology this provides an estimate to how much Hedonic Quality Adjustments have been used to understate the head-line CPI figures. Currently, the CPI uses quality adjustments on over 32% of the items used in its calculation. 

Annual inflation figures show a similar story, occasionally showing divergences greater than 1.5% in the two measures of annual inflation:


In addition to being used as a benchmark for policymaker’s worldwide, CPI’s influence a myriad of payments:

  • calculating cost-of-living adjustments to social security and federal retirement programs,
  • calculating payments on over $700 billion worth of inflation protected securities (TIPs)
  • determining pay-bands in public and private entities
  • cost of living adjustments to collective bargaining agreements
  • determining IRS tax brackets and numerous tax-related levels (exemptions, for example)
  • Basic CPI indices feed into the PCE price index which is the preferred measure of inflation by the FOMC

Overall, CPI’s are used to index payments on over $10 trillion in liabilities. Any underestimation of  CPI benefits the federal government at the expense of the taxpayer and amounts to a backdoor default on its financial obligations. Please understand this could be deliberate or the result of dubious econometric methods. Using a bad model is like using a random number generator as a compass. 

To get an idea of the sums of money involved, for every 1% of CPI underestimation the federal government saves $8.4 billion on social security payments alone.  The lack of transparency regarding quality adjustments (and their perceived complexity) provides a perfect smoke-screen for covert debt management through CPI under-reporting.

There is no reason for this to be the case. Hedonic Quality Adjustments are computed via linear regressions, one of the least complicated models in statistics. Any statistician provided with the data could verify the BLS’s regressions using an off-the-shelf statistical computing package like R. Release the data, plain and simple; any other response smacks of corruption in the name of regression or an attempt to justify one’s existence.

Manipulating the Consumer Price Index: Hedonic Quality Adjustments

Have you heard the one about CPI?

Suppose that a TV manufacturer retires a product and replaces it with a newer, better, and much more expensive one. If the new TV costs 5 times more than the old one, how can we gently massage the price of the old TV to make it look like the price fell? By using the dark arts of econometrics, my son!

If you believe the public comments made by the world’s central bankers, the prices that consumers pay for items are not rising fast enough; in some places like Europe they worry that prices might actually fall (a tragedy for the possessing classes, as their manic one-way long bets might not work then). Central bankers are terrified of this outcome. Setting aside for a second the apparent insanity of this logic for your average consumer, who experiences price rises on a near continuous basis, let’s examine in detail one of the gauges economists use for measuring prices: the Consumer Price Index (CPI).

Ostensibly, the CPI is a linear combination of the “prices” of things/stuff consumers could actually purchase weighted by a percentage that the “ideal consumer” spends on any particular stuff/thing in his “ideal” basket. The main problem here is that the “prices” used are not the prices a consumer would actually pay; instead the real price for an item is scaled by what the BLS calls a “Hedonic Quality Adjustment (HQA)”. The HQA was designed to solve a real world problem economists face: the market keeps pumping out new and better devices. In practice the HQA is used to artificially depress the prices used in the calculation of the CPI.

Intuitively, the HQA scales prices by their “perceived” quality. We’re not talking about human perception here, but that of a kitchen sink regression model created by BLS economists. Essentially it throws every quality an item might possess into a linear model and performs a regression of these qualities against the prices found in the market for a given product. The prices that feed into the CPI can be intuitively modeled as:


This means that as far as the CPI is concerned, prices can “decrease” for three reasons:

  • The price actually decreases, holding quality constant
  • The “quality” as measured by the Hedonic Quality Regression (HQR) could go up, holding price constant
  • The “quality” goes up by more than prices go up (<<<<<< WE’RE HERE RIGHT NOW)

In a time of rapid technological development, the quality as measured by HQR will increase by orders of magnitude more than prices. Consider Moore’s Law, which correctly postulated that the number of transistors on computer chips would double every two years; prices can’t possibly keep up with that kind of quality increase (save for hyperinflation, more on that later).

The BLS neatly illustrates this effect with an example from their website (emphasis is mine):


Item A is a television that is no longer available and it has been replaced by a new television, Item B. The characteristics in bold differ between the two TVs. There is a large degree of quality change and there is a very large (400%) difference in the prices of these TVs. Rather than use the 400 percent increase in price between Item A and Item B, the quality adjusted rate of price change is measuredby the ratio of the price of Item B in the current period ($1,250.00) over an estimated price of Item B in the previous period – Item B’.

Here is an example of a hedonic regression model (including coefficients) for televisions.


This is just an OLS linear regression model. The dependent variable is the natural log of prices for televisions, the explanatory variables and their coefficients are listed in the table below (most are dummy variables)


Where PB,t+s-1 is the quality adjusted price, PA,t+s-1 is the price of Item A in the previous period, and is the constant e [SIC], the inverse of the natural logarithm, exponentiated by the difference of the summations of the ßs for the set of characteristics that differ between items A and B. The exponentiation step is done to transform the coefficients from the semi log form to a linear form before adjusting the price.

To put it another way, the HQR extrapolates a price for the new TV using the Hedonic Quality model estimated from the population of old TV’s

To derive the estimated price of Item B’, we use the following equation:

For our television example, [the equation above] looks like this:


When this quality adjustment is applied, the ratio of price change looks like this:


The resulting price change is -7.1 percent after the quality adjustment is applied.

Oh good! You see, my neighbor, John Q., thought that prices were going up and was about to riot in the streets because he couldn’t buy anything now. How relieved he was to live next to an economist and mathematician; I merely explained that even though he couldn’t afford the new TV (or anything else) it was actually less expensive once quality was taken into account. Boy was his face red. He went home and explained it to his wife and kids and they laughed and laughed about their mistake.

Few modern people would consider progress to be a bad thing. Quality improvements should be celebrated and technological change embraced. Yet when a policymaker says that she wants inflation to pick up and trots out the CPI as evidence, she doesn’t care whether that comes about from actual price inflation or quality decreases. Given the accelerating pace of technological improvements, it’s hard to imagine an outcome besides hyperinflation that will satisfy central bankers and their slavish dependence on indicators which have been so far abstracted from reality as to have little actionable value.

Alternatively, causing a complete economic meltdown by manipulating the price of money and inflating the mother of all bubbles will probably slow down technological development, so either way, well played.

Since economists are largely concerned with “real” prices (actual prices scaled by inflation as measured by the CPI), any error in the calculation of real prices introduces a bias that propagates to every corner of economic thought. This is a central flaw in economics that largely explains the gap between actual human experiences (“Wow! Things are expensive!”) with central bankers gambling our collective future on fighting deflation.

More than likely the deflation is used as cover for the agency problem faced by central bankers every day. Most market practitioners know we are in a classic debt-fueled bubble initiated by wildly loose monetary policy – central bankers included. Given that the public will rightfully blame policymakers when the bubble bursts, no central banker wants to run the risk that it pops on their watch. That would make them look stupid, and might endanger their future lives as highly paid consultants. In that context printing endless supplies of money makes perfect sense.