What do commodity prices reflect about broader economic expectations and risks?

Gold prices are pushing repeatedly record highs.  But there is no inflation on the horizon; indeed, deflation is a far greater risk right now.  This can be gleaned by the fact that prices of other commodities, ranging from oil to minerals to agricultural commodities, are trending down, by record-low long-term treasury rates and low rates on inflation-indexed bonds, and by high unemployment and stagnant wages. The trend in CPI data is down too.

What gives?  Jim Hamilton nails it:

There's a common thread to all the above figures, and it's not fears about inflation. Instead it's worries about the level of real economic activity, showing up in a flight to safety. U.S. Treasuries remain the instrument of choice for investors who think nothing looks safe.

But with the long-run fiscal challenges facing the United States, are 10-year Treasuries really the safest place to put your money? The yellow metal seems to be one way some people are hedging that bet.

The broader economy has a lot to fear from fear itself, especially given the Fed seems either unwilling or unable to do anything to compensate for flight to safety while still stuck at the zero lower bound.  Plus, further fiscal stimulus is all but off the table given current fears feeding broad governmental flight to austerity, which only seems to make matters worse. (Also see: here--this is a long, tired debate)

Anyway, aside from the macro wars, I find it interesting that commodity price fluctuations appear to generally reflect a lot about the broader macro economy and perceived risks across a wide class of assets.

Consider:

1) During the 70s and 80s when fear of limited oils supplies ruled the day, commodity prices were high, and energy commodities in particular fluctuated negatively with stock market returns and positively with inflation.  This was a rare example of real-business cycle phenomenon stemming from uncertainty about supply of key natural resource commodities.

2) During the naughts, we saw a reversal of the earlier patterns, with commodity prices fluctuating positively with growth and stock market returns.  It appears this was because the larger uncertainties were about broader economic growth and demand, particularly from Asia; and after the recession hit, uncertainties about the size and timing of world economic recovery.

3) Now that gold is breaking away from the other commodities, I see signs of the same ominous fears that Jim Hamilton sees. Thus, today commodity prices reflect expectations of meager to declining growth mixed with sublime fears that governments will be unable to fullfil debt obligations, or perhaps fears of broader institutional decline.

Given commodity prices reflect so much about expectations about broader economy, it seems there should be a nice way to systematically evaluate expectations from commodity price patterns. But there really isn't much in the literature that I'm aware of.  I'm thinking in the back of my mind we need to somehow merge analysis like that of Kilian (ungated version here), which disentangles supply and demand factors in oil price determination, with asset pricing theory.

I should note that fears reflected in gold prices, while real, are still pretty subdued.  One way to try to put the size of the hedge in perspective is to approximate the total value of gold worldwide and compare that to, say, the capitalized value of the stock market, oil reserves, and bond issues.  Currently the world produces about 75 million ounces of gold per year.  With gold trading at $1220 an ounce, that's a world production value of about $91.5 billion.  That's pretty tiny in comparison to, well, just about anything.  It's hard to say what the worldwide stock of gold is, but it's hard for me to imagine how it could be more than 20 times the flow, which would put the value of the worldwide stock at something less than $2 trillion.  That's a lot of money, but it's still pretty trivial share of the world's assets.  "Proven" oil reserves equal about 1.2 trillion barrels (a value of some $90 trillion).  The global capitalized value of stocks was about $50 billion trillion in 2007. I expect real estate value exceeds both these figures by a large margin.  Our U.S. national debt is some $13-14 trillion, which also sounds scary but really isn't in comparison to our potential annual output and value of all our assets.  

Note that a hedge on gold is just that--an iddy bit of insurance in case things get much worse than expected.  It's a really bad basket for all your eggs.  The overarching expectation is that bets on gold will lose, especially if investing at today's high prices.