Asymmetric Delusions and Pragmatism

Sometimes I joke that to conservatives, the solution to every problem is to cut taxes; to liberals, the solution to every problem is to eat local.  Of course, purported panaceas on both the left and right are snake oil, even if peddled by true believers.

So, the other day I picked on my own tribe: foodies.  To be frank, I have a love-hate relationship with the movement.  It has thin and oftentimes paranoid underpinnings.  But while a lot of things advocated by the movement are illogical or scientifically baseless (like the dangers of GMOs), the movement also strikes me as mostly harmless, and sometimes even beneficial.  There are some important and disastrous exceptions, like the movement to kill golden rice, which could save the lives of millions and save millions more from blindness.

But for the most part, the movement to "eat local," and all it's offspring, doesn't strike me as particularly harmful.   Here in Hawai'i there are delusional ideas that we ought to stop importing food and go back to the traditional ways of living off taro.  Obviously this isn't going to happen.  Even though they banned GMOs on some of our islands, exceptions were made for key crops actually grown.  Still, the movement does seem to be strong enough to help protect cultural heritage, an important public good.  It also cultivates a food culture that breeds great restaurants and fresh local produce.  After all, wasn't the whole movement inspired by my former Berkeley neighbor, restaurant extraordinaire, Alice Waters? (I never met her, but lived two doors down, in an in-law studio during grad school)  The movement isn't going to save the planet or feed the world, but it sure makes my privileged little world a lot nicer and a little healthier.

What about delusions on the right?  Front and center would be climate change denialism.  Not far behind would be anti-Keynesianism, or the austerity movement.  As Paul Krugman reminds us every day, quite persuasively in my view, these delusions have hardly been harmless.  And in contrast to foodies, the radical right has a whole lot of power, controlling both houses of Congress and rich backing by Wall Street, the Koch brothers and friends.

So, while reasoning and herd behavior on both extremes seems equally delusional at times, the delusional right strikes me as more destructive and much more powerful.

But then, speaking honestly, I identify more with the left than the right.  So am I being too soft on the lefties?  I don't think so, but feel free to weigh in. I think the lefty culture tends to be a bit more introspective by nature than the right.  The right doesn't tend to tease their own quite like I did the other day, not without swift excommunication.  And I think lefties, by their nature, are a little bit more susceptible to evidence and persuasion.  They also, by nature, possess visceral independence and eschew the party line.  Maybe that's what I was doing the other day.

Given the real political asymmetry here, I do feel a little guilty for picking on foodies.  But not too much.  I don't really want to go out and do a lot of research on the topic of food waste to prove how silly this stuff is, or to prove that while GMOs can have some undesirable side effects they aren't Frankenfoods. For the most part this is a fight that isn't worth fighting.  I figure it's better to focus on the things that really do matter.

But I figured it was worth a blog post, because I think we all do better when we eschew our innate tendency toward tribalism, try to figure out what's really going on, and find the most pragmatic solutions to real problems.

Food Waste Delusions

A couple months ago the New York Times convened a conference "Food for Tomorrow: Farm Better. Eat Better. Feed the World."  Keynotes predictably included Mark Bittman and Michael Pollan.  It featured many food movement activists, famous chefs, and a whole lot of journalists. Folks talked about how we need to farm more sustainably, waste less food, eat more healthfully and get policies in place that stop subsidizing unhealthy food and instead subsidize healthy food like broccoli.

Sounds good, yes? If you're reading this, I gather you're familiar with the usual refrain of the food movement.  They rail against GMOs, large farms, processed foods, horrid conditions in confined livestock operations, and so on.  They rally in favor of small local farms who grow food organically, free-range antibiotic free livestock, diversified farms, etc.  These are yuppies who, like me, like to shop at Whole Foods and frequent farmers' markets.  

This has been a remarkably successful movement.  I love how easy it has become to find healthy good eats, bread with whole grains and less sugar, and the incredible variety and quality of fresh herbs, fruits, vegetables and meat.  Whole Paycheck Foods Market has proliferated and profited wildly.  Even Walmart is getting into the organic business, putting some competitive pressure on Whole Foods. (Shhhh! --organic isn't necessarily what people might think it is.)

This is all great stuff for rich people like us. And, of course, profits.  It's good for Bittman's and Pollan's book sales and speaking engagements.  But is any of this really helping to change the way food is produced and consumed by the world's 99%?  Is it making the world greener or more sustainable?  Will any of it help to feed the world in the face of climate change?

Um, no.  

Sadly, there were few experts in attendance that could shed scientific or pragmatic light on the issues.  And not a single economist or true policy wonk in sight. Come on guys, couldn't you have at least invited Ezra Klein or Brad Plummer?  These foodie journalists at least have some sense of incentives and policy. Better, of course, would be to have some real agricultural economists who actually know something about large-scale food production and policies around the world. Yeah, I know: BORING!

About agricultural polices: there are a lot of really bad ones, and replacing them with good policies might help.  But a lot less than you might think from listening to foodies.  And, um, we do subsidize broccoli and other vegetables, fruits, and nuts.  Just look at the water projects in the West. 

Let me briefly take on one issue du jour: food waste.  We throw away a heck of a lot of food in this country, even more than in other developed countries.  Why?  I'd argue that it's because food is incredibly cheap in this country relative to our incomes.  We are the world's bread basket.  No place can match California productivity in fruit, vegetables and nuts.  And no place can match the Midwest's productivity in grains and legumes.  All of this comes from remarkable coincidence of climate, geography and soils, combined with sophisticated technology and gigantic (subsidized) canal and irrigation systems in the West.  

Oh, we're fairly rich too.  

Put these two things together and, despite our waste, we consume more while spending less on food than any other country.  Isn't that a good thing?  Europeans presumably waste (a little) less because food is more scarce there, so people are more careful and less picky about what they eat. Maybe it isn't a coincidence that they're skinnier, too.

What to do? 

First, it's important to realize that there are benefits to food waste.  It basically means we get to eat very high quality food and can almost always find what we want where and when we want it.  That quality and convenience comes at a cost of waste.  That's what people are willing to pay for.  

If anything, the foodism probably accentuates preference for high quality, which in turn probably increases waste.  The food I see Mark Bittman prepare is absolutely lovely, and that's what I want.  Don't you?

Second, let's suppose we implemented a policy that would somehow eliminate a large portion of the waste.  What would happen?  Well, this would increase the supply of food even more.  And sinse we have so much already, and demand for food is very inelastic, prices would fall even lower than they are already.  And the temptation to substitute toward higher quality--and thus waste more food--would be greater still.  

Could the right policies help?  Well, maybe.  A little. The important thing here is to have a goal besides simply eliminating waste.  Waste itself isn't problem. It's not an externality like pollution.  That goal might be providing food for homeless or low income families.  Modest incentive payments plus tax breaks might entice more restaurants, grocery stores and others to give food that might be thrown out to people would benefit from it.  This kind of thing happens already and it probably could be done on a larger scale. Even so, we're still going to have a lot of waste, and that's not all bad. 

What about correcting the bad policies already in place?  Well, water projects in the West are mainly sunk costs.  That happened a long time ago, and water rights, as twisted as they may be, are more or less cemented in the complex legal history.   Today, traditional commodity program support mostly takes the form of subsidized crop insurance, which is likely causing some problems.  The biggest distortions could likely be corrected with simple, thoughtful policy tweaks, like charging higher insurance premiums to farmers who plant corn after corn instead of corn after soybeans.  But mostly it just hands cash (unjustly, perhaps) to farmers and landowners.  The odds that politicians will stop handing cash to farmers is about as likely as Senator James Inhofe embracing a huge carbon tax.  Not gonna happen.

But don't worry too much.  If food really does get scarce and prices spike, waste will diminish, because poorer hungry people will be less picky about what they eat.

Sorry for being so hard on the foodies.  While hearts and forks are in the right places, obviously I think most everything they say and write is naive.  Still, I think the movement might actually do some good.  I like to see people interested in food and paying more attention to agriculture.  Of course I like all the good eats.  And I think there are some almost reasonable things being said about what's healthy and not (sugar and too much red meat are bad), even if what's healthy has little to do with any coherent strategy for improving environmental quality or feeding the world.  

But perhaps the way to change things is to first get everyones' attention, and I think foodies are doing that better than I ever could.

The Hottest Year on Record, But Not in the Corn Belt

Here's Justin Gillis in his usual fine reporting of climate issues, and the map below from NOAA, via the New York Times.

Note the "warming hole" over the Eastern U.S., especially the upper Midwest, the all important corn belt region.  We had a bumper crop this year, and that's because while most of the world was remarkably warm, the corn belt was remarkably cool, especially in summer.

Should we expect the good fortune to continue?  I honestly don't know...

Searching for critical thresholds in temperature effects

Update 2: Okay, I think it's fixed.

Update: I just realized the code posted badly.  I don't know why.  It looks good in the cross-post at G-FEED.  I'll try to fix.

If google scholar is any guide, my 2009 paper with Wolfram Schlenker on the nonlinear effects of temperature on crop outcomes has had more impact than anything else I've been involved with.

A funny thing about that paper: Many reference it, and often claim that they are using techniques that follow that paper.  But in the end, as far as I can tell, very few seem to actually have read through the finer details of that paper or try to implement the techniques in other settings.  Granted, people have done similar things that seem inspired by that paper, but not quite the same.  Either our explication was too ambiguous or people don't have the patience to fully carry out the technique, so they take shortcuts.  Here I'm going to try to make it easier for folks to do the real thing.

So, how does one go about estimating the relationship plotted in the graph above?

Here's the essential idea:  averaging temperatures over time or space can dilute or obscure the effect of extremes.  Still, we need to aggregate, because outcomes are not measured continuously over time and space.  In agriculture, we have annual yields at the county or larger geographic level.  So, there are two essential pieces: (1) estimating the full distribution of temperatures of exposure (crops, people, or whatever) and (2) fitting a curve through the whole distribution.

The first step involves constructing the distribution of weather. This was most of the hard work in that paper, but it has since become easier, in part because finely gridded daily weather is available (see PRISM) and in part because Wolfram has made some STATA code available.  Here I'm going to supplement Wolfram's code with a little bit of R code.  Maybe the other G-FEEDers can chime in and explain how to do this stuff more easily.

First step:  find some daily, gridded weather data.  The finer scale the better.  But keep in mind that data errors can cause serious attenuation bias.  For the lower 48 since 1981, the PRISM data above is very good.  Otherwise, you might have to do your own interpolation between weather stations.  If you do this, you'll want to take some care in dealing with moving weather stations, elevation and microclimatic variations.  Even better, cross-validate interpolation techniques by leaving one weather station out at a time and seeing how well the method works. Knowing the size of the measurement error can also help correcting bias.  Almost no one does this, probably because it's very time consuming... Measurement error in weather data creates very serious problems (see here and here)

Second step:  estimate the distribution of temperatures over time and space from the gridded daily weather.  There are a few ways of doing this.  We've typically fit a sine curve between the minimum and maximum temperatures to approximate the time at each degree in each day in each grid, and then aggregate over grids in a county and over all days in the growing season.  Here are a couple R functions to help you do this:

# This function estimates time (in days) when temperature is
# between t0 and t1 using sine curve interpolation.  tMin and
# tMax are vectors of day minimum and maximum temperatures over
# range of interest.  The sum of time in the interval is returned.
# noGrids is number of grids in area aggregated, each of which 
# should have exactly the same number of days in tMin and tMax
 
days.in.range <- span=""> function( t0, t1 , tMin, tMax, noGrids )  {
  n   =  length(tMin)
  t0  =  rep(t0, n)
  t1  =  rep(t1, n)
  t0[t0 < tMin]   =  tMin[t0 < tMin]
  t1[t1 > tMax]  =  tMax[t1 > tMax]
  u  =  function(z, ind) (z[ind] - tMin[ind])/(tMax[ind] - tMin[ind])  
  outside  =  t0 > tMax | t1 < tMin
  inside  =  !outside
  time.at.range  =  ( 2/pi )*( asin(u(t1,inside)) - asin(u(t0,inside)) ) 
  return( sum(time.at.range)/noGrids ) 
}
 
# This function calculates all 1-degree temperature intervals for 
# a given row (fips-year combination).  Note that nested objects
# must be defined in the outer environment.
aFipsYear  =  function(z){
  afips   = Trows$fips[z]
  ayear    = Trows$year[z]
  tempDat  = w[ w$fips == afips & w$year==ayear, ]
  Tvect = c()
  for ( k in 1:nT ) Tvect[k] = days.in.range(
              t0   = T[k]-0.5, 
              t1   = T[k]+0.5, 
              tMin = tempDat$tMin, 
              tMax = tempDat$tMax,
              noGrids = length( unique(tempDat$gridNumber) )
              )
  Tvect
}

The first function estimates time in a temperature interval using the sine curve method.  The second function calls the first function, looping through a bunch of 1-degree temperature intervals, defined outside the function.  A nice thing about R is that you can be sloppy and write functions like this that use objects defined outside of the environment. A nice thing about writing the function this way is that it's amenable to easy parallel processing (look up 'foreach' and 'doParallel' packages).

Here are the objects defined outside the second function:

w       # weather data that includes a "fips" county ID, "gridNumber", "tMin" and "tMax".
        #   rows of w span all days, fips, years and grids being aggregated
 
tempDat #  pulls the particular fips/year of w being aggregated.
Trows   # = expand.grid( fips.index, year.index ), rows span the aggregated data set
T       # a vector of integer temperatures.  I'm approximating the distribution with 
        #   the time in each degree in the index T

To build a dataset call the second function above for each fips-year in Trows and rbind the results.

Third step:  To estimate a smooth function through the whole distribution of temperatures, you simply need to choose your functional form, linearize it, and then cross-multiply the design matrix with the temperature distribution.  For example, suppose you want to fit a cubic polynomial and your temperature bins that run from from 0 to 45 C.  The design matrix would be:

D = [    0          0          0   
            1          1           1
            2          4           8
             ...
           45     2025    91125]

These days, you might want to do something fancier than a basic polynomial, say a spline. It's up to you.  I really like restricted cubic splines, although they can over smooth around sharp kinks, which we may have in this case. We have found piecewise linear works best for predicting out of sample (hence all of our references to degree days).  If you want something really flexible, just make D and identity matrix, which effectively becomes a dummy variable for each temperature bin (the step function in the figure).  Whatever you choose, you will have a (T x K) design matrix, with K being the number of parameters in your functional form and T=46 (in this case) temperature bins. 

To get your covariates for your regression, simply cross multiply D by your frequency distribution.  Here's a simple example with restricted cubic splines:

library(Hmisc)
DMat = rcspline.eval(0:45)
XMat = as.matrix(TemperatureData[,3:48])%*%DMat
fit  = lm(yield~XMat, data=regData)
summary(fit)

Note that regData has the crop outcomes.  Also note that we generally include other covariates, like total precipitation during the season,  county fixed effects, time trends, etc.  All of that is pretty standard.  I'm leaving that out to focus on the nonlinear temperature bit. 

Anyway, I think this is a cool and fairly simple technique, even if some of the data management can be cumbersome.  I hope more people use it instead of just fitting to shares of days with each maximum or mean temperature, which is what most people following our work tend to do.  

In the end, all of this detail probably doesn't make a huge difference for predictions.  But it can make estimates more precise, and confidence intervals stronger.  And I think that precision also helps in pinning down mechanisms.  For example, I think this precision helped us to figure out that VPD and associated drought was a key factor underlying observed effects of extreme heat.