Follow The Money: How to interpret “peer review” science.

I regularly hear or read arguments from agencies compromising our natural heritage that such and such studies support their management decisions.  However, often the agencies overlook or ignore contrary science that does not support the policy or management decision.

To give them a break, the average district ranger or even specialists like wildlife biologists, fire managers, and others often do not have time to keep up with the latest science. So, recognize that they may not really know the “best” science.

Yet, the public, and often the media, naively accepts without question the assertions of public agencies as “unbiased” observers. One often hears agencies defend their statements and policies by suggesting if everyone is angry with their positions, they must be “doing something right.”

Well not everything is splitting the baby. The Earth is round, not flat or half way flat.  Gravity exists whether you believe it or not—just try jumping off a cliff. There are some things that are more accurate than others.

Both proponents and opponents of various public policies rely upon scientific studies to give credibility to their positions and garner confidence in their assertions. So how does one determine whose science is reliable? Most scientists do not purposefully distort their findings, but how they interpret their findings and what they emphasize can influence how people understand the outcomes.

There are several ways that I decide the relative veracity of scientific research and whether to grant authority to agency representatives—I follow the money.

The Upton Sinclair quote that “It is difficult to get a man to understand something when his salary depends upon his not understanding it” often is a good starting point for determining the accuracy of statements.

I know few foresters, for instance, who are opposed to logging. If you are a forester, your job depends on cutting trees.

The first thing I do is look at the occupation and affiliation of the spokesperson. Obviously if you are reading a study about the safety of smoking cigarettes and the authors work for tobacco companies, this would raise a yellow flag of caution. But it is not always as obvious that there are conflicts.

For instance, one of the ideas we hear surrounding livestock grazing is that grazing can prevent large wildfires by targeted removal of the fuel—grasses—that sustains fires.

There have been several studies that purport to show that “targeted” grazing can halt large fires. So, the first thing I do is look at who the authors are and where they are employed. The studies that suggest that livestock grazing—at least the ones I’m aware of—are all done by people in range departments.

Why is this important? Because if you are a professor or graduate student in a range program, your entire budget and survival as a professor and department depends on maintaining livestock grazing on public lands. Hence you have a vested interested in promoting livestock “benefits” whether real or imagined, and minimizing any known negative effects.

Next I look at the journal where the research was published. Not all journals are equal. Some are much more discriminating in the papers accepted for publication. Some journals also have unspoken biases. For instance, the Journal of Rangeland Ecology and Management published by the Society for Range Management is biased towards promoting livestock production.

You will find few papers in this journal that recommends removal of livestock as the best management option, even when the authors may document significant resource damage from livestock. They almost always recommend “proper grazing management” as the solution to problems, whether proper management can work under field conditions.

Beyond the journal publication, one then must look at the funding source for the study. If you are doing range studies in the western United States, most of your funding will be coming from either livestock organizations, and/or federal/state money appropriated to demonstrate why livestock grazing is a beneficial use of the landscape.

In all instances of the above examples, money and jobs dictates the perspective of the individual, and woe to the person who steps over the line and does not promote the accepted policy positions.

So, in the case of targeted grazing, you must read crucially the study methods, and conclusions. Most scientists have integrity—even those who are advocates of grazing, logging, oil extraction or whatever. They do not out-right lie or distort their findings.

What they do instead is restrict the kinds of questions they ask, how they set up their experimental design, how they interpret their findings and what they propose for solutions.

For instance, I studied wildlife biology in college. There was not one professor of mine that questioned or even raised the question whether hunting wildlife was appropriate and perhaps harmful to the long-term survival of the animals. (And yes, there is evidence that even “regulated” hunting can negatively impact wildlife).

We simply never discussed this idea because almost all wildlife biology professors get a substantial amount of their research funding from Fish and Game agencies.

As an example of the unexamined assumptions, one of the papers widely referenced by the BLM in its management plans to save sage grouse champions “targeted grazing” to reduce western range fires. The original study was based on the grazing of several small plots where the cattle were confined by fences and herding. While the grazed areas did have less fuels, the applicability of this management of confinement to large public lands allotments is questionable. The cost of such confinement would be prohibitive.

While the research might show that “targeted” grazing could perhaps reduce some wildfires, the practical application of this approach to wide-open public lands allotments is questionable.

Beyond the costs and the lack of landscape scale application, the study relied on “models” of fire behavior to conclude that grazing would reduce wildfires.

Models are notoriously imprecise. As the saying goes, what goes into the model affects what comes out.

One of the factors in their model was limiting the wind speed in the fire models.

Why is this important? Because nearly all large wildfires are driven by high winds. Under less than high winds, wildfires do not spread rapidly and are easy to control—whether grazed or not.

Models are better than nothing, but they are no substitute for empirical data. In other words, direct observation of how real wildfires interact with grazed landscapes.

Finally, in at least one of these studies, the authors admitted in the very last paragraph that their findings only applied to wildfires burning under low to moderate weather conditions.

One had to read the entire paper to find this one line which is a dead giveaway that targeted grazing is not likely to significantly influence the large wildfires burning across the West. These large wildfires all burn under extreme fire weather conditions.

The same caution applies to other science as well. Nearly all the science supporting thinning/logging to reduce high severity wildfire is done by forestry schools and/or researchers who work for federal or state forestry agencies like the US Forest Service.

For instance, the Oregon State University Forestry School gets 10% of its funding from a tax on logging, which alone would be enough incentive for the department to have a favorable perspective on logging issues, not to mention that timber industry dollars also directly fund some of the department’s research.

No one wants to bite the hand that feeds them.

I hasten to add that this does not mean all research done by forestry schools, nor are all professors in such departments minions for industry. Nevertheless, there are often unquestioned assumptions that permeates the science. A reasonable person would exercise caution in accepting all “peer reviewed” science as equally valid.

Print Friendly