By accident, I have been drawn into a discussion surrounding a fairly innocent mistake made by NASA’s GISS in their October global temperature numbers. It began for me when I compared the October GISS and UAH satellite numbers for October, and saw an incredible diversion. For years these two data sets have shown a growing gap, but by tiny increments. But in October they really went in opposite directions. I used this occasion to call on the climate community to make a legitimate effort at validating and reconciling the GISS and satellite data sets.
Within a day of my making this post, several folks started noticing some oddities in the GISS October data, and eventually the hypothesis emerged that the high number was the result of reusing September numbers for certain locations in the October data set. Oh, OK. A fairly innocent and probably understandable mistake, and far more minor than the more systematic error a similar group of skeptics, (particularly Steve McIntyre, the man whose name the GISS cannot speak) found in the GISS data set a while back. The only amazing thing to me was not the mistake, but the fact that there were laymen out there on their own time who figured out the error so quickly after the data release. I wish there were a team of folks following me around, fixing material errors in my analysis before I ran too far with it.
So Gavin Schmidt of NASA comes out a day or two later and says, yep, they screwed up. End of story, right? Except Dr. Schmidt chose his blog post about the error to lash out at skeptics. This is so utterly human — in the light of day, most will admit it is a bad idea to lash out at your detractors in the same instant you admit they caught you in an error (however minor). But it is such a human need to try to recover and sooth one’s own ego at exactly this same time. And thus we get Gavin Schmidt’s post on RealClimate.com, which I would like to highlight a bit below.
He begins with a couple of paragraphs on the error itself. I will skip these, but you are welcome to check them out at the original. Nothing about the error seems in any way outside the category of “mistakes happen.” Had the post ended with something like “Many thanks to the volunteers who so quickly helped us find this problem,” I would not even be posting. But, as you can guess, this is not how it ends.
It’s clearly true that the more eyes there are looking, the faster errors get noticed and fixed. The cottage industry that has sprung up to examine the daily sea ice numbers or the monthly analyses of surface and satellite temperatures, has certainly increased the number of eyes and that is generally for the good. Whether it’s a discovery of an odd shiftin the annual cycle in the UAH MSU-LT data, or this flub in the GHCN data, or the USHCN/GHCN merge issue last year, the extra attention has led to improvements in many products. Nothing of any consequence has changed in terms of our understanding of climate change, but a few more i’s have been dotted and t’s crossed.
Uh, OK, but it is a bit unfair to characterize the “cottage industry” looking over Hansen’s and Schmidt’s shoulders as only working out at the third decimal place. Skeptics have pointed out what they consider to be fundamental issues in some of their analytical approaches, including their methods for compensating statistically for biases and discontinuities in measurement data the GISS rolls up into a global temperature anomaly. A fairly large body of amateur and professional work exists questioning the NOAA and GISS methodologies which often result in manual adjustments to the raw data larger in magnitude than the underlying warming signal tyring to be measured. I personally think there is a good case to be made that the GISS approach is not sufficient to handle this low signal to noise data, and that the GISS has descended in to “see no evil, hear no evil” mode in ignoring the station survey approach being led by Anthony Watt. Just because Schmidt does not agree doesn’t mean that the cause of climate science is not being advanced.
The bottom line, as I pointed out in my original post, is that the GISS anomaly and the satellite-measured anomaly are steadily diverging. Given some of the inherent biases and problems of surface temperature measurement, and NASA’s commitment to space technology as well as its traditional GISS metric, its amazing to me that Schmidt and Hansen are effectively punting instead of doing any serious work to reconcile the two metrics. So it is not surprising that into this vacuum left by Schmidt rush others, including us lowly amateurs.
By the way, this is the second time in about a year when the GISS has admitted an error in their data set, but petulently refused to mention the name of the person who helped them find it.
But unlike in other fields of citizen-science (astronomy or phenology spring to mind), the motivation for the temperature observers is heavily weighted towards wanting to find something wrong. As we discussed last year, there is a strong yearning among some to want to wake up tomorrow and find that the globe hasn’t been warming, that the sea ice hasn’t melted, that the glaciers have not receded and that indeed, CO2is not a greenhouse gas. Thus when mistakes occur (and with science being a human endeavour, they always will) the exuberance of the response can be breathtaking – and quite telling.
I am going to make an admission here that Dr. Schmidt very clear thinks is evil: Yes, I want to wake up tomorrow to proof that the climate is not changing catastrophically. I desperately hope Schmidt is overestimating future anthropogenic global warming. Here is something to consider. Take two different positions:
- I hope global warming theory is correct and the world faces stark tradeoffs between environmental devastation and continued economic growth and modern prosperity
- I hope global warming theory is over-stated and that these tradeoffs are not as stark.
Which is more moral? Why do I have to apologize for being in camp #2? Why isn’t it equally “telling” that Dr. Schmidt apparently puts himself in camp #1.
Of course, we skeptics would say the same of Schmidt. As much as we like to find a cooler number, we believe he wants to find a warmer number. Right or wrong, most of us see a pattern in the fact that the GISS seems to constantly find ways to adjust the numbers to show a larger historic warming, but require a nudge from outsiders to recognize when their numbers are too high. The fairest way to put it is that one group expects to see lower numbers and so tends to put more scrutiny on the high numbers, and the other does the opposite.
Really, I don’t think that Dr. Schmidt is a very good student of the history of science when he argues that this is somehow unique to or an aberration in modern climate science. Science has often depended on rivalries to ensure that skepticism is applied to both positive and negative results of any experiment. From phlogistan to plate techtonics, from evolution to string theory, there is really nothing new in the dynamic he describes.
A few examples from the comments at Watt’s blog will suffice to give you a flavour of the conspiratorial thinking: “I believe they had two sets of data: One would be released if Republicans won, and another if Democrats won.”, “could this be a sneaky way to set up the BO presidency with an urgent need to regulate CO2?”, “There are a great many of us who will under no circumstance allow the oppression of government rule to pervade over our freedom—-PERIOD!!!!!!” (exclamation marks reduced enormously), “these people are blinded by their own bias”, “this sort of scientific fraud”, “Climate science on the warmer side has degenerated to competitive lying”, etc… (To be fair, there were people who made sensible comments as well).
Dr. Schmidt, I am a pretty smart person. I have lots of little diplomas on my wall with technical degrees from Ivy League universities. And you know what – I am sometimes blinded by my own biases. I consider myself a better thinker, a better scientist, and a better decision-maker because I recognize that fact. The only person who I would worry about being biased is the one who swears that he is not.
By the way, I thought the little game of mining the comments section of Internet blogs to discredit the proprietor went out of vogue years ago, or at least has been relegated to the more extreme political blogs like Kos or LGF. Do you really think I could not spend about 12 seconds poking around environmentally-oriented web sites and find stuff just as unfair, extreme, or poorly thought out?
The amount of simply made up stuff is also impressive – the GISS press release declaring the October the ‘warmest ever’? Imaginary (GISS only puts out press releases on the temperature analysis at the end of the year). The headlines trumpeting this result? Non-existent. One clearly sees the relief that finally the grand conspiracy has been rumbled, that the mainstream media will get it’s comeuppance, and that surely now, the powers that be will listen to those voices that had been crying in the wilderness.
I am not quite sure what he is referring to here. I will repeat what I wrote. I said “The media generally uses the GISS data, so expect stories in the next day or so trumpeting ‘Hottest October Ever.'” I leave it to readers to decide if they find my supposition unwarranted. However, I encourage the reader to consider the 556,000 Google results, many media stories, that come up in a search for the words “hottest month ever.” Also, while the GISS may not issue monthly press releases for this type of thing, the NOAA and British Met Office clearly do, and James Hansen has made many verbal statements of this sort in the past.
By the way, keep in mind that that Dr. Schmidt likes to play Clinton-like games with words. I recall one episode last year when he said that climate models did not use the temperature station data, so they cannot be tainted with any biases found in the stations. Literally true, I guess, because the the models use gridded cell data. However, this gridded cell data is built up, using a series of correction and smoothing algorithms that many find suspect, from the station data. Keep this in mind when parsing Dr. Schmidt.
Alas! none of this will come to pass. In this case, someone’s programming error will be fixed and nothing will change except for the reporting of a single month’s anomaly. No heads will roll, no congressional investigations will be launched, no politicians (with one possible exception) will take note. This will undoubtedly be disappointing to many, but they should comfort themselves with the thought that the chances of this error happening again has now been diminished. Which is good, right?
I’m narrowly fine with the outcome. Certainly no heads should roll over a minor data error. I’m not certain no one like Watt or McIntyre suggested such a thing. However, the GISS should be embarrassed that they have not addressed and been more open about the issues in their grid cell correction/smoothing algorithms, and really owe us an explanation why no one there is even trying to reconcile the growing differences with satellite data.
In contrast to this molehill, there is an excellent story about how the scientific community really deals with serious mismatches between theory, models and data. That piece concerns the ‘ocean cooling’ story that was all the rage a year or two ago. An initial analysisof a new data source (the Argo float network) had revealed a dramatic short term cooling of the oceans over only 3 years. The problem was that this didn’t match the sea level data, nor theoretical expectations. Nonetheless, the paper was published (somewhat undermining claims that the peer-review system is irretrievably biased) to great acclaim in sections of the blogosphere, and to more muted puzzlement elsewhere. With the community’s attention focused on this issue, it wasn’t however long before problemsturned up in the Argo floats themselves, but also in some of the other measurement devices – particularly XBTs. It took a couple of years for these things to fully work themselves out, but the most recent analysesshow far fewer of the artifacts that had plagued the ocean heat content analyses in the past. A classic example in fact, of science moving forward on the back of apparent mismatches. Unfortunately, the resolution ended up favoring the models over the initial data reports, and so the whole story is horribly disappointing to some.
OK, fine, I have no problem with this. However, and I am sure that Schmidt would deny this to his grave, but he is FAR more supportive of open inspection of measurement sources that disagree with his hypothesis (e.g. Argo, UAH) than he is willing to tolerate scrutiny of his methods. Heck, until last year, he wouldn’t even release most of his algorithms and code for his grid cell analysis that goes into the GISS metric, despite the fact he is a government employee and the work is paid for with public funds. If he is so confident, I would love to see him throw open the whole GISS measurement process to an outside audit. We would ask the UAH and RSS guys to do the same. Here is my prediction, and if I am wrong I will apologize to Dr. Schmidt, but I am almost positive that while the UAH folks would say yes, the GISS would say no. The result, as he says, would likely be telling.
Which brings me to my last point, the role of models. It is clear that many of the temperature watchers are doing so in order to show that the IPCC-class models are wrong in their projections. However, the direct approach of downloading those models, running them and looking for flaws is clearly either too onerous or too boring. Even downloading the output (from here or here) is eschewed in favour of firing off Freedom of Information Act requests for data already publicly available – very odd. For another example, despite a few comments about the lack of sufficient comments in the GISS ModelE code (a complaint I also often make), I am unaware of anyone actually independently finding any errors in the publicly available Feb 2004 version (and I know there are a few). Instead, the anti-model crowd focuses on the minor issues that crop up every now and again in real-time data processing hoping that, by proxy, they’ll find a problem with the models.
I say good luck to them. They’ll need it.
Red Alert! Red Alert! Up to this point, the article was just petulant and bombastic. But here, Schmidt becomes outright dangerous, suggesting a scientific process that is utterly without merit. But I want to take some time on this, so I will pull this out into a second post I will label part 2.
Typos: “phlogiston” and “tectonics”.
Warren,
Very interesting, thanks for the level headed analysis. I am looking forward to Part II.
Russ
When the Catholic Church wants to canonise someone they appoint a Devil’s Advocate whose job it is to point out flaws in the life of the person nominated for canonisation. In the case of climate change, which in negative effects or spurious efforts to combat them will cost trillions, most of the media and government scientific community only want to hear one side of the story. Hence the need for vigilant self-appointed and self-motivated sceptics.
Yes, very good post. Aside from the technical issues, just reading his posts I simply do not trust anything Gavin Schmidt says – I can barely find anything good to say about his posts. The general tone of his posts and responses is just awful, and immediately invalidates his supposed credentials. And he tries to bamboozle people with mathematical terms to make things sound more complex than they are – I’ve seen far too much of this to be taken in by it.
Your point about lay people catching errors reminds me of an idea I was throwing around about a central place like a wiki/blog hybrid where an abstract from a published (or even unpublished) work could be posted and comments made. But unlike a wiki where everything has to be merged into one document have the comments attached to the poster. It’s just that so many people comment on published works in their personal blogs that matching them up or remembering where a particular comment came from can be annoying, especially when search terms are too general for google to find.
the GISS anomaly and the satellite-measured anomaly are steadily diverging – um, no they aren’t.
Yes, I want to wake up tomorrow to proof that the climate is not changing catastrophically. I desperately hope Schmidt is overestimating future anthropogenic global warming. Here is something to consider. Take two different positions:
1. I hope global warming theory is correct and the world faces stark tradeoffs between environmental devastation and continued economic growth and modern prosperity
2. I hope global warming theory is over-stated and that these tradeoffs are not as stark.
I’m sure if you ran someone over in your car, you’d desperately hope to wake up to the news that they would make a full recovery. I’m sure we would all like to wake up to news that human activities had not in fact triggered an ongoing steep rise in global temperatures. What we wish for is quite irrelevant to the observations and our understanding of them, and your two ‘positions’ make no sense.
The media generally uses the GISS data, so expect stories in the next day or so trumpeting ‘Hottest October Ever. – can you give a link to any such story?
Jennifer,
I can appreciate your point of view, but you are just wrong. The warming we are measuring is well within the range of “natural variation”.
The only place that human activities has triggered an ongoing steep rise in global temperatures, is within the computer models.
This is the crux of the issue. We both see some warming. You say it is due to the burning of hydrocarbons, I say the majority of it is due to an ongoing warming since the end of the Little Ice Age. The data is turning in my favor, and that has caused those in the AGW research industry to increase the verbal assaults.
That is because the data won’t support the claims, and it in no way, calls for massive intervention in the use of energy.
If the models are right, we still have plenty of time to adjust to the change that is coming, regardless of what we do today. If they are wrong, we will waste resources on a non-problem, and leave the world poorer, with more people dying from preventable causes.
Nice article, and the subtext gets at a big problem I have with AGW. I’m not an expert, not even an amateur. I need to rely on others to collect, analyze and interpret the data for me. But as I look around, I find that my understanding of logic is constantly offended by the multiple layers of adjustment factors, assumptions and smoothing that goes on without any recognition that each level of estimating injects an increasing level of uncertainty in the result.
I find in the end that I simply don’t trust the AGW crowd to speak the truth as best they can determine it. Sure, the mistake that led to bad October numbers is understandable, even unavoidable in that mistakes will always happen. But I have no doubt that if the data showed unusual cooling instead of unusual warming, people like Schmidt would have caught it themselves. There is rampant confirmation bias at work that renders me unable to trust their claims. That would be quite a disaster if they end up being right.
Gavin’s response reeks of prior and deceptive knowledge.
Before the AGW promo industry does a memory hole, past GISS graphs should be scrutinized to discover if and how often they have done this.
I seriously doubt if this is the first time, and I will not be at all surprised to find out they have been massaging this stuff for years.
The unavoidable fact that they are using corrupt land temp data yet claiming accuracy levels in the .1o degree range leaves deliberate corruption as the logical conclusion to much of their work.
Russ R.: there is a big difference between “within the range of natural variation” and “due to natural causes”. If you live in New Orleans and your living room is flooded, you may be experiencing something that is within the range of natural variation, but that doesn’t rule out the possibility that you left the bath running. Sure, there have been higher temperatures on Earth in the past than there are now. That tells us nothing about what is causing the temperature to rise like it is doing currently.
“the majority of it is due to an ongoing warming”, you say about the current warming. So, it is warming because it is warming? Not a very good explanation I’m afraid.
“The data is turning in my favor” – how so?
“the majority of it is due to an ongoing warming”, you say about the current warming. So, it is warming because it is warming? Not a very good explanation I’m afraid.”
What’s the explanation for the millions of years of regular warming and cooling cycles before anthropomorphic CO2 could be a factor? Don’t know either? Oh well…
Jennifer,
If the temps and weather phenom. are within normal bounds, and have been, then the point is why deconstruct our civilization over the equivalent of a “Seinfeld” plot- a whole bunch of nothing?
Will Nitschke: I recommend you do two things. First, google ‘paleoclimatology’. Second, look up ‘anthropomorphic’ in a dictionary.
hunter: 4.5 billion years ago, the Earth’s atmosphere was almost pure carbon dioxide. So, if today’s atmosphere was suddenly replaced by one of almost pure carbon dioxide, that would be within normal bounds, and therefore nothing to worry about. Right? If you don’t agree, please explain more carefully what you mean by ‘normal bounds’.
“First, google ‘paleoclimatology'”
The usual ignorant response. I know far more than you do Jennifer about paleoclimatology, since you get all your information from RealClimate. And your smart alec reference to paleoclimatology has nothing to do with the answer to the question I posed. Why not just hit keys at random on your keyboard? What is the point of what you are doing, except make warmists look like fools? Very counter productive.
Will, with your schoolboyish hatred of realclimate (which I don’t believe I have ever even so much as given a link to) and your immature insults like “warmists”, you make yourself look like a fool. Grow up, please.
Perhaps, in addition to your list of personal insults directed at Gavin Schmidt, there are other reasons for his reaction. From Usenet:
“The liar at NASA does it again.”
“October The Hottest Month? Don’t Fall For Fudgin’ Hansens Tricks!”
“Hansen: September Is the Hottest October!!!”
“Cherry picked data and propaganda from a government organization
taking taxpayer’s money”
Clearly, the people checking the work of climate scientists have nothing but the highest regard for scientific rigor. And Dr. Schmidt obviously has no reason to complain about the eminently fair treatment the lying denialist community dishes out to climate scientists. lol
BTW, when does Tony Watts’ GISS “gossip” site finally get around to proving the existence of bad data from the stations he badmouths – say, using statistical tests on their output – and showing the bad data were used and actually contaminated climate work? So far, I see little evidence he has ever done much more than post funny pictures and prejudicial verbiage..?
Did I mention I have a bridge you might be interested in? 😉
“with your schoolboyish hatred of realclimate”
RealClimate is a very useful site because it presents the best arguments for AGW, and how its ideas on the subject are evolving over time. I recommend everyone visit that site, but think critically when doing so. What it does not address is very useful to understand also. I have no objection to anyone posting links to RealClimate, but do not assume this is a ‘balanced’ presentation of facts. RealClimate was setup by an environmental advocacy group. Same goes for, say, WWF or Sierra Club, if you want to talk science. Being objective is not what these sites are about.
I suspect that if RealClimate is ruled out as a source to quote, you will have nothing to quote from. As your links point to sites that point back to RealClimate. But this is not true if you are not intellectually lazy. Lots of good scientific reviews you can read, if you put the effort in.
‘Jennifer’ why do you keep posting under all these different names? Because you want people to believe there is an army of catastrophists objecting to the postings on this site? All with a vocabulary and repitare of insults identical to your own? Post with your real name and come across as a credible person who is not the intellectual superior of everyone you address, and we can all have a constructive discussion about the issues.
Will Nitschke, it’s now quite clear that you’re just trolling. I don’t believe you’ve said anything of scientific substance, or offered anything other than climateaudit to back your thoroughly misguided beliefs. You can blather away in your own little dream world (in which apparently there is only one person in the world who doesn’t believe that ‘global warming’ is a dirty evil conspiracy). I will be ignoring you from now on.
Will sez: “RealClimate was setup by an environmental advocacy group. Same goes for, say, WWF or Sierra Club, if you want to talk science. Being objective is not what these sites are about.”
Funny stuff, considering it’s posted on a lie site like climate-skeptic. Of course, when one actually checks, one finds a dramatic difference between the CVs of authors on those sites and RealClimate.
It’s interesting to me that denialists continually attack RealClimate – when it’s obviously just a substitute for being able to actually _contradict_ RealClimate with some “RealData.” =)
When do they get around to doing that?
Jennifer/CMB:
Why post a flood of insults once you can no longer respond rationally? If you believe you’ve got something balanced and intelligent to say, try respond in a balanced and reasonable way. Note your choice of words in your last few posts:
“denialists”
“you…blather away”
“your own little dream world”
“your thoroughly misguided beliefs”
“conspiracy theories”
Are you going to convince anyone that you are behaving rationally by writing like this? Flinging insults discredits the person making the accusations because it shows they are not thinking straight any more. Not everyone is a fool because they won’t agree with your opinion. As I pointed out before, be patient and be polite and you have a much better chance of getting people to see your point of view.
CMB,
“Tony Watts” has already run some rudimentary experiments demonstrating the bias that can be caused in poorly sited measurement stations. The “surveys” which he and his irregulars have been painstakingly documenting also show that GISS, NOAA… and done virtually NOTHING to insure the quality of the data Hansen and Schmidt are using to bludgeon the world with.
Sorry you don’t have the time to read his site, yet, apparently have plenty of time to bad mouth him. You are a typical DENIALIST!!
HAHAHAHAHAHAHAHAHAHA
Schmidt blogged: “the GISS press release declaring the October the ‘warmest ever’? Imaginary.”
Warren replied: “I am not quite sure what he is referring to here. I will repeat what I wrote.”
Schmidt is probably referring to the UK Daily Telegraph article* and a good number of other media and blog sources that erroneously claimed GISS made an ‘announcement’ about October. The RC post isn’t just about you, sir.
* http://www.telegraph.co.uk/opinion/main.jhtml?xml=/opinion/2008/11/16/do1610.xml
“RealClimate was setup by an environmental advocacy group.”
And which group is that? Can you corroborate?
kuhnkat, I’m not sure that Watts has demonstrated significant problems with the US temperature record owing to station bias. Using the data from the ‘good’ stations collated so far (the stations Watts’ team reckon are of high quality), and running the data at climateaudit, the ‘good’ station temperature record is a very close fit to the GISTEMP record. The deviations are very minor and well within the error bounds – but they’re only a third of the way through the project.
Barry, there is a significant problem with station siting (in the context of where within 1000 feet you place your thermometer) but errors may cancel out, but, alas, we cannot KNOW that. Once the data is corrupted there is no way back. Also, station bias can come from other things than location on the small scale. Location on a grander scale is also important. The prevalence of thermometers to be placed at airports for example, is one problem. Obviously, good communications attract economic activity.
One reasonable approach would be to assess siting bias as the difference in trend between GISS/HadCRU and UAH/RSS divided with 1,3.