The Blame Game: When results are disproven, is someone at fault?

Reid Sherman
Oct 3, 2014

Sometimes scientific progress is bumpy. Look at what happened to the experiment Background Imaging of Cosmic Extragalactic Polarization (BICEP), for example. The cosmologists on the BICEP team worked for more than a decade to plan and build their instruments at the South Pole for one specific goal: to measure the polarization pattern in the Cosmic Microwave Background - the radiation left over from when the Universe was hot and dense. After years of data collection and intense analysis, BICEP scientists were confident that they had succeeded. Their results gave observational evidence - for the first time - that the theory of inflation, which describes the tiny fraction of a second after the Big Bang, was indeed correct. This, to put it mildly, was a big deal. The New York Times article about the finding was headlined “Space Ripples Reveal Big Bang’s Smoking Gun,” and articles and interviews hinted that the work was Nobel-worthy.

Events that took place in the subsequent six months, however, cast significant doubt on the findings. In September, measurements by the Planck satellite of microwave radiation from micron-sized particles (called ‘dust’) in the Milky Way were released. This new data indicated that the BICEP team did not include enough dust when modeling their results, and hence greatly overestimated how much of their signal was from the ‘primordial’ polarization for which they were searching.

Did the BICEP group do something wrong? Their analysis was careful and precise – they didn’t rush the announcement but rather held back the results for months while checking that they had controlled for every possible source of noise in the instrument. They did make the announcement to the press before their paper had gone through peer-review, which some criticized. But the headline-making results were intact after publication, so the announcement would only have been slightly delayed.

As Adrian Cho says in a report on the controversy in Science magazine, perhaps the issue was the press release and not the work itself:

Charles Bennett, a cosmologist at Johns Hopkins University in Baltimore, Maryland, says his impression is that BICEP researchers thought they'd nailed the discovery. “They just got overenthusiastic,” he says, “but it's tough to know when you really have something.” BICEP researchers might have done better to simply post their preprint as the Planck team has done, Bennett says.

What if they hadn’t made a big deal about it? Would people have spent any time thinking about the beginning of the Universe that Monday afternoon? As an academic, I often thought of science as packaged too much in discrete steps and publications, which is biased against unexciting but important negative results. Without the occasional big event, however, it’s challenging to get the public’s attention.

This article from CNN reports on particle physicists who have put an instrument aboard the International Space Station and are measuring incoming positrons, which may originate from collisions of dark matter particles. They have a successful detection, which could be the first direct evidence of dark matter, but, as with BICEP’s polarization measurement, there are many possible sources. In astrophysics, there is only one Universe, so controlled experiments can’t be run. All extra variables have to be understood and removed after the fact.

Because the results are preliminary, or out of fear of having to backtrack later, the scientists searching for evidence of dark matter particles hedge their statements so much that the article ends up rather dull. Evidence of dark matter particles would be a huge breakthrough and incredibly exciting for the field. When that breakthrough comes, it will be in incremental steps and there will be a frustrating period while the results are intriguing but need further confirmation. But it would be a shame if we never take the opportunity to fully express our excitement and celebrate scientific progress with the public.

All scientists consider how to market their results to have an impact. So what is the best way to package discovery to accurately represent the science and also engage the public?

Image: By NASA, ESA, ESO, D. Lennon and E. Sabbi (ESA/STScI), J. Anderson, S. E. de Mink, R. van der Marel, T. Sohn, and N. Walborn (STScI), N. Bastian (Excellence Cluster, Munich), L. Bedin (INAF, Padua), E. Bressert (ESO), P. Crowther (Sheffield), A. de Koter (Amsterdam), C. Evans (UKATC/STFC, Edinburgh), A. Herrero (IAC, Tenerife), N. Langer (AifA, Bonn), I. Platais (JHU) and H. Sana (Amsterdam) [Public domain], via Wikimedia Commons

Reid Sherman

Reid Sherman is a AAAS S&T Policy Fellow with NASA, working with the U.S. Group on Earth Observations. He previously spent 10 years in astrophysics research and education. Reid has a long-standing interest in science communication and how the public interacts with the research community.


This blog does not necessarily reflect the views of AAAS, its Council, Board of Directors, officers, or members. AAAS is not responsible for the accuracy of this material. AAAS has made this material available as a public service, but this does not constitute endorsement by the association.

Leave a comment

This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.

Subscribe to our blog