...a companion blog to "Math-Frolic," specifically for interviews, book reviews, weekly-linkfests, and longer posts or commentary than usually found at the Math-Frolic site.

"Mathematics, rightly viewed, possesses not only truth, but supreme beauty – a beauty cold and austere, like that of sculpture, without appeal to any part of our weaker nature, without the gorgeous trappings of painting or music, yet sublimely pure, and capable of a stern perfection such as only the greatest art can show." ---Bertrand Russell (1907) Rob Gluck

"I have come to believe, though very reluctantly, that it [mathematics] consists of tautologies. I fear that, to a mind of sufficient intellectual power, the whole of mathematics would appear trivial, as trivial as the statement that a four-legged animal is an animal." ---Bertrand Russell (1957)

******************************************************************** Rob Gluck

Tuesday, September 15, 2015

Research: Good (sometimes), Often Bad and/or Ugly

A few days back Andrew Gelman favorably mentioned a Bloomberg column ("Why We Fall For Bogus Research") that touched on recent problems of unreplicability in psychology research:

The specific column referenced is here:

Then, elsewhere, Tom Siegfried put up a great dove-tailing piece on the problems of so-called "evidence-based" science, centered around medicine here:

 These pieces focus primarily, not-so-much on the quality, integrity, or analysis issues (which are significant themselves) of research studies, but on the issue of generalizability: how far out (or how big a population) can the results of a given study really be applied to? This is especially an issue for "social" or "soft" sciences. And the generalizability problem in turn touches on the even broader issue of "induction" in science. Previous work has focused on the "WEIRD" study samples employed in much western research (subjects drawn solely from a "Western, Educated, Industrial, Rich and Democratic" population pool; hardly a random sample). This deserves a longer post, but I don't have time.

I'll just say that people like to discuss "science" as if it were some monolithic practice, when in fact inherent differences reside between various fields of science: psychology, biology, medicine, anthropology, physics, engineering.... (there are even huge differences between practitioners WITHIN any given field!). Science is a continuum (ranging from excellent to good to average to mediocre to poor to piss-poor!), there is NOT some clean binary science/pseudoscience division; plenty of published science barely ranks above pseudoscience, if critically adjudged.
Even though I was a psychology major, I rarely saw psychology research that I'd hold up as good science. As recently as this year I took part in a psychology study under a well-funded, fairly well-known researcher (and author)... that I believe was essentially junk science (as commonly funded by the likes of NIH -- and to those who shudder every time NIH's funding is threatened, I have to wonder how you regard your own research... because chaff (of which there is plenty) is what is intended to be cut from the NIH budget; if you're doing excellent research you have little to fear from such cuts, but if you're just publishing-or-perishing for the sake of publishing-or-perishing pressures, well.... Anyway, enough soapbox, take a gander at the Bloomberg and Siegfried pieces (especially the latter).

One reason, by the way, that I think these issues are SO important is because the growing (even scary) anti-science sentiment in this country is, I believe, the result of a citizenry growing up with a false, idealized version of science in their heads.  If they understood the 'messiness' of science from the get-go they could better accept it, but learning of it only later in life (as if they'd been fed an elitist lie, before) they turn against it. :-(

...ADDENDUM:  Gelman has now done a further followup on this general subject:

1 comment:

  1. Thanks for sharing links of Bloomberg column about "Why We Fall For Bogus Research". Among these sites i see thebloomberreview is most authentic one and i will have read this. research statement physics