Much Ado About Turing Tests (ho-hum)

When I was much younger the first thing I read from Kurt Gödel was his "proof" of the existence of God. It seemed so weak and shallow I couldn't imagine there was any point in reading anything else generated by such a mind. So it was decades before I returned to Gödel to discover the delicious, insightful and logical profundity of which he was capable.

In a similar vein, and not to take anything away from his other accomplishments, I never could see much significance in the "Turing test" of Alan Turing. As others have occasionally written, passing a Turing test probably says more about the gullibility, subjectivity, and naivete of humans (and the ambiguity/imprecision of language), than it does about actual artificial intelligence (AI); or put more simply, Turing tests likely say more about the human interrogators than about the machines being tested.
I fully expect in the not-too-distant future a program will be written that can fool well over half of any judges interacting with it into believing they are communicating with a human being (and once one program does it, there will be a flood of others)… to which I'll yawn and quizzically mumble 'soooo?' I see it as akin to (while admitting it's different from) IBM's "Deep Blue" supercomputer beating chess grandmaster Gary Kasparov… interesting, noteworthy, even fun, but neither truly profound nor necessarily cognitively meaningful.
Of course the big news yesterday in some circles was that a program (or "Chatbot algorithm"), purporting to be a 13-year-old non-native-English-speaking boy named Eugene Goostman, for the first time passed the Turing test (by persuading 1/3 of a sample of judges that they were conversing in brief keyboarding sessions with a live human).  io9.com was one of the many outlets covering this story of human judges falling for some mimicry or simulation, if somehow you missed it:
http://io9.com/a-chatbot-has-passed-the-turing-test-for-the-first-ti-1587834715
also, this skeptical approach from Buzzfeed:
http://www.buzzfeed.com/kellyoakes/no-a-computer-did-not-just-pass-the-turing-test

One mathematician, critical of all the hype the story produced, tweeted: "isn’t it a wacky coincidence that the result came on the anniversary of Turing’s death? What were the CHANCES?!" And followed that with: "It’s almost as if the University of Reading is doing a Turing Test to see if journalists can tell the difference between ‘science’ and ‘PR’. I expect there will be more skeptical takes on the story in ensuing days, and will add links here as I see fit* (and I'm hoping certain bloggers will cover the story more fully and critically). Already, some are questioning the number of judges, the 5-minute conversations, the 30% benchmark, and other general circumstances of the test.

The Turing test is, I s'pose, better than nothing… which is part of the problem: that AI enthusiasts have so few benchmarks for measuring true AI, and perhaps one might even assert that a successful Turing test is a necessary, though insufficient, prerequisite for AI.  In the longer run, I suspect Turing tests (and certainly this particular one) will be seen as interesting curiosities that didn't deserve the ballyhoo they attracted. Anyway, here's a longish article from 2006 giving voice to some of the cynicism around "Turing tests":
http://www.thenewatlantis.com/publications/the-trouble-with-the-turing-test

The article ends on this simple note with which I'd concur: "It may be uncomfortable to live with uncertainty, but it’s far better than insisting, against all evidence, that we have accomplished a task that we have in fact scarcely begun."

(Admittedly, though, we cynics of the TT are probably in the minority, and the arguments pro/con are a lot more nuanced than I have the time/space for here.)


* ADDENDUM:  Martin Robbins now has a critical post up on the story (h/t Ed Yong):
http://www.vice.com/en_uk/read/eugene-goostman-alan-turing-test-kevin-warwick 

I was hoping that perhaps Scott Aaronson might have something to say on this matter... and now he has:  http://www.scottaaronson.com/blog/?p=1858 

And another view here, from a Turing judge, giving matters more context:
http://tinyurl.com/ogm8vol

Finally, Ian Sample/Alex Hern coverage here:
http://tinyurl.com/knm7dl8