What will we find as we grow and analyse massive databases?
I've argued before that analysis of massive databases may, like Galileo's telescope, open up whole new worlds of knowledge. Here is what may be an example. Make what you will of it.
A 2002 paper called "CORRELATIONS OF CONTINUOUS RANDOM DATA WITH MAJOR WORLD EVENTS" by R. D. Nelson, D. I. Radin, R. Shoup and, P. A. Banceld looks at random number generators. (RNGs) Apparently there is an international network of some 50 (now 65) random number generator sites which publish a comparative database of their results. All the RNGs are independent of each other and every precaution has been taken to make them completely random and independent of any external influence. (Serious scientific stuff, not just one of your microtime seeded PHP RNGs.)
Yet there is a "small, but replicable effect on data from a global network of random generators that is correlated with designated periods of intense collective human activity or engagement, but not with any physical sources of influence." The authors use the 9/11 terrorist attacks as their main example. In statistical terms "The 109 experimental replications ... distribute normally, but have a shifted mean z-score of 0.53, representing a five sigma departure from expectation." I don't understand this, but they presumably do. I assume it means random numbers are less random at these times. After a great deal of technical analysis they add:
"Barring demonstration of a conventional interaction that can affect the random generators on a global scale, we are obliged to confront the possibility that the measured correlations may be directly associated with some aspect of consciousness attendant to global events. In particular, this evidence, if confirmed, would support the idea that some processes in nature that have been assumed to be fundamentally random are in fact somewhat mutable."
The idea is also developed here which includes links to the RNG network, and some graphical visualisations of the results. The Global Consciousness Project at Princeton call the RNGs 'eggs', an acronbym for Electro-Gaia-Grams.
Mystical buffoonery or new scientific frontiers? Remind yourself of the establishment's thought processes about Galileo, before you answer. According to the Catholic league website, Cardinal Bellarmine "...stated his personal belief that the Copernican theory was not viable as it defied human reason. However, he found no reason for it not to be treated as a hypothesis. More important, he noted that if the Copernican theory was ever proven – which he doubted could ever be accomplished – then it would be necessary to re-think the interpretation of certain Scriptural passages...."
FOOTNOTE: shortly after I wrote this, the Princeton group lost its fundng and was heavily criticised in several mathematical blogs.
ADD YOUR COMMENT:
COMMENTS SO FAR:
