The half life of facts

Facts have life spans, and they change in a predictable way.

That’s the big idea from the 2004 book The Half-Life of Facts: Why Everything We Know Has an Expiration Date written by mathematician Samuel Arbesman. His thesis has been referenced in other books I’ve read, so I wanted to go back to the source.

Or as Arbesman wrote: “Facts have a half life and obey mathematical rules.” He also references “fact excavation.”

The idea is that this will help us navigate an in increasingly changing world. Consider different “kinds” of facts: the “facts” of the weather changes hourly, and the “fact” of the height of Mount Everest changes over years. Arbesman introduce the idea of a “mesofact” or a piece of information that is somewhere between a variable and a constant.

Below I share my notes on the matter.

My notes are below:

• “The Great Trigonometrical Survey was a project that aimed to survey the entire Indian subcontinent with scientific precision. It was a multigenerational effort to survey, and so no one person saw its entirety
• Taxonomic bias or chauvinism: We know a lot more about cute animals than the rest
• Fast changing (weather!) and slow changing (height of Mount Everest) facts. Then there are the “mesofacts” like what we know about dinosaurs, which could change with new information but is generally fixed
• William Macneile Dixon wrote “The facts of the present won’t sit still for a portrait. They are constantly vibrating, full of clutter and confusion.”
• Derek J. de Solla Price saw a stack of encyclopedias and recognized their growth followers an exponential growth (see chart below)

• Napier’s constant: “After ?, e is probably the most well known mathematical constant. Everyone uses e to denote 2.718…; in fact, it is often referred to as Euler’s number. However, some people, notably the Brits, refer to it as Napier’s constant.”
• Harvey Lehman did similar research in social sciences
• Close Proximity Leads to Better Science, via 2010 Harvard Medical School research
• H-index: “an author-level metric that measures both the productivity and citation impact of the publications”
• “Scientometrics is the field of study which concerns itself with measuring and analysing scholarly literature”
• Harriet Zuckermam showed Nobel laureates were more likely to let junior colleagues have their names first on papers. (They shared the stage)
• Automated science like TheoryMine which “developed artificial intelligence techniques to discover new mathematical concepts and automate proof”
• Eurekametrics: author found a reverse compound interest decay in the difficulty of making discoveries; In 2005, Ben Jones published the paper “The burden of knowledge and the death of the Renaissance man” (Nice Economist story on the topic here, and this here on the lack of pandemic productivity gains)
• The only scientific field we are done discovering: new internal organs
• Lazarus taxa ” can refer to species or populations that were thought to be extinct, and are rediscovered,” and shows a third of all once-thought extinct mammals were discovered alive
• Half life of other certain fields of science. Like indivisible isotope, no individual bit of research can be known when it will be disproven but an average churn of learning appears routine
• Library science also shows when average book volumes can be turned over
• Isaac Asimov wrote “When people thought the earth was flat, they were wrong. When people thought the earth was spherical, they were wrong. If you think that thinking the earth is spherical is just as wrong as thinking the earth is flat, then your view is wronger than both of them put together.”
• We build to get closer to truth
• Definition of exponential growth: it grows at a constant rate not a constant amount
• Like Moore’s Law
• Logistic curve describes how exponential growth slows due to carrying capacity. Often called an S curve, and Clayton Christenson shows it can describe tech rollout
• Kevin Kelly’s 2010 book “What Technology Wants” shows lots of tech follows patterns of growth over time
• Jonathan Cole wrote “ Science and technology are closely related but they are not the same thing. Science involves the body of knowledge that is accumulated over time to the process of scientific inquiry…technology in its broadest sense is the process by which we modify nature to meet our needs and wants.”
• Or as Henry Petrowski wrote “Science is about understanding the origins nature and behavior of the universe and all it contains; engineering is about solving problems by rearranging the stuff of the world to make new things.”
• The author writes “Science modifies the facts of what we know about the world while technology modifies the facts of what we can do in the world.”
• Science often leads to inventions, but there are times when it goes the other way. The steam engine was invented over 100 years before we had a clear understanding of thermodynamics
• Actuarial escape velocity by Aubrey de Grey
• Henry Ellsworth, the U.S. patent office commissioner, wrote to Congress in 1943 “the advancement of the arts from year to year taxes our credulity and seems to appreciate the arrival of that when human improvement must end.” He noted exponential growth and was arguing that it would have to end at some point
• Physicist Tom Murphy has shown that there are upper limits to growth (Updated 2022 paper)
• When Tasmania broke off from Australia, the Tasmanians only maintained 24 tools but aborigines had hundreds with their larger population. It takes population to maintain expertise
• Michael Kremer published in 1993 Population Growth and Technological Change: One Million B.C. to 1990: There is a relationship between population growth and technological change
• Human population has grown at a hyperbolic growth rate, which means the rate keeps growing, and Kramer says technological and population change grows to get her
• A first order model is a rough sketch of shape
• In 1972 David Schwartz used Consumer Research Corporation’s call center to test just how fast facts traveled, when they asked people if they heard of the Wallace assassination attempt: how fast does news spread? They spread at a constant and predictable rate
• “The creation of facts as well as their decay is governed by mathematical rules“
• Black Death spread at the rate of movement of people, and the printing press spread where social contacts brought its complex learning. Jeremiah Dittmar: “economic growth was higher by as much as 60 percentage points in cities that adopted the technology.”
• Information travels through social rather physical space
• Network science: six degrees of separation
• We average 4 close social connections at a time
• Mark Granovetter’s 1973 paper on weak ties and strong ties (weak ties keep different networks together): he found we get jobs from our weak ties because they don’t have same info
• Later research using mobile phone data found it is helpful to include medium ties because they really spread information
• Google Ngrsm shows how persistent the term “brontosaurus” is even though it is just an apotasaurus: (Stephen Jay Gould has a 1991 essay collection referencing this)
• In 2006, James Fallows criticized (especially journalists) spreading the false truth that a frog will stay in boiling water
• Game of telephone is affected by what information scientist call a “noisy channel”
• Paleography studies ancient writing and so that includes mistakes in transcription. These mistakes happen in the same pattern as the errors caused by polymerase enzymes, the proteins responsible for copying DNA strands. Chaucer’s Canterbury tales was researched this way
• In a 2003 paper, Simkin and Roychowdury argue just 20 percent of scientists who cite an article have actually read it
• David Liben-Nowell and Jon Kleinberg work on chain letters and how falsehoods spread
• InnoCentive
• Don Swanson “ undiscovered public knowledge”
• “Fact excavation” author calls it
• “A long tail of expertise (everyday people in large numbers) has a greater chance of solving a problem than do experts”
• Government innovation prizes have a long istory: British for longitude in 1714, Netherlands in 1627, Spain 1567, French in 1771 sought a famine resistant source of nutrition: Antoine Parmentier suggested the potato
• Albert Laszlo Barbaasi and Reka Albert in 1999 in Science publish “preferential attachment” or the rich getting richer phenomenon in research (science of success). Interestingly they overlooked a similar paper in 1970s by Derek Price who missed Herbert Simon from 1950s, who missed Udny Yule decades earlier (Herbert Simon’s bounded rationality was influential in behavioral economics)
• This concept is known as Matthew effect by Robert Merton in sociology and Gibrat’s Law in cities and company growth . Simkin and Roychowdurty did similar for physics and probability distributions
• Stiglers law: “No scientific law is named after its discoverer“
• Godwin’s law: “an Internet adage asserting that as an online discussion grows longer (regardless of topic or scope), the probability of a comparison to Nazis or Adolf Hitler approaches”
• George Green: it’s a mystery how he solved such complex math with no known training
• Charles Babbage and Gregor Mendel did advanced work that no one understood in their lives
• Robinson and Goodman showed clinical trials cite fewer than 25% of relevant trials that precede it and have a recency bias
• Overlooked knowledge: combining different areas (like Don Swanson) or going deeper in existing discipline (Cumulative meta analysis)
• CoPub Discovery: computational discovery
• John Cisne’s 2005 paper “how science survived” showed the half life of books: 4-9 centuries (destroyed or not reproduced)
• Mendeley citation tool, and DEVONthnk
• I ask: what would be a modern version of the burning of the Library of Alexandria?
• Phase transitions: magically one more tiny shift makes water steam
• Ising model describes this phase transition
• His math predicted the discovery of first earth like planet in 2010 and says by 2024 author predicts an answer to P vs. NP Problem
• ‘Rosetta Stone’ for scaling cities: Luis Bettencourt and Sante Fe institute on cities, which require fewer gas stations per capita
• “The yearly number of patterns produced in a city per person is higher for bigger cities in a mathematically precise way. This sort of scaling is called super linear because things grow faster than they would at linear speeds, faster than a straight line. Double the population of a city and it doesn’t simply double edge productivity; it yields productivity and innovation that is more than double.” This shows up in patents, a cities gross metropolitan product, R&D budgets and even the presence of so-called “super creative individuals “
• Cities and their growth rely on paradigmatic innovations, like modern sewage systems or skyscrapers, which allow for it to continue its growth without the growth subsuming itself
• Precision: how accurate measurements are from time time
• Accuracy: accuracy to the real measure
• Biologist John Maynard Smith once said: “‘s”statistics is the science that lets you do 20 experiments a year and publish one false result in Nature.”
• Publication bias
• Low p value to get published
• Ioanndis’s 2005 paper “ why most published research findings are false”, decline effect and replication problem
• The old clinical trial joke: it was reported that 1/3 responded positively to the treatment, 1/3 had no response and the third mouse ran away (160 Ioandis)
• The core of science is very different than the frontier which are terms used by Stoneybrook’s Stephen Cole. Thomas Kuhn paradigm shift is at the frontier
• Francis Galton ushered in statistical enlightenment (first cousin to Darwin BTW)
• Stanley Migram did both the famous shock experiment with strangers and six degree of separation
• Dan Ariely “ predictably irrational“
• Shifting baseline syndrome
• David Hull showed Planck’s Principle isn’t entirely true: younger scientist don’t necessary take new ideas (like Darwin) faster than older
• Language change: idiolect is each persons approach
• Situation based dialect (Oprah introduced guests differently based on race)
• Great vowel shift
• Informational index funds (news orgs as one example)
• What’s your media diet?
• Stop memorizing facts and rely on search tools, you’ll be more accurate . “stop memorizing things and just give up“
• Luca Pacioli wrote first account of double entry bookkeeping: Mary Poovey argues in History of the modern fact that this was the beginning was thinking of information as an objective fact, the middle ages
• In her book “Being Wrong”, Kathryn Shoup writes that “errors do not lead us away from the truth. Instead they edge us incrementally toward it.”
• Chris Magee of MIT studied two periods of rapid technological change including more accurate time keeping which changed lives. “No mention is made of Klock riots even though there was resistance and adaptation was needed. And given communities the large changes apparently happened within less than a generation“
• Like our future?
• “Many medical schools inform their students that, within several years, half of what they’ve been taught will be wrong, and the teachers just don’t know which half”