In my last post, I argued that claims, properly understood, are not evidence. Admittedly, my definition of “claim” was very narrow. What I really wanted to demonstrate in that post was a few things:

The importance of relevance and causality when evaluating evidence, and that vague gestures to Bayesianism do not actually address those problems.
A clear definition of what a claim was, which I defined as merely a declarative sentence.
A better vocabulary to talk about evidence, to differentiate irrelevant data, dormant evidence, and compelling evidence.

In this post, I’m going to go a step farther. Namely: I think there’s good reason to treat testimonial claims as non-evidential in some circumstances (those attesting to highly improbable events). Similar to my conclusion in my previous post, this assessment is an a posteriori one. It could have been otherwise that testimony is evidential in all circumstances. But it’s not, so it isn’t.

Subscribe now

Why Testimony is Flawed

In our everyday lives, testimony is evidential. This is because, through the testament (pun intended) of human knowledge, we find that people claiming to have perceived x is correlated with x being true at a sufficiently high probability to where we can conclude that x is more likely, given testimony. So far, so good.

But testimony is also flawed. Everyone agrees on this point, though different philosophers disagree about the implications.

Often in these conversations between skeptics and non-skeptics, the shortcomings of testimony aren’t explained in much detail. To differentiate this post, I want to outline (very briefly!) some of the science of why testimony is unreliable. Please note: what I’m listing here is not a comprehensive overview of the literature, but a snapshot in less than an hour of research.1

In 1974, Elizabeth Loftus and John Palmer showed how leading questions and post event information can alter a person’s recollection of an event. You can read about the details here. These findings were replicated in 2025, which you can learn more here.

In 2015, Julia Shaw and Stephen Porter convinced participants that they had committed crimes in adolescence using suggestive interviewing techniques plus real autobiographical details supplied by family members. From the abstract:

“After three interviews, 70% of participants were classified as having false memories of committing a crime (theft, assault, or assault with a weapon) that led to police contact in early adolescence and volunteered a detailed false account. These reported false memories of crime were similar to false memories of noncriminal events and to true memory accounts, having the same kinds of complex descriptive and multisensory components. It appears that in the context of a highly suggestive interview, people can quite readily generate rich false memories of committing crime.”

In 2020, Elizabeth Loftus and Rachel Leigh Greenspan showed eyewitness confidence can be inflated by misleading feedback after identification. Witnesses become more certain after receiving subtle confirmation, even when their original identification was wrong. You can read more about that here.

Finally, in 2014 Krissy Wilson and Christopher French used a staged “paranormal” magic event to test how suggestion and social influence alter eyewitness reports. From the abstract:

“Participants watched a video of an alleged psychic seemingly bending a metal key by the power of psychokinesis. Half the participants heard the fake psychic suggest that the key continued to bend after it had been put down on a table and half did not. Additionally, participants were exposed to either a negative social influence (a stooge co-witness reporting that the key did not continue to bend), no social influence, or a positive social influence (a stooge co-witness reporting that the key did continue to bend). Participants who were exposed to the verbal suggestion were significantly more likely to report that the key continued to bend. Additionally, more participants reported that the key continued to bend in the positive social influence condition compared to the other two social influence conditions. Finally, believers in the paranormal were more likely to report that the key continued to bend than non-believers.”

One way I like to summarize all of this evidence is that our memories are weird. We don’t remember the past as it was, but through the lens of how we feel and think in the present. We are impressionable, and minor social prodding can lead us to change our testimony, feel more confident in what we perceive, or believe things we did not actually perceive.

Because of that, memory and testimony are always subject to change and doubt.

Why Testimony Is Inferior To Non-Testimonial Evidence

Okay, so testimony is evidential, but it’s weaker evidence relative to other forms of evidence because of all the aforementioned ways in which it can be misremembered, revised, deceptive, etc. In this way, I want to advance a simple principle to communicate why testimony is inferior to other forms of evidence:

Testimony is inferior to other forms of evidence because objective evaluators have to draw inferences about the motives and reliability of testifiers, in conjunction with drawing inferences about the data the testifiers are reporting.

If we were to put this in probabilistic terms, the difference between the reliability of testimonial and non-testimonial evidence is that the probability of the accuracy (though not necessarily interpretation) of non-testimonial data is 1 (because you can observe it as it is), while testimonial evidence will always have a probability of less than 1 because of the unreliable nature of testimony.2

Let’s illustrate this point. Take two points of evidence that support the same conclusion: A child testifying that they got a C on a test and the actual graded paper. Together, both of these pieces of evidence support the conclusion that the child got a C. But over time the child’s memory of the paper will change, while barring the paper decaying away or being lost, objective observers will have access to the paper.

The child’s memory and testimony could change for any number of reasons. He may feel shame and want to lie about his grade. He may have been an inattentive student and couldn’t recall the grade he got on that particular paper years later. He could have been a bad student to begin his academic career, and then later improved tremendously causing him to misremembered his grade. And so on. The child’s original testimony on this mundane matter may be accurate (provided the child isn’t being deceptive) but its accuracy degrades over time. Sometimes that degradation is as soon as minutes after the event!

Meanwhile, barring an incident, you, me, and anyone can all look at the child’s paper that is graded as a C and infer that the child got a C. Though the inferences we draw from non-testimonial data points may change with better understanding, explanations, or new evidence, the raw data that we perceive, have access to, and so on, remains stable, relative to the child’s memory of the event.

In this way, the reason why we trust non-testimonial forms of evidence over testimony is because, relative to other forms of evidence, the medium of the data of testimony is the human brain, and the human brain has limited capabilities and abundant opportunities to make mistakes or change.

Again, nothing about the unreliability of testimony is written in the laws of logic, but are matters of fact, derived from experience. Not only do we have well documented flaws and mistakes in testimony from witnesses sincerely trying to recall an event, we also have numerous instances of witnesses lying about what they witnessed in service of an ulterior motive. This didn’t have to be the case! Humans could have psychologies completely optimized for accurate testimony at all times. But that’s not the world we live in.

Subscribe now

Hume, The Base-Rate Fallacy, and Testimony By Itself

As the Hume guy on substack, I feel the need to point out the inferior nature of testimony relative to other forms of evidence is one of the primary reasons that Hume says it’s unwise to believe in a miracle on testimony alone. At the end of the day, Hume’s argument is a first draft of what’s today called the base-rate fallacy.

Per EBSCO:

“The base-rate fallacy, also known as base-rate neglect, is a cognitive error in which individuals fail to adequately consider the general prevalence of an event when evaluating the probability of specific outcomes. This phenomenon occurs when people focus on particular details while overlooking relevant statistical information, leading to misjudgments about likelihoods.”

For Hume, the laws of nature are reliable, unchanging, and without deviation. This isn’t because there’s some sort of rule written into the universe that is enforced by some sort of universal governor, but because we quite literally never (credibly) observe a deviation in these natural events. As Hume views probability as something weighed and not measured, the tally of a natural law is uniformly and massively on one side of a metaphorical scale, while credible deviations/violations measured on the other side of the scale are non-existent (hence why he says that miracle testimony doesn’t even amount to a probability).

So for Hume, the baseline probability of a miracle is extremely low, while the probability of a witness being wrong is much higher, with religious testimony for miracles being especially bad, increasing the probability of a witness testimony being wrong. Altogether, a miracle may seem probable given the relative reliability of a witness, but when that probability is contextualized with the base-rate probability of a miracle, it’s vanishingly unlikely to be true.

Let’s illustrate this a little more concretely, albeit quick and dirty. Let’s say you have a witness to a miracle that is 90% reliable (which I would argue is generous because of the flaws of human psychology). The 10% chance that the witness is wrong is much higher than the vanishingly small probability of a miracle (which I would say is lower than 10%, or 1%, or .1% or even .00001%). So when we put them on the scales of probability, it’s more likely that the witness is wrong.3

The key point: Hume himself doesn’t say that testimony for miracles isn’t evidence, just that it’s bad evidence, and not enough to establish a miracle.

I’m emphasizing this because pop skeptics make a mistake when they say testimony isn’t evidence for a hypothesis or that a claim made in the context of testimony to an event isn’t some kind of evidence.

In this way, they’ve misunderstood the Humean argument, conflating:

Hume’s understanding of evidence with
What Hume believes is wise to believe about miracles based on testimony.

Namely, Hume does not say that miracle testimony isn’t some kind of evidence, just that the evidential standards of establishing a miracle is high, while religious miracle testimonial evidence is so uniformly bad, at such a consistent level, that it doesn’t meet that standard, and so you shouldn’t believe in religious miracles on testimony alone.

In this way, I think lazily (emphasis on lazily!) saying “claims aren’t evidence” is wrong because it confuses the practical wisdom of Hume (miraculous testimony is uniformly bad) and the actual argument against miracles and the supernatural (the base-rate fallacy of testimony exclusively). And so, when this standard or argumentation is used against theists and miracle believers, it seemingly begs the question, skipping arguments about the base-rate fallacy and conversations about the nature of evidence; it’s asserting the conclusion without showing the work.

Still, I think when we take a step back, properly explain the base-rate fallacy, the inferior nature of testimony to other forms of evidence, and so on (as I have), we can re-assess our pragmatic definition of terms like evidence and conclude “Actually, testimonial claims aren’t evidence for extremely low probability events.” Here’s how.

A More Nuanced And Controversial Version of “Claims Aren’t Evidence”

To be clear, I still think testimony is evidential in many circumstances, but we need to be explicit about what those circumstances are. The answer is pretty clear cut: We should trust testimony in direct relation to the baseline probability of the event the testimony describes.

In this way, if someone attests to a probable (or at least non-improbable) event, we should trust that testimony. But if someone attests to an unlikely or every impossible event, we need to possess non-testimonial evidence to be confident in it.

Let’s use some examples to illustrate the point.

If someone tells you that they bought a soccer ball, you should probably believe them because that’s not an unlikely event. It’s common and perhaps likely.
If a single witness tells you a very unlikely thing happened, like a super underdog beat the defending champion at a specific sport,4 you shouldn’t believe them (they could have just as easily misread a score). Indeed, this sounds like stubborn skepticism, but if you audit your behavior, your first reaction when hearing a single person testify to a very unlikely event is to investigate it, and find other evidence affirming or denying it.
If multiple people tell you the very unlikely event happened, you should believe it probably happened, or at least be encouraged to investigate for yourself, looking for non-testimonial evidence. Similar to the previous point, many people attesting to an improbable event piques our curiosity. It may be the case that we stop our investigation before we find “objective evidence” because of the volume of people affirming the event. I think a good example of this is any national news story. I don’t think there’s anything wrong with this, so long as one does not make important decisions merely on this testimonial.5
As events become increasingly improbable, testimony becomes less evidential. If someone tells you that they won the lottery, you probably shouldn’t believe them until they produce non-testimonial evidence, like a newspaper clipping or financial records. People attesting to such things often like to brag or to use such information to exploit or scam others. Testimony still has some weight in this situation because winning the lottery isn’t impossible and people win it all the time, but it would be unwise to act upon the belief that someone won the lottery merely because they told you they did (more on that in a moment).
For events that are downright miraculous, in that they seem impossible by definition, you shouldn’t believe them based on mere testimony. Say your friend Jake says he saw your other friend Sinclair walk on water. You shouldn’t believe Jake. The baseline probability that Sinclair walked on water is so low while the likelihood of Jake being wrong is so relatively high, that Jake’s testimony alone does not have evidential weight. It’s not about hating or disliking or distrusting Jake, it’s just being wise with probabilities. Now, that doesn’t mean that Sinclair didn’t walk on water, just that you shouldn’t believe Jake absent other evidence.

To use the language of my previous post, testimonial evidence for improbable and miraculous events is dormant evidence. It’s better than an irrelevant datapoint in that you would expect this kind of evidence if the attested event happened, but it’s inferior to compelling (or non-testimonial) evidence because this data is also compatible with the attested event not happening.

The main distinction I want to draw here, though, is that:

Needing non-testimonial evidence to affirm a testimony6 is functionally indistinguishable from saying testimony is not evidential for improbable or miraculous events.

Maybe I’m just playing word games, but I don’t think so. When I reflect on what “wisdom” entails, not believing improbable events based merely on testimony is wise!

If you re-examine my examples above, you’ll see one of the emergent motives is the costs and risks of believing witnesses at different stages of probability. There’s little risk and cost to believing people (especially those you know and trust) when they testify to likely/not-unlikely events, but that risk and cost increases as witnesses testify to increasingly less likely events. As the base-rate improbability of an event increases, the costs and benefits of believing testimony become more salient.

This is because beliefs have costs and human societies are structured around our collective assessment of the probability of events and beliefs. Improbable events are rarely acted upon because they are improbable by definition, but when they are judged to instead be probable, it can radically change the world, causing rational actors to make big decisions with their time, money, and attention. Think of a natural disaster, a market crash, a pandemic, etc. Dishonest actors know this about human psychology and try to exploit it to harm others for their own gain.

And so, this heuristic of recognizing the costs and risks of believing witnesses as they attest to increasingly improbable events is rational. A wise person should not act in response to only witnesses attesting to an improbable event.

For example, if I saw my wife an hour ago and three people I barely know messaged me at this moment and tell me she was just murdered or imprisoned or hospitalized, it may be psychological denial that says I shouldn’t believe them, yet at the same time, it would be irrational of me to act as if it were true, absent further investigation. It would further be very silly of me to give these witnesses money for her burial, bail, or something else. We all know this is how scams work!

And yet, when it comes to miracles, supernatural claims, and other improbable events, some people consider it wise to hand over their metaphorical money based on testimony alone!

We also don’t live in a world where institutions put strong confidence in witness testimony alone, and we also wouldn’t want to. If I hypothetically angered a brigade of online commenters so much that they decided to make 1000 phone calls to police, telling them I murdered my wife, the police would not arrest or charge me without confirming my wife was deceased by unnatural causes. I would not want to live in a society where they would, and I imagine you wouldn’t either!

Now, I would hope law enforcement would exercise due diligence to check in on my wife and I, just as I would hope witnesses to the miraculous would lead to independent investigation, but anomalous data (such as testimony) motivating someone to further investigate an event is not the same as that person believing an account of an event, based exclusively on anomalous data.

Subscribe now

Closing Thoughts

So to wrap all of this up, is testimony evidence? Most of the time, when we’re dealing with probable events, it is. But when we’re debating religion, supernaturalism, and miraculous or improbable events, for all practical purposes, testimonial claims are not evidence.

Let’s close with some rules on testimony and evidence:

Testimonial evidence is inferior to non-testimonial evidence because it requires observers to draw inferences from the data of the testifier, which is prone to error and deception, on top of the data that it testifies to.
For probable (or at least not-improbable) events, testimony should be seen as evidential.
For improbable events, testimony should be seen as dormant evidence, not decisive evidence by itself. For these improbable events, decision makers should not make big decisions on testimony alone (and they usually don’t).
For events so improbable that they appear impossible, given our understanding of the laws of nature, testimony is not evidential. The reason why is because needing non-testimonial evidence to affirm testimonial evidence is functionally indistinguishable from saying testimonial evidence isn’t evidence.
This position isn’t merely a word game, as it reflects a reliable heuristic when balancing the costs and risks of believing improbable events. Human beings that act unwisely on insufficiently evidenced belief in improbable events are more vulnerable to scams, exploitation, deception, and other bad outcomes.

Closing out, I’ll say that “Testimonial claims by themselves are not evidence for miraculous events because the probability of erroneous testimonial evidence will always fail when balanced against the probability of the observed laws of nature being in error.”

If we were to crudely abbreviate this sentence to say “Claims aren’t evidence in religious, supernatural, or miraculous debates,” I’d be okay with it as an a posteriori understanding of evidence in this subject, given that it deals with unlikely and improbable subject matter.

All to say, I think in this narrow context, the even briefer abbreviation “Claims aren’t evidence” when talking about religion, the supernatural, and miraculous, is defensible, once again on a posteriori grounds. That is, claims are evidence, broadly speaking, but when we get into the nuances of assessing evidence for highly improbable events, it’s functionally non-evidential for wise assessors.

And so, [testimonial] claims aren’t evidence [for highly improbable, near impossible events].

1

To be clear, I’m not a psychologist, but over the years I have read lots of books on psychology and other topics that brought up various ways testimony, even for true things, is flawed. To get a rundown of some examples, I asked chatgpt to provide me with some peer reviewed studies that demonstrated the unreliability of testimony, with a preference for those that have been replicated. I tried my best to verify these studies as not-hallucinated. If I made a mistake, let me know in the comments, and I’ll make a correction.

2

That doesn’t mean the probability of testimony is always at, say .2, or something like that. Most honest testimony is going to be closer to 1 than 0, but it’s certainly not 1.

3

Now, there are some nuances in Hume’s argument that may be flawed that have to do with false positives and false negatives, per Peter Millican. However, I’m not going to hijack this post with this digression. I’ll write about it sometime in the future!

4

Say, a random FCS team beat Indiana in college football - a sentence that was not improbable five years ago but seemingly impossible today

5

I think some “claims are evidence” fans like to say that our world relies on testimony, and I think they use too expansive of a definition of testimony. For instance, I don’t think checking my weather app and believing it will rain tomorrow is the same as believing 10 people who say it will rain. But that’s an argument for another time.

6

In that the testimonial evidence alone is not sufficient to affirm an improbable or miraculous event.

Mitigated Skepticism

Testimonial Claims Are Practically Not…