It has been more than a month since my paper, “Past Longevity as Evidence for the Future,” appeared in Philosophy of Science. (Update, 6/29/2011: Read about my revised and expanded paper, now available online.) The most important parts of the paper present my own positive ideas on an objective means for using knowledge of the past as evidence for the future. However, part of the paper presents what is, in my judgment, a new and definitive refutation of the Doomsday Argument popularized by philosopher John Leslie. This Doomsday Argument has been debated for the past two decades in leading journals of philosophy and of science, and even discussed often in the mainstream media. It has been widely held that the controversy remains unresolved.

Therefore, when my paper was published in arguably the world’s leading journal for the philosophy of science—the other candidate for that designation is the British Journal for the Philosophy of Science—I thought that there would be a great deal of interest. However, to date, there has not been a single discussion of my paper, except by my colleague Glenn Marcus and me, anywhere on the Internet. Moreover, I have not received a single correspondence from anyone who has read the paper. In fact, as far as I know, the only people who have read the published version of the paper are Glenn and I.

I realize that it is summer, and philosophers and scientists have their own research to think about. But I am an impatient person about everything but my own slowness. Therefore, to help publicize my own work, I am here posting a brief description of my refutation of the Doomsday Argument.

There is also a second part of this post. Glenn finished re-reading the paper on Saturday, and he found an error on a side issue that is yet important, because the error is an incorrect criticism of another person’s work; therefore, I want to correct the error, and apologize for it, right away. I was incorrect in claiming that one of the derivations, by J. Richard Gott III, of Gott’s “delta t argument” commits the same basic error committed by the Doomsday Argument. Glenn pointed out to me that Gott’s derivation avoids that particular error.

Here then is the basic error in the Doomsday Argument.

(Warning: The rest of this post is a little technical.)

The Doomsday Argument relies on the equivalent of this equation, which is an attempted statement of Bayes’s theorem:

P(H_TS|D_pX)/P(H_TL|D_pX)
= [P(H_FS|X)/P(H_FL|X)] •[P(D_p|H_TSX)/P(D_p|H_TLX)],

where:

H_FS = the hypothesis that the future duration of the phenomenon will be short;

H_FL = the hypothesis that the future duration of the phenomenon will be long;

H_TS = the hypothesis that the total duration of the phenomenon will be short—i.e., that t_t, the phenomenon’s total longevity, = t_TS;

H_TL = the hypothesis that the total duration of the phenomenon will be long—i.e., that t_t the phenomenon’s total longevity, = t_TL, with t_TL > t_TS.

Clearly, this equation is an invalid application of Bayes’ theorem, as it conflates future duration and total duration.

Basically, that’s it! That’s the solution to a two-decade-old problem.

In my paper, I take numerical examples based on two possible corrections to this equation: considering only future durations, and considering only total durations. In both cases, I conclude that the Doomsday Argument’s claim, that there is a ‘Bayesian shift’ in favor of the shorter future duration, is fallacious. In fact, unless more information is specified, the solution to this equation is undetermined. Moreover, in many cases, the Doomsday Argument’s uniform-distribution assumption—that P(t_p/t_t | t_t) = U(0,1) where t_p = past duration and t_t = total duration—contradicts the prior information.

Now to Gott. In a famous paper in Nature in 1993, Gott presents his “delta t argument” for deriving a probability distribution for the total longevity of any phenomenon, based solely on the phenomenon’s past longevity. In a follow-up discussion in Nature in 1994, Gott presents a new derivation of his delta t formula. This new derivation is widely held to be equivalent to Leslie’s Doomsday Argument; see, for example, here and here. However, now that I have identified the conflation error in the Doomsday Argument, it is obvious that Gott’s derivation is different: Gott’s derivation avoids the conflation error! Unfortunately, this obvious fact did not become obvious to me until Glenn pointed it out to me on Saturday.

Though Gott’s derivation from the Jeffreys prior avoids the conflation error I mention above, I think that Gott’s derivation is still vulnerable to other criticisms that I mention in my paper. Nevertheless, I argue in my paper that Gott’s delta t formula, with some important constraints and modifications, has validity. Indeed, in my paper, I present an alternative derivation of Gott’s formula starting from the Jeffreys prior.

Here then is the wrong passage from my paper:

Since the Doomsday Argument is invalid, Gott’s use of the Doomsday Argument to derive his delta t argument from the Jeffreys prior is also invalid. There is, however, a valid way to derive the delta t argument from the Jeffreys prior.

Here is how I would correct that passage:

Interestingly, Gott avoids the Doomsday Argument’s conflation error in his Bayesian derivation of his delta t formula. Gott’s version of the Bayesian equation deals consistently with total durations, and no future durations. Gott is able to do so because he assumes the Jeffreys prior as the prior for total duration. Such an option is not available for the Doomsday Argument, which is intended to hold for any prior.

There is, moreover, a way to derive Gott’s delta t formula from the Jeffreys prior without having to invoke the uniform distribution assumption; this way has some other benefits as well.

My paper then presents my derivation: I use the Jeffreys prior as the prior distribution of λ in the exponential distributions for past and future longevities. I think that my derivation is more robust and precise than Gott’s for these reasons:

– My derivation makes explicit the assumption that there is the same, constant rate of risk, λ, per unit time, in both the past and the future—and that we have no knowledge of the value of λ. [Italicized phrase added on July 1.]. (Gott makes an assumption resembling this one in other derivations.) This assumption means that the same causal factors are present throughout the past and future, and the assumption specifies how knowledge of the past updates the probability distribution for the future.

– My derivation removes the need for Gott’s uniform-distribution assumption, that P(t_p/t_t | t_t) = U(0,1) where t_p = past duration and t_t = total duration. The uniform distribution of t_p/t_t is a consequence of the exponential distributions, with the same value of λ, for past and future durations.

– In my paper, I explain that the exponential distribution for past duration actually overestimates—or places an upper bound on the possibilities of—past duration. Therefore, Gott’s delta t formula is actually a worst-case bound—worst-case, assuming that we want a long future duration—on the final probability distribution for total longevity. My derivation, which makes explicit the exponential-distribution assumption for past duration as well as future duration, thereby makes clear that the result is only a bound. That Gott’s result is only a bound is consistent with a result by Frank Coolen (“Low Structure Imprecise Predictive Inference for Bayes’ Problem” and “On Probabilistic Safety Assessment in the Case of Zero Failures”), which I discuss at length in my paper.

I hope that my alternative derivation, along with my correction and apology, will somewhat atone for my incorrect criticism of Gott’s argument.

Update, 10/6/2009:

In Pisaturo (2009), I argued that the Doomsday Argument commits the error of conflating total duration and future duration. I subsequently realized that Dieks 2007 makes a similar identification, though he presents arguments somewhat different from mine, and I should have cited this work.

Also, I stated in my post above that Gott (1994) is able to avoid the Doomsday Argument’s error of conflating future duration and total duration because Gott uses the Jeffreys prior for total duration, not future duration. Such a prior is not available for the more general Doomsday Argument. Nevertheless, an argument can be made that the Jeffreys prior should indeed be used by Gott as the prior for future duration and not for total duration. (See Caves 2000, 151 for a related argument.) If that argument is correct, and I think it is, then two errors by Gott cancel each other out. –Deleted on November 7, 2009.

References

Caves, Carlton M. (2000), “Predicting Future Duration from Present Age: A Critical Assessment”, Contemporary Physics 41: 143–153.

Dieks, Dennis (2007), “Reasoning about the future: Doom and Beauty”, Synthese 156: 427–439.

Pisaturo, Ronald (2009), “Past Longevity as Evidence for the Future”, Philosophy of Science 76: 73–100.