The core of this post comes from Mathematical Statistics and Data Analysis by John A. Rice which is a useful resource for subjects such as UCT’s STA2004F.

Introduction

The Cauchy distribution has a number of interesting properties and is considered a pathological (badly behaved) distribution. What is interesting about it is that it is a distribution that we can think about in a number of different ways*, and we can formulate the probability density function these ways. This post will handle the derivation of the Cauchy distribution as a ratio of independent standard normals and as a special case of the Student’s t distribution.

Like the normal- and t-distributions, the standard form is centred on, and symmetric about 0. But unlike these distributions, it is known for its very heavy (fat) tails. Whereas you are unlikely to see values that are significantly larger or smaller than 0 coming from a normal distribution, this is just not the case when it comes to the Cauchy distribution.

Building a Cauchy distribution

Ratio of standard normals

If X_1 and X_2 are independent Standard Normal variables and Y = \frac{ X_1}{X_2}

To find the the distribution of Y we will use the method of transforming the old variables to find the joint pdf of new ones. First we will need the joint distribution of (X_1,X_2). This can be found quite easily as the distributions are independent, which means that we have:

f_{X_1,X_2}(x_1,x_2) = f_{X_1}(x_1)f_{X_2}(x_2)

Since both of these are standard normal we have:

f_{X_1,X_2}(x_1,x_2) = f_{X_1}(x_1)f_{X_2}(x_2) = \sqrt{\frac{1}{2\pi}}\sqrt{\frac{1}{2\pi}}e^{-\frac{x^2_1}{2}}e^{-\frac{x^2_2}{2}} = \frac{e^{\frac{x^2_1 + x^2_2}{2}}}{2\pi}

with -\infty < X_1,X2 < \infty.

We can then define the transformations of the variables as follows. We already have our first variable, the variable of interest Y = \frac{ X_1}{X_2} but we will need two new variables to use this method. We introduce the dummy variable Z = X_2.

The next step is to find the inverses of these, i.e. we now need to find Y,Z in terms of X_1, X_2. This is fairly simple to do and we can find them as:

X_1 = YZ and X_2= Z

with Jacobian, J = \det \left [ {\begin{array}{c} Y, Z  \\ 0,1  \\ \end{array} } \right]

Which reduces to |Z|

then we have:

f_Y(y) = \int_{-\infty}^{\infty} \frac{|z|}{2 \pi} e^{-z^2/2} e^{-z^2y^2/2} dz

We may then use symmetry of this distribution about 0 (which comes from the absolute value and squaring operations) to re-write this as:

f_Y(y) = \int_{0}^{\infty} \frac{z}{2 \pi} e^{-z^2/2} e^{-z^2y^2/2} dz

From here, we first integrate by making a u substitution and then by recognising the form of the Exponential  (or Gamma) distribution.

Let u =z^2 then du = 2zdz and

\int_{0}^{\infty} \frac{z}{2 \pi} e^{-z^2/2} e^{-z^2y^2/2} dz= \frac{1}{2 \pi} \int_{0}^{\infty} e^{-u(1+y^2)/2}du

Noting that y_2 is constant with respect to u and that

 \int_{0}^{\infty} e^{-\lambda x}dx = \frac{1}{\lambda}  we can let \lambda = (1+y^2)/2

and get:

f_Y(y) =\frac{1}{2 \pi} \int_{0}^{\infty} e^{-\frac{u(1+y^2)}{2}}du = \frac{1}{\pi (1+y^2)} for -\infty < y < \infty.

which is the pdf of a Cauchy random variable.

From the t-distribution

The Cauchy distribution can also be thought of a Student’s t distribution with 1 degree of freedom. To show this, we consider the pdf of a t-distribution:

f(t) = \frac{\Gamma((\nu+1)/2)}{\sqrt{\nu \pi} \Gamma(\frac{\nu}{2})} ( 1 + \frac{t^2}{\nu})^{-\frac{1}{2}(\nu+1)} where \nu is the degrees of freedom and \Gamma is the gamma function.

When we set \nu=1 and note that \Gamma(\frac{1}{2}) = \pi and \Gamma(1) = 1 we recover the Cauchy density function.

This makes sense if we think of the t-distribution as a ratio of a standard normal and the square root of a chi-squared distribution divided by its degrees of freedom. If the degrees of freedom is 1 then the denominator of this ratio is a standard normal. This is exactly what we had above.

Intuitively if we think about the t-distribution as the distribution of a standardised sample mean and the mean is calculated from only 2 observations (giving a single degree of freedom) then the distribution of the standardised sample mean is Cauchy. We shall see shortly that the moments of the Cauchy distribution are not defined. This is an intuitive reason to collect a lot of data- if we have a very small sample, the distribution of the standardised sample mean behaves badly and we cannot do good inference on it.

Moments

The Cauchy distribution is notable because the integer moments are not defined.

This is a particular problem if we want to apply the central limit theorem, which requires a finite mean and variance.

We might expect the mean to be 0, because the distribution is symmetric and centred on 0, but this intuition is incorrect. This can be seen by considering the integral:

\int_{-\infty}^{\infty} \frac{|x|}{(1+x^2)} dx which diverges. This means that the density in the tails decreases slowly enough that the probability of extreme values is still reasonably high. As a result the expectation, and by extension, higher order moments, are not defined. It is in this sense that the distribution is pathological and quite interesting.

 

*The (general) Cauchy distribution can also be thought of as the distribution of the x-co-ordinate of a ray from a given point drawn at a uniformly distributed angle.

 

 

How clear is this post?