Method of Moments
The Method of Moments is an inferential technique for estimating Population Parameters.
These can be: the mean, variance, skewness, kurtosis and more.
We call Moments to the Expected Values of a Random Variable.
We generate each moment by:
. Arriving to the Moment Generating Function
.Finding the corresponding derivative and plug in 0 where we have t
To use this process we start with the Moment Generating Function.
By definition:
and at
p(x) refers to the discrete distribution’s PMF (Probability Mass Function)
f(x) refers to the continuous distribution’s PDF(Probability Density Function)
So the first step we check what type of distribution we are handling. If it is a discrete we use the summation function, if it is a continuous distribution, we use the integrating function.
After arriving at the Moment Generating Function we can then work on each individual Moment.
So each moment , is the same order number of the derivative of the Moment Generating Function with 0 plugged in. M1 is the first derivative, M2 the second derivative and so on.
From here we can infer the following:
M1 is the Expected Value
And we can obtain the Variance of x by using M2 and M1
Moment Generating Function for the Bernoulli Distribution
The Bernoulli Distribution is discrete. It’s PMF is:
We start by the definition of the MGF, where:
And since this is a discrete distribution , we use the discrete branch:
We have x=0 and x=1, so we do the substitution:
and get:
which is our Moment Generating Function (MGF)
So now, we can find M1 to get E[x] and with M2 we can then obtain Var(x). Let’s do that.
And to find Var(x) we need M2, so:
Var(x) uses both moments, so we do the substitution and square M1 to arrive to the correct expression:
So, the Variance is:
Moment Generating Function for the Exponential Distribution
We start by firstly retrieving the Exponential distribution
The exponential distribution is continuous so we use the integration branch.
we now substitute function branch in our MGF
to get our MGF:
To find the first moment we do the first derivative of MGF and then plugin in 0 where t is
To find the second moment we do the second derivative of MGF and then plugin in 0 where t is
To find the variance we use the first and second moment.
So, the Variance is:
