Maximum Entropy Distribution for Random Variable of Extent [0,Infinity] and a Mean Value Mu

Maximum Entropy Principle Table of Contents TOC

End TOC

The maximum entropy constraints are as follows:

Over the interval [0,infinity]
$sum{kappa=0}{N}{P(x_i)}=1$ …. sum over all probabities must = 1
$sum{kappa=0}{N}{P(x_i){x_i}}=mu$ …. given an average value AKA "mean"

The langrangian is formed as follows:

$L=sum{kappa=0}{N}{{-P(x_i)}{log_2 P(x_i)}}+lambda_0(1-sum{kappa=1}{N}{P(x_i)} )+lambda_1(mu-sum{kappa=1}{N}{{P(x_i)}{x_i}})$

${partial L} / {partial P_i}= {-log_2 P(x_i)}-1-lambda_0-lambda_1{x_i}=0$ ….setting equal to zero to find the extrema point

${log_2 P(x_i)}=-1-lambda_0-lambda_1{x_i}$

Allowing ${lambda_1}$ to take up the slack to turn base 2 log into natural log:

${P(x_i)}=e^{-1-lambda_0} e^{-lambda_1{x_i}}$

Using the sum of probabilities =1 criteria

$sum{kappa=0}{N}{P(x_i)}=e^{-1-lambda_0} {1/{1-e^{-lambda_1}}}=1$

$sum{kappa=0}{N}{x_i}{P(x_i)}={e^{-lambda_1{x_i}}/(1-e^{-lambda_1{x_i}})^2}={mu}$ ( See below for derivation)

Derivation of mean value infinite sum:

$sum{kappa=0}{N}{x_i}{e^{-lambda_1{x_i}}}=0+1*e^{-lambda_1}+2*e^{-2{lambda_1}}+3*e^{-3{lambda_1}}cdots$

$e^{-lambda_1}{sum{kappa=0}{N}{x_i}{e^{-lambda_1{x_i}}}=..................1*e^{-2{lambda_1}}+2*e^{-3{lambda_1}}+3*e^{-4{lambda_1}}}cdots$

Subtracting we get the same old geometric series that we all know

$(1-e^{-lambda_1{x_i}}){sum{kappa=0}{N}{x_i}{e^{-lambda_1{x_i}}}}=0+1*e^{-lambda_1}+1*e^{-2{lambda_1}}+1*e^{-3{lambda_1}}cdots$

Rearranging terms:

$(1-e^{-lambda_1{x_i}}){sum{kappa=0}{N}{x_i}{e^{-lambda_1{x_i}}}}={e^{-lambda_1{x_i}}/(1-e^{-lambda_1{x_i}})}$

${sum{kappa=0}{N}{x_i}{e^{-lambda_1{x_i}}}}={e^{-lambda_1{x_i}}/(1-e^{-lambda_1{x_i}})^2}$

Another way of looking at the series:

Infinite Series Multiplication Table – The product of the 2 series is the sum of all the product entries ad infinitum
$e^{-3{lambda_1{x_i}}}$	$e^{-3{lambda_1{x_i}}}$	…	…	…
$e^{-2{lambda_1{x_i}}}$	$e^{-2{lambda_1{x_i}}}$	$e^{-3{lambda_1{x_i}}}$	…	…
$e^{-lambda_1{x_i}}$	$e^{-lambda_1{x_i}}$	$e^{-2{lambda_1{x_i}}}$	$e^{-3{lambda_1{x_i}}}$	…
		$e^{-lambda_1{x_i}}$	$e^{-2{lambda_1{x_i}}}$	$e^{-3{lambda_1{x_i}}}$
		$e^{-lambda_1{x_i}}$	$e^{-2{lambda_1{x_i}}}$	$e^{-3{lambda_1{x_i}}}$

The table uses 2 exponential series each starting with 1. In order to get the same series as the solution in the derivation above multiple the result by $e^{-lambda_1{x_i}}$

It forms a sort of number wedge or number cone. I wonder if it extends to 3 dimensions?

Observations ( Need to complete this )

….delay like Z transform
continuous form correspondence with discrete form

Research Links

Maximum Entropy Distribution for Random Variable of Extent [0,Infinity] and a Mean Value Mu

Published by Fudgy McFarlen on June 22, 2014June 22, 2014

2 Comments

Prof Von NoStrand · June 29, 2014 at 2:11 pm

Freemon SandleWould · June 29, 2014 at 2:13 pm

Leave a Reply