Changes between Version 8 and Version 9 of OfficialTolArchiveNetworkBysPrior

Timestamp:

Dec 26, 2010, 12:45:33 AM (14 years ago)

Author:

Víctor de Buen Remiro

Comment:

--

OfficialTolArchiveNetworkBysPrior

@@ -32 +32 @@
 can be expressed as a constrained uniform distribution.
 
-Let [[LatexEquation( \beta )]] a uniform random variable in a region 
+Let [[LatexEquation( x )]] a uniform random variable in a region 
 [[LatexEquation(\Omega\in\mathbb{R}^{n} )]] which likelihood function is [[BR]]
 
-[[LatexEquation(lk\left(\beta\right) \propto 1 )]]
+[[LatexEquation(lk\left(x\right) \propto 1 )]]
 
 Since the logarithm of the likelihood but a constant is zero, when 
@@ -51 +51 @@
 bounds:[[BR]][[BR]]
 
-[[LatexEquation( \beta\in\Omega\Longleftrightarrow l_{k}\leq\beta_{i_k}\leq u_{k}\wedge-\infty\leq l_{k}<u_{k}\leq\infty\forall k=1\ldots r )]]
+[[LatexEquation( x\in\Omega\Longleftrightarrow l_{k}\leqx_{i_k}\leq u_{k}\wedge-\infty\leq l_{k}<u_{k}\leq\infty\forall k=1\ldots r )]]
 
 If both lower and upper bounds are non finite, then we call it the neutral 
@@ -63 +63 @@
 A polytope prior is defined by a system of compatible linear inequalities [[BR]]
 
-[[LatexEquation( A\beta\leq a\wedge A\in\mathbb{R}^{r\times n}\wedge a\in\mathbb{R}^{r} )]]
+[[LatexEquation( Ax\leq a\wedge A\in\mathbb{R}^{r\times n}\wedge a\in\mathbb{R}^{r} )]]
 
 || ''bounded region''[[BR]][[Image(source:/tolp/OfficialTolArchiveNetwork/BysPrior/doc/image003.png)]] || ''unbounded region''[[BR]][[Image(source:/tolp/OfficialTolArchiveNetwork/BysPrior/doc/image002.png)]] ||
@@ -70 +70 @@
 An special and common case of polytope region is the defined by order relations like
 
-[[LatexEquation( \beta_{i}}\leq\beta_{j}})]]
+[[LatexEquation( x_{i}}\leqx_{j}})]]
 
 We can implement this type of prior by means of a set of [[LatexEquation( r )]] 
@@ -78 +78 @@
 that is equivalent to the full set of linear inequations. 
 
-||If we define [[BR]][[BR]] [[LatexEquation( d\left(\beta\right)=A\beta-a=\left(d_{k}\left(\beta\right)\right)_{k=1\ldots r} )]] then [[br]][[br]] [[LatexEquation( D_{k}\left(\beta\right)=\begin{cases} 0 & \forall d_{k}\left(\beta\right)\leq0\\ d_{k}\left(\beta\right) & \forall d_{k}\left(\beta\right)>0\end{cases} )]] [[br]][[br]] is a continuous function in [[LatexEquation( \mathbb{R}^{n}  )]] and [[br]][[br]] [[LatexEquation( D_{k}^{3}\left(\beta\right)=\begin{cases} 0 & \forall d_{k}\left(\beta\right)\leq0\\ d_{k}^{3}\left(\beta\right) & \forall d_{k}\left(\beta\right)>0\end{cases}  )]] [[br]][[br]] is continuous and differentiable in [[LatexEquation( \mathbb{R}^{n} )]] [[br]][[br]] [[LatexEquation( \frac{\partial D_{k}^{3}\left(\beta\right)}{\partial\beta_{i}}=\begin{cases} 0 & \forall d_{k}\left(\beta\right)\leq0\\ 3d_{k}^{2}\left(\beta\right)A_{ki} & \forall d_{k}\left(\beta\right)>0\end{cases}  )]] || [[Image(source:/tolp/OfficialTolArchiveNetwork/BysPrior/doc/image004.png)]] ||
+||If we define [[BR]][[BR]] [[LatexEquation( d\left(x\right)=Ax-a=\left(d_{k}\left(x\right)\right)_{k=1\ldots r} )]] then [[br]][[br]] [[LatexEquation( D_{k}\left(x\right)=\begin{cases} 0 & \forall d_{k}\left(x\right)\leq0\\ d_{k}\left(x\right) & \forall d_{k}\left(x\right)>0\end{cases} )]] [[br]][[br]] is a continuous function in [[LatexEquation( \mathbb{R}^{n}  )]] and [[br]][[br]] [[LatexEquation( D_{k}^{3}\left(x\right)=\begin{cases} 0 & \forall d_{k}\left(x\right)\leq0\\ d_{k}^{3}\left(x\right) & \forall d_{k}\left(x\right)>0\end{cases}  )]] [[br]][[br]] is continuous and differentiable in [[LatexEquation( \mathbb{R}^{n} )]] [[br]][[br]] [[LatexEquation( \frac{\partial D_{k}^{3}\left(x\right)}{\partialx_{i}}=\begin{cases} 0 & \forall d_{k}\left(x\right)\leq0\\ 3d_{k}^{2}\left(x\right)A_{ki} & \forall d_{k}\left(x\right)>0\end{cases}  )]] || [[Image(source:/tolp/OfficialTolArchiveNetwork/BysPrior/doc/image004.png)]] ||
 
 The feasibility condition can be defined as a single nonlinear 
 inequality continuous and differentiable everywhere
 
-[[LatexEquation( g\left(\beta\right)=\underset{k=1}{\overset{r}{\sum}}D_{k}^{3}\left(\beta\right)\leq0 )]]
+[[LatexEquation( g\left(x\right)=\underset{k=1}{\overset{r}{\sum}}D_{k}^{3}\left(x\right)\leq0 )]]
 
 The gradient of this function is
 
-[[LatexEquation( \frac{\partial g\left(\beta\right)}{\partial\beta_{i}}=3\underset{k=1}{\overset{r}{\sum}}D_{k}^{2}\left(\beta\right)A_{ki} )]]
+[[LatexEquation( \frac{\partial g\left(x\right)}{\partialx_{i}}=3\underset{k=1}{\overset{r}{\sum}}D_{k}^{2}\left(x\right)A_{ki} )]]
 
 == Random priors ==
@@ -101 +101 @@
 distribution
 
- [[LatexEquation(  \beta\sim N\left(\mu,\Sigma\right) )]] 
+ [[LatexEquation(  x\sim N\left(\mu,\Sigma\right) )]] 
 
 which likelihood function is
 
- [[LatexEquation(  lk\left(\beta\right)=\frac{1}{\left(2\pi\right)^{n}\left|\Sigma\right|^{\frac{1}{2}}}e^{^{-\frac{1}{2}\left(\beta-\mu\right)^{T}\Sigma^{-1}\left(\beta-\mu\right)}} )]]
+ [[LatexEquation(  lk\left(x\right)=\frac{1}{\left(2\pi\right)^{n}\left|\Sigma\right|^{\frac{1}{2}}}e^{^{-\frac{1}{2}\left(x-\mu\right)^{T}\Sigma^{-1}\left(x-\mu\right)}} )]]
 
 The log-likelihood is
  
- [[LatexEquation( L\left(\beta\right)=\ln\left(lk\left(\beta\right)\right)=-\frac{n}{2}\ln\left(2\pi\right)-\frac{1}{2}\ln\left(\left|\Sigma\right|\right)-\frac{1}{2}\left(\beta-\mu\right)^{T}\Sigma^{-1}\left(\beta-\mu\right) )]]
+ [[LatexEquation( L\left(x\right)=\ln\left(lk\left(x\right)\right)=-\frac{n}{2}\ln\left(2\pi\right)-\frac{1}{2}\ln\left(\left|\Sigma\right|\right)-\frac{1}{2}\left(x-\mu\right)^{T}\Sigma^{-1}\left(x-\mu\right) )]]
  
 The gradient is
 
- [[LatexEquation( \left(\frac{\partial L\left(\beta\right)}{\partial\beta_{i}}\right)_{i=1\ldots n}=-\Sigma^{-1}\left(\beta-\mu\right) )]]
+ [[LatexEquation( \left(\frac{\partial L\left(x\right)}{\partialx_{i}}\right)_{i=1\ldots n}=-\Sigma^{-1}\left(x-\mu\right) )]]
 
 and the hessian 
 
- [[LatexEquation( \left(\frac{\partial^{2}L\left(\beta\right)}{\partial\beta_{i}\partial\beta_{j}}\right)_{i,j=1\ldots n}=-\Sigma^{-1} )]]
+ [[LatexEquation( \left(\frac{\partial^{2}L\left(x\right)}{\partialx_{i}\partialx_{j}}\right)_{i,j=1\ldots n}=-\Sigma^{-1} )]]
  
 
-== Transformed prior ==
+=== Inverse chi-square prior ===
+
+In a model with normal waste is permissible to award the unknown variance an 
+inverse chi-square distribution with scale parameter equal to the average of 
+squares of residuals and freedom degrees the data length.
+
+The likelihood is now the scalar function
+
+ [[LatexEquation( lk\left(x\right)=\frac{\left(\frac{\nu}{2}\right)^{\frac{\nu}{2}}}{\Gamma\left(\frac{\nu}{2}\right)}x^{-\frac{\nu}{2}-1}e^{-\frac{\nu}{2x}} )]]
+ 
+The log-likelihood is
+ 
+ [[LatexEquation( L\left(x\right)=\frac{\nu}{2}\ln\left(\frac{\nu}{2}\right)-\ln\left(\Gamma\left(\frac{\nu}{2}\right)\right)-\left(\frac{\nu}{2}+1\right)x-\frac{\nu}{2x} )]]
+
+The first derivative is
+ 
+ [[LatexEquation( \frac{dL\left(x\right)}{dx}=-\left(\frac{\nu}{2}+1\right)+\frac{\nu}{2x^{2}} )]]
+
+The second derivative is
+ 
+ [[LatexEquation( \frac{d^{2}L\left(x\right)}{d^{2}x}=-\frac{\nu}{6x^{3}} )]]
+ 
+
+
+=== Transformed prior ===
 
 Sometimes we have an information prior that has a simple distribution over a 
@@ -127 +151 @@
 as in the case of latent variables in hierarquical models
 
-[[LatexEquation( \beta_{i}\sim N\left(\beta_{1},\sigma\right)\forall i=2\ldots n )]]
+[[LatexEquation( x_{i}\sim N\left(x_{1},\sigma\right)\forall i=2\ldots n )]]
 
 Then we can define a variable transformation like this
 
-[[LatexEquation( \gamma \left(\beta\right)=\left(\begin{array}{c} \beta_{2}-\beta_{1}\\ \vdots\\ \beta_{n}-\beta_{1}\end{array}\right)\in\mathbb{R}^{n-1} )]]
+[[LatexEquation( \gamma \left(x\right)=\left(\begin{array}{c} x_{2}-x_{1}\\ \vdots\\ x_{n}-x_{1}\end{array}\right)\in\mathbb{R}^{n-1} )]]
 
 and define the simple normal prior
@@ -140 +164 @@
 transformed one as
 
-[[LatexEquation( L\left(\beta\right)=L^{*}\left(\gamma\left(\beta\right)\right) )]]
+[[LatexEquation( L\left(x\right)=L^{*}\left(\gamma\left(x\right)\right) )]]
 
 If we know the first and second derivatives of the transformation
 
-[[LatexEquation( \frac{\partial\gamma_{k}}{\partial\beta_{i}}  )]]
+[[LatexEquation( \frac{\partial\gamma_{k}}{\partialx_{i}}  )]]
 
-[[LatexEquation( \frac{\partial^{2}\gamma_{k}}{\partial\beta_{i}\partial\beta_{j}}  )]]
+[[LatexEquation( \frac{\partial^{2}\gamma_{k}}{\partialx_{i}\partialx_{j}}  )]]
 
 the we can calculate the original gradient and the hessian after the gradient 
 and the hessian of the transformed prior as following
 
-[[LatexEquation( \frac{\partial L\left(\beta\right)}{\partial\beta_{i}}=\underset{k=1}{\overset{K}{\sum}}\frac{\partial L^{*}\left(\gamma\right)}{\partial\gamma_{k}}\frac{\partial\gamma_{k}}{\partial\beta_{i}}  )]]
+[[LatexEquation( \frac{\partial L\left(x\right)}{\partialx_{i}}=\underset{k=1}{\overset{K}{\sum}}\frac{\partial L^{*}\left(\gamma\right)}{\partial\gamma_{k}}\frac{\partial\gamma_{k}}{\partialx_{i}}  )]]
 
-[[LatexEquation( \frac{\partial L^{2}\left(\beta\right)}{\partial\beta_{i}\partial\beta_{j}}=\underset{k=1}{\overset{K}{\sum}}\left(\frac{\partial^{2}L^{*}\left(\gamma\right)}{\partial\gamma_{k}\partial\beta_{j}}\frac{\partial\gamma_{k}}{\partial\beta_{i}}+\frac{\partial L^{*}\left(\gamma\right)}{\partial\gamma_{k}}\frac{\partial^{2}\gamma_{k}}{\partial\beta_{i}\partial\beta_{j}}\right)=\underset{k=1}{\overset{K}{\sum}}\left(\frac{\partial^{2}L^{*}\left(\gamma\right)}{\partial\gamma_{k}\partial\gamma_{k}}\frac{\partial\gamma_{k}}{\partial\beta_{i}}\frac{\partial\gamma_{k}}{\partial\beta_{j}}+\frac{\partial L^{*}\left(\gamma\right)}{\partial\gamma_{k}}\frac{\partial^{2}\gamma_{k}}{\partial\beta_{i}\partial\beta_{j}}\right)  )]]
+[[LatexEquation( \frac{\partial L^{2}\left(x\right)}{\partialx_{i}\partialx_{j}}=\underset{k=1}{\overset{K}{\sum}}\left(\frac{\partial^{2}L^{*}\left(\gamma\right)}{\partial\gamma_{k}\partialx_{j}}\frac{\partial\gamma_{k}}{\partialx_{i}}+\frac{\partial L^{*}\left(\gamma\right)}{\partial\gamma_{k}}\frac{\partial^{2}\gamma_{k}}{\partialx_{i}\partialx_{j}}\right)=\underset{k=1}{\overset{K}{\sum}}\left(\frac{\partial^{2}L^{*}\left(\gamma\right)}{\partial\gamma_{k}\partial\gamma_{k}}\frac{\partial\gamma_{k}}{\partialx_{i}}\frac{\partial\gamma_{k}}{\partialx_{j}}+\frac{\partial L^{*}\left(\gamma\right)}{\partial\gamma_{k}}\frac{\partial^{2}\gamma_{k}}{\partialx_{i}\partialx_{j}}\right)  )]]
 
+Thus it is possible to define a variety of information a priori from a 
+pre-existing instance of a transformation defined with their first and 
+second derivatives.
 
+For example we can define a log-normal prior without to define explicitly 
+its log-likelihood, gradient and hessian.