Changes between Version 4 and Version 5 of OfficialTolArchiveNetworkBysPrior

Timestamp:

Dec 25, 2010, 8:14:19 PM (14 years ago)

Author:

Víctor de Buen Remiro

Comment:

--

OfficialTolArchiveNetworkBysPrior

@@ -75 +75 @@
 [[LatexEquation( D_{k}^{3}\left(\beta\right)=\begin{cases} 0 & \forall d_{k}\left(\beta\right)\leq0\\ d_{k}^{3}\left(\beta\right) & \forall d_{k}\left(\beta\right)>0\end{cases}  )]]
 
-is a continuous and differentiable in [[LatexEquation( \mathbb{R}^{n} )]]
+is continuous and differentiable in [[LatexEquation( \mathbb{R}^{n} )]]
 
 [[LatexEquation( \frac{\partial D_{k}^{3}\left(\beta\right)}{\partial\beta_{i}}=\begin{cases} 0 & \forall d_{k}\left(\beta\right)\leq0\\ 3d_{k}^{2}\left(\beta\right)A_{ki} & \forall d_{k}\left(\beta\right)>0\end{cases}  )]]
 
-The feasibility condition can then be defined as a single continuous nonlinear 
-inequality and differentiable everywhere
+The feasibility condition can be defined as a single nonlinear 
+inequality continuous and differentiable everywhere
 
 [[LatexEquation( g\left(\beta\right)=\underset{k=1}{\overset{r}{\sum}}D_{k}^{3}\left(\beta\right)\leq0 )]]
@@ -90 +90 @@
 == Multinormal prior ==
 
+When we know that a single variable should fall symmetrically close to a known value 
+we can express telling that it have a normal distribution with average in these value.
+This type of prior knowledge can be extended to higher dimensions by the multinormal
+distribution
 
+ [[LatexEquation(  \beta\sim N\left(\mu,\Sigma\right) )]] 
+
+which likelihood function is
+
+ [[LatexEquation(  lk\left(\beta\right)=\frac{1}{\left(2\pi\right)^{n}\left|\Sigma\right|^{\frac{1}{2}}}e^{^{-\frac{1}{2}\left(\beta-\mu\right)^{T}\Sigma^{-1}\left(\beta-\mu\right)}} )]]
+
+The log-likelihood is
+ 
+ [[LatexEquation( L\left(\beta\right)=\ln\left(lk\left(\beta\right)\right)=-\frac{n}{2}\ln\left(2\pi\right)-\frac{1}{2}\ln\left(\left|\Sigma\right|\right)-\frac{1}{2}\left(\beta-\mu\right)^{T}\Sigma^{-1}\left(\beta-\mu\right) )]]
+ 
+The gradient is
+
+ [[LatexEquation( \left(\frac{\partial L\left(\beta\right)}{\partial\beta_{i}}\right)_{i=1\ldots n}=-\Sigma^{-1}\left(\beta-\mu\right) )]]
+
+and the hessian 
+
+ [[LatexEquation( \left(\frac{\partial^{2}L\left(\beta\right)}{\partial\beta_{i}\partial\beta_{j}}\right)_{i,j=1\ldots n}=-\Sigma^{-1} )]]
+ 
+ 
 == Inverse chi-square prior ==
 
 
+== Transformed prior ==
+
+Sometimes we have an information prior that has a simple distribution over a 
+transformation of original variables. For example, if we know that a set of 
+variables has a normal distribution with average equal to another variable, 
+as in the case of latent variables in hierarquical models
+
+[[LatexEquation( \beta_{i}\sim N\left(\beta_{1},\sigma\right)\forall i=2\ldots n )]]
+
+Then we can define a variable transformation like this
+
+[[LatexEquation( \gamma \left(\beta\right)=\left(\begin{array}{c} \beta_{2}-\beta_{1}\\ \vdots\\ \beta_{n}-\beta_{1}\end{array}\right)\in\mathbb{R}^{n-1} )]]
+
+and define the simple normal prior
+
+[[LatexEquation( \gamma\sim N\left(0,\sigma^{2}I\right) )]]
+
+Then the log-likelihood of original prior will be calculated from the 
+transformed one as
+
+[[LatexEquation( L\left(\beta\right)=L^{*}\left(\gamma\left(\beta\right)\right) )]]
+
+If we know the first and second derivatives of the transformation
+
+[[LatexEquation( \frac{\partial\gamma_{k}}{\partial\beta_{i}}  )]]
+
+[[LatexEquation( \frac{\partial^{2}\gamma_{k}}{\partial\beta_{i}\partial\beta_{j}}  )]]
+
+the we can calculate the original gradient and the hessian after the gradient 
+and the hessian of the transformed prior as following
+
+[[LatexEquation( \frac{\partial L\left(\beta\right)}{\partial\beta_{i}}=\underset{k=1}{\overset{K}{\sum}}\frac{\partial L^{*}\left(\gamma\right)}{\partial\gamma_{k}}\frac{\partial\gamma_{k}}{\partial\beta_{i}}  )]]
+
+[[LatexEquation( \frac{\partial L^{2}\left(\beta\right)}{\partial\beta_{i}\partial\beta_{j}}=\underset{k=1}{\overset{K}{\sum}}\left(\frac{\partial^{2}L^{*}\left(\gamma\right)}{\partial\gamma_{k}\partial\beta_{j}}\frac{\partial\gamma_{k}}{\partial\beta_{i}}+\frac{\partial L^{*}\left(\gamma\right)}{\partial\gamma_{k}}\frac{\partial^{2}\gamma_{k}}{\partial\beta_{i}\partial\beta_{j}}\right)=\underset{k=1}{\overset{K}{\sum}}\left(\frac{\partial^{2}L^{*}\left(\gamma\right)}{\partial\gamma_{k}\partial\gamma_{k}}\frac{\partial\gamma_{k}}{\partial\beta_{i}}\frac{\partial\gamma_{k}}{\partial\beta_{j}}+\frac{\partial L^{*}\left(\gamma\right)}{\partial\gamma_{k}}\frac{\partial^{2}\gamma_{k}}{\partial\beta_{i}\partial\beta_{j}}\right)  )]]
 
 
+