<table width = 900 border = 0><tr><td>
# <b>NMST539 | Lab Session 3</b>

## <b>Multivariate Normal Distribution</b>
### (joint, marginal, and conditional distributions)
### <font color = "#880000"><b>Summer Term 2021/2022 |  01/03/22 </b></font>
###### <a href = "Lab3_2022.Rmd">source file (UTF8 encoding)</a>

**************


<b>Outline for the third NMST539 lab session </b> 
<ul>
<li>multivariate normal distribution (joint, marginal, and conditional;</li>
<li>linear combinations of random vectors with multivariate normal distribution;</li>
<li>random vector properties: normality vs. independence/dependence;</li>
</ul>


The R-software is available for download from the website: <a href = "https://www.r-project.org">https://www.r-project.org</a>

A user-friendly interface (one of many): <a href = "https://www.rstudio.com">RStudio</a>.<br><br>

<b>Manuals and introduction into <i>R</i>
(in Czech or English):</b><br>
		
<ul>
<li>Bína, V., Komárek, A. a Komárková, L.: <i>Jak na jazyk R.</i>  (<a href = "NMFM301/Rmanual2.pdf">PDF súbor</a>)</li>
<li>Komárek, A.: <i>Základy práce s R.</i>  (<a href = "http://msekce.karlin.mff.cuni.cz/~komarek/vyuka/dataRko/Rmanual1.pdf">PDF súbor</a>)</li>
<li>Kulich, M.: <i>Velmi stručný úvod do R.</i> (<a href = "NMFM301/uvodr.pdf" >PDF súbor</a>)</li>
<li>De Vries, A. a Meys, J.: <i>R for Dummies.</i> (ISBN-13: 978-1119055808) </li>		
</ul>

<b>Theoretical/statistical references</b><br>
<ul>
<li>Hardle, W. and Simar.L.: Applied Multivariate Statistical Analysis. Springer, 2015</li>
<li>Mardia, K., Kent, J., and Bibby, J.:Multivariate Analysis, Academic Press, 1979.</li>
</ul>


**************

#### <font color = "#880000"><b>1. Conditional normal distribution</b></font>

Let us start by considering  a simple two-dimensional normal distribution of a random vector $\Big(\begin{array}{x}X_{1}\\X_{2}\end{array}\Big) \in \mathbb{R}^2$. 
This is usually formally denoted as 

<center>
$\Big(\begin{array}{x}X_{1}\\X_{2}\end{array}\Big) \sim N_{2}\left(\boldsymbol{\mu} = \Big(\begin{array}{c} \mu_{1} \\ \mu_{2}\end{array}\Big), \Sigma = \left( \begin{array}{cc} \sigma_{1}^{2} & \sigma_{12} \\\sigma_{21} & \sigma_{2}^{2} \end{array} \right) \right)$,
</center>

where $\boldsymbol{\mu} \in \mathbb{R}^2$ is the  vector of the mean values and $\Sigma$ is the variance-covariance matrix, which is positive definite and symmetric, thus $\sigma_{12} = \sigma_{21}$. 
The correspoding density function (of the two dimensional normal distrubution) is given by the expression 

<center>
$\large{f(\boldsymbol{x}) = \frac{1}{2 \pi |\Sigma|^{1/2}} exp\Big\{ -\frac{1}{2} (\boldsymbol{x} - \boldsymbol{\mu})^{\top} \Sigma^{-1} (\boldsymbol{x} - \boldsymbol{\mu}) \Big\},}$
</center><br>

for an arbitrary element of a two dimensional real plane, thus, $\boldsymbol{x} = (x_{1}, x_{2})^{\top} \in \mathbb{R}^{2}$.<br><br>

This density can be used to derive the marginal distrubution of the random variables $X_{1}$ and $X_{2}$ or the conditional distrubution of $X_{1}$ given $X_{2}$ (or $X_{2}$ given $X_{1}$ respectively). In the following we will do both. 

<ul>
<li>For the marginal density of $X_{1}$ we need to obtain  $f_{X_1}(x_{1}) = \int_{\mathbb{R}} f(x_{1}, x_{2}) \mbox{d}x_{2}$ and, analogously,  for the marginal density of $X_{2}$, 
one needs to  integrate the join density wrt. the first covariate instead, i.d., $f_{X_2}(x_{2}) = \int_{\mathbb{R}} f(x_{1}, x_{2}) \mbox{d}x_{1}$. Both marginals are again normaly distributed and it holds that 
<center>
$X_{1} \sim N(\mu_1, \sigma_1^2)~~~$ and $~~~X_{2} \sim N(\mu_2, \sigma_2^2)$.
</center><br><br>
</li>

<li>For a simple example with a two dimensional normal distribution the conditional distribution distribution of $X_{2}$ given $X_{1} = x_{1}$ is, again, normal  and it holds 
that 
<center>
$$(X_{2} | X_{1 } = x_{1}) \sim N\Big(\mu_{2} + \frac{\sigma_{21}(x_1 - \mu_1)}{\sigma_{1}^2}, \sigma_{2}^2 - \frac{\sigma_{12}\sigma_{21}}{\sigma_{1}^2}\Big).$$
</center>
An analogous expression can be obtained also for the conditional distribution of $X_{1}$ given $X_{2}$.
</li>
</ul>


<br><br>




Now we can apply the formulas given above to obtain the marginal and conditional distributions for some given two-dimensional normal distribution. 
We will use the R library `mvtnorm` (which needs to be firstly installed on R). The library is loaded into the working environment by running the command 

```{r, message=F, warning=F}
library("mvtnorm")
```

Let us consider a simple example with two dimensional normal distribution with the zero mean vector $\boldsymbol{\mu} = (0,0)^\top$, and  the variance-covariance matrix $\Sigma =  \left( \begin{array}{cc} 1 & 0.8 \\0.8& 1\end{array} \right)$. We would like to calculate the conditional distribution of $X_{2}$ given $X_{1} = 0.7$.<br><br>

#### <font color = "#880000">Individual task</font>
<hr size="3" width="100%" noshade style="color:#440000" align="left" />
<ul>
<li>Is there any linear relationship between the covariates $X_1$ and $X_2$? Can you quantitatively express how strong this relationship is?</li><br>

<li>In terms of the linear regression modelling approach: imagine you obtain a sample from the given two dimensional normal distribution and you fit 
a simple regression line to the data. Do you have some expectation about the parameter estimates you obtain when fitting the linear regression model? <br>
What about the correlation coefficient (respectively its sample version) and the Multiple $R^2$ value in the summary output?

```{r, eval = T, fig.width= 10}
n <- 100
sample <- rmvnorm(n, c(0, 0), matrix(c(1, 0.8, 0.8, 1),2,2))
(summary(lm(sample[,1] ~ sample[,2])))
(cor(sample[,1], sample[,2]) * sd(sample[,1]) /sd(sample[,2]))

plot(sample[,1] ~ sample[,2], pch = 21, bg = "gray")
abline(lm(sample[,1] ~ sample[,2]), col = "red", lwd = 2)
lines(c(-4, 4) ~ rep(mean(sample[,2]), 2), col = "blue", lty = 2)
lines(rep(mean(sample[,1]), 2) ~ c(-4, 4), col = "blue", lty = 2)
```
</li>

<li>Note, that the blue dashed lines do not represent the point $[0,0] \in \mathbb{R}^2$. Instead, they denote `c(mean(sample[,2]), mean(sample[,1]))`. </li>
</ul>
<hr size="3" width="100%" noshade style="color:#440000" align="left" />
<br><br>


Use the following piece of the R code to obtain a comparison between the joint distribution, marginal distribution, and the conditional distribution
of $X_{2}$ given $X_{1} = 0.7$. Derive the theoretical expressions for both, the marginal distributions and the conditional distribution.  


```{r, fig.width = 10}
Sigma <- matrix(c(1,.8,.8,1), nrow=2) ## variance-covariance matrix

x <- seq(-3,3,0.01)
contour(x,x,outer(x,x,function(x,y){dmvnorm(cbind(x,y),sigma=Sigma)}), col = "blue")

abline(v=.7, lwd=2, lty=2, col = "red")
text(0.75, -2, labels=expression(x[1]==0.7), col = "red", pos = 4)

### conditional distribution of X2 | X1 = 0.7
y <- dnorm(x, mean =  0.8 * 0.7, sd = sqrt(1 - 0.8^2))
lines(y-abs(min(x)),x,lty=2,lwd=2, col = "red")

### marginals
m1 <- m2 <- dnorm(x, 0, 1)
lines(x, m1 - abs(min(x)), lty = 1, lwd = 2, col = "gray30")
lines(m2 - abs(min(x)), x, lty = 1, lwd = 2, col = "gray30")
```

The conditional distribution can be obtained for any value $X_{1} = x_1$, for instance, in the following we vizualize the conditional distribution of $(X_{2} | X_{1} = -1)$.
Marginal distributions remain both the same. The conditional distribution changed correspondingly. 

```{r, fig.width = 10}
contour(x,x,outer(x,x,function(x,y){dmvnorm(cbind(x,y),sigma=Sigma)}), col = "blue")
abline(v=-1, lwd=2, lty=2, col = "red")

### marginals
lines(x, m1 - abs(min(x)), lty = 1, lwd = 2, col = "gray30")
lines(m2 - abs(min(x)), x, lty = 1, lwd = 2, col = "gray30")

### conditional distribution given X1 = 0.7
abline(v=.7, lwd=2, lty=2, col = "gray")
lines(y + 0.7,x,lty=2,lwd=2, col = "gray")
lines(y - abs(min(x)),x,lty=2,lwd=2, col = "gray")
text(0.75, -2, labels=expression(x[1]==0.7), col = "gray", pos = 4)

### conditional distribution of X2 | X1 = - 1
y2 <- dnorm(x, mean = 0.8 * (- 1), sd = sqrt(1 - 0.8^2))
lines(y2 - abs(min(x)),x,lty=2,lwd=2, col = "red")
```

Another option in R to obtain the same output:  

```{r, fig.width = 10}
contour(x,x,outer(x,x,function(x,y){dmvnorm(cbind(x,y),sigma=Sigma)}), col = "blue")
abline(v=-1, lwd=1, lty=1, col = "red")
lines(y2 - 1 ,x,lty=2,lwd=2, col = "red")
```


And even more generalized version for a set of different values of $x_{1} \in \mathbb{R}$:

```{r, fig.width = 10}
contour(x,x,outer(x,x,function(x,y){dmvnorm(cbind(x,y),sigma=Sigma)}), col= "blue")
condN <- function(x, cx){dnorm(x, mean = 0.8 * cx, sd = sqrt(1 - 0.8^2))}
for (i in seq(-2, 2, by = 0.25)){col <- colors()[grep("red",colors())][4*i + 9]; 
                                 lines(condN(x, i) + i, x, lwd = 2, col = col);
                                 abline(v = i, lwd=1, lty=1, col = col)}
```

The theoretical model above can be now compared with the sample version below: 


```{r, fig.width = 10}
set.seed(1234)
s <- rmvnorm(5000, c(0, 0), Sigma)
plot(s, pch = 21, bg = "lightblue", xlim = c(-3, 3), ylim = c(-3, 3), xlab = "", ylab = "", cex = 0.6)
abline(lm(s[,2] ~ s[,1]), col = "red", lwd = 2)

```

```{r, fig.width = 10}
set.seed(1234)
s <- rmvnorm(5000, c(0, 0), Sigma)
plot(s, pch = 21, bg = "lightblue", xlim = c(-3, 3), ylim = c(-3, 3), xlab = "", ylab = "", cex = 0.8)
abline(lm(s[,2] ~ s[,1]), col = "red", lwd = 2)

for (i in seq(-2, 2, by = 0.25)){col <- colors()[grep("red",colors())][4*i + 9]; 
                                 lines(condN(x, i) + i, x, lwd = 2, col = col);
                                 abline(v = i, lwd=1, lty=1, col = col)}
```

<br><br>

#### <font color = "#880000">Individual task</font>
<hr size="3" width="100%" noshade style="color:#440000" align="left" />
<ul>
<li>Consider the conditional distribution we obtained above. Can you numerically reconstruct (obtain) the values which are used as the parameters in the normal distribution?</li><br>

<li>Recall the general formula for a conditional distribution of a random vector $\boldsymbol{X}_{2}$ given $\boldsymbol{X}_{1} = \boldsymbol{x}_{1}$ if the join distribution of $(\boldsymbol{X}_{1}^\top, \boldsymbol{X}_{2}^\top)^\top$ is normal 
and $\boldsymbol{X}_{1}$ is a $p$-dimensional random vector for $p > 1$ and $\boldsymbol{X}_{2}$ is a $q$-dimensional random vector with $q > 1$.</li><br>

<li>Recal a normal linear model $Y_{i} = \alpha + \beta X_i + \varepsilon_i$, for $i = 1, \dots, n$. What is the distribution of $Y$ given $X = x$? Do you see the relationship of the model 
with the countour plot above? </li><br>

<li>Consider a multivariate example for at least a four dimensional normal distribution and calculate the conditional distribution of the first two elements $(X_{1}, X_{2})$ given the remaining elements. Create a contour plot for the conditional two-dimensional normal distribution with various values for the condition.</li>

<li>Recall, that the joint distribution of some random vector (for instance, a normally distributed random vector) 
can be straightforwardly used to obtain any marginal distribution of interest. Analogously, the corresponding conditional distributions can be formed. 
On the other hand, however, having all possible marginal distributions is still not enough to reconstruct the whole joint distribution. Specifically, 
normally distributed marginals of some random vector <b>do not imply</b> normality of the whole random vector. 
</ul>
<hr size="3" width="100%" noshade style="color:#440000" align="left" />

<br>

#### <font color = "#880000">Comment</font>
<hr size="3" width="100%" noshade style="color:#440000" align="left" />
For a general multivariate normal distribution denoted as 

<center>
$\Big(\begin{array}{x}\boldsymbol{X}_{1}\\\boldsymbol{X}_{2}\end{array}\Big) \sim N_{p + q}\left(\boldsymbol{\mu} = \Big(\begin{array}{c} \boldsymbol{\mu}_{1} \\ \boldsymbol{\mu}_{2}\end{array}\Big), \Sigma = \left( \begin{array}{cc} \Sigma_{11} & \Sigma_{21} \\\Sigma_{12} & \Sigma_{22} \end{array} \right) \right)$
</center><br><br>

for some random vectors $\boldsymbol{X}_{1} \in \mathbb{R}^{p}$ and $\boldsymbol{X}_{2} \in \mathbb{R}^{q}$ for $p, q \in \mathbb{N}$ and the corresponding expectation vectors $\boldsymbol{\mu}_{1}$ and $\boldsymbol{\mu}_{2}$ and the variance-covariance matrix $\Sigma$ the conditional distribution of $\boldsymbol{X}_{1}$ given $\boldsymbol{X}_{2} = \boldsymbol{x}_{2}$ is defined by the following expression: 

<center>
$(\boldsymbol{X}_{1} | \boldsymbol{X}_{2} = \boldsymbol{x}_{2}) \sim N_{p}\left(\boldsymbol{\mu}_{1} + \Sigma_{12}\Sigma_{22}^{-1}(\boldsymbol{x}_{2} - \boldsymbol{\mu}_{2} ), \Sigma = \Sigma_{11} - \Sigma_{12}\Sigma_{22}^{-1}\Sigma_{21} \right)$.
</center>
<hr size="3" width="100%" noshade style="color:#440000" align="left" />
<br><br>

#### <font color = "#880000"><b>2. Conditional multivariate normal distribution with the 'condMVNorm()' package</b></font>

There is a nice (and quite brief) R library available from the R cran repository which can be used to calculate the conditional distribution of some random vector $\boldsymbol{X}_{1}$ given another random vector (value) $\boldsymbol{X}_{2} = \boldsymbol{x}_{2}$, where the joint distribution of $(\boldsymbol{X}_{1}^{\top}, \boldsymbol{X}_{2}^{\top})^{\top}$ is assuned ti be multivariate normal (with the corresponding dimension and the given parameters). The library can be installed by using command `install.packages("condMVNorm")`. <br><br>

```{r}
library("condMVNorm")
```

Functions `dcmvnorm()` and `condMVN` can be directly used to obtain the conditional distribution of interest (see the help session and use `?dcmvnorm` and `?condMVN` for more details). 

We apply these functions to the previous example with a two dimensional normal distribution with a zero mean vector and the variance-covariance matrix $\Sigma = \left( \begin{array}{cc} 1 & 0.8 \\ 0.8 & 1 \end{array} \right)$. We want to obtain the conditional distribution of $X_{2}$ given $X_1 = 0.7$.

```{r}
condDist <- condMVN(mean = c(0,0), sigma = Sigma, dependent.ind = 2, given.ind = 1, X.given = 0.7)
```

and also

```{r}
dcmvnorm(0, mean = c(0,0), sigma = Sigma, dependent.ind = 2, given.ind = 1, X.given = 0.7)
```

which should be compared with 

```{r}
dnorm(0, condDist$condMean, sqrt(condDist$condVar))
```

<br>

#### <font color = "#880000">Comment</font>
<hr size="3" width="100%" noshade style="color:#440000" align="left" />
Keep in minds that some functions in R require standard deviation to be specified when working 
with the normal distribution (e.g. functions `dnorm()`, `pnorm()`, `qnorm()`, etc.) while others
need the variance-covariance matrix (variance in one dimension) to be provided instead. 

<hr size="3" width="100%" noshade style="color:#440000" align="left" />
<br>


#### <font color = "#880000"><b>3. Conditional normal distribution -- illustration example</b></font>

In this example we again consider the well-known dataset `mtcars` which is standardly available in the R environment (for a more detailed description of the dataset see the help session `?mtcars`). 

```{r}
summary(mtcars)
attach(mtcars)
```

For illustrative purposes only, we will simply assume that (continuous) covariates 'mpg' (miles per gallon), 'hp' (horse power) and 'disp' (displacement)
are all normaly distributed---the triple $(X_{1}, X_{2}, X_{3})^{\top} = (mpg, hp, disp)^{\top}$ follows a three dimensional normal distribution with some unknown mean vector $\boldsymbol{\mu} = (\mu_{1}, \mu_{2}, \mu_{3})^{\top} \in \mathbb{R}^{3}$ and some positive definite and symmetric variance-covariance matrix $\Sigma$. 

<ul>
<li>Use the R software to obtain the corresponding estimates for the mean vector $\boldsymbol{\mu} = (\mu_{1}, \mu_{2}, \mu_{3})^{\top} \in \mathbb{R}^{3}$ and the variance-covariance matrix $\Sigma$.</li>
</ul>

```{r, echo=FALSE, fig.width = 10}
par(mfrow = c(1,3))
hist(mpg, xlab = "Miles per gallon", ylab = "Frequency", col = "lightblue", freq = F)
lines(density(mpg, adjust = 2), col = "red", lwd = 2)
hist(hp, xlab = "Gross horsepower", ylab = "", col = "lightblue", freq = F)
lines(density(hp, adjust = 2), col = "red", lwd = 2)
hist(disp, xlab = expression(paste("Displacement [", cu^3, "]", sep = "")), ylab = "", col = "lightblue", freq = F)
lines(density(disp, adjust = 2), col = "red", lwd = 2)
```

What is the (estimated) distribution if the miles per gallon ('mpg') given the horse power and displacement, where $(X_{2}, X_{3})^{\top} = (120,120)^{\top}$? 

The corresponding estimates can be obtained by 

```{r}
mu <- apply(cbind(mpg, hp, disp), 2, mean) 
Sigma <- cov(cbind(mpg, hp, disp))
```

Thus, the corresponding conditional mean and variance are obtained as

```{r}
condMean <- mu[1] + Sigma[1, 2:3] %*% solve(Sigma[2:3,2:3]) %*% (c(120, 120) - mu[2:3])
condSigma <- Sigma[1,1] - Sigma[1, 2:3] %*% solve(Sigma[2:3,2:3]) %*% Sigma[2:3, 1]
```

and given the assumptions of multivariate normality for the triple $(X_{1}, X_{2}, X_{3})^{\top} = (mpg, hp, disp)^{\top}$ we get the univariate normal distribution with 
the following moments: 

```{r, fig.width = 10}
condMean
condSigma

plot(dnorm(x <- seq(0, 35, length = 500), mean = condMean, sd = sqrt(condSigma)) ~ x, 
                                           xlab = "Miles per gallon", ylab = "Conditional Density", 
                                           type = "l", lwd = 2, col = "red", 
                                           xlim = c(0, 35), ylim = c(0,0.18))
lines(dnorm(x, mean(mpg), sd(mpg)) ~ x, lwd = 1)
legend(0, 0.175, legend = c("Conditional distribution: MPG | HP = 120 & Disp = 120", 
                           "Estimated Normal Distribution of MPG"), col = c("red", "black"), lty = c(1,1))
```


#### <font color = "#880000">Individual task</font>
<hr size="3" width="100%" noshade style="color:#440000" align="left" />
<ul>
<li>Consider a simple linear regression model where the miles per gallon information is used as a dependent covariate and 
the horse power and the displacement value are both used as the explanatory variables. Estimate the model and interpret the estimated coefficient.
```{r, eval = F}
m <- lm(mpg ~ hp + disp, data = mtcars)
```
</li>

<li>Use the model above and estimate the expected value of the consuption (miles per gallon) for a car with the horse power being equal to 
120 and the displacement being 120 too. 
```{r, eval = F}
CondMean <- m$coeff[1] + m$coeff[2] * 120 + m$coeff[3] * 120
```
</li>

<li>Compare the conditional distribution you obtained above with the prediction obtained by the linear regression model. Discuss the results.</li>

</ul>
<hr size="3" width="100%" noshade style="color:#440000" align="left" />

<br><br>

#### <font color = "#880000">Theoretical Exercises</font>
<hr size="3" width="100%" noshade style="color:#440000" align="left" />
<ul>
<li><b>Exercise 1</b><br>
Let $\boldsymbol{X}_1 \sim N_{q}(\boldsymbol{\mu}_1, \boldsymbol{\Sigma}_{11})$ and $(\boldsymbol{X}_2 | \boldsymbol{X}_1 = \boldsymbol{x}_1) \sim N_{p - q}(\mathbb{A}\boldsymbol{x}_1 + \boldsymbol{b}, \boldsymbol{\Omega})$, where $\boldsymbol{\Omega}$ does not depend on $\boldsymbol{x}_1 \in \mathbb{R}^q$. Show, that the joint distribution of the whole $p$-dimensional random vector $(\boldsymbol{X}_1^\top, \boldsymbol{X}_2^\top)^\top$ satisfy 
$$
(\boldsymbol{X}_1^\top, \boldsymbol{X}_2^\top)^\top \sim N_{p}(\boldsymbol{\mu}, \boldsymbol{\Sigma}),  
$$
where 
$$
\boldsymbol{\mu} = \left(
\begin{array}{c}
\boldsymbol{\mu}_1\\
\mathbb{A}\boldsymbol{\mu}_1 + \boldsymbol{b}
\end{array}
\right) \qquad \textrm{and} \qquad \boldsymbol{\Sigma} = 
\left(
\begin{array}{cc}
\boldsymbol{\Sigma}_{11} & \boldsymbol{\Sigma}_{11} \mathbb{A}^\top\\
\mathbb{A}\boldsymbol{\Sigma}_{11} & \boldsymbol{\Omega} + \mathbb{A}\boldsymbol{\Sigma}_{11}\mathbb{A}^\top
\end{array}
\right).
$$

</li><br><br>

<hr size="3" width="100%" noshade style="color:#440000" align="left" />

<li><b>Exercise 2</b><br> Consider the random vector $\boldsymbol{X}$ with the two dimensional normal distribution $N_{2} \left( \left( \begin{array}{c}1\\2\end{array} \right),  \left( \begin{array}{cc} 2 & 1\\1 & 2\end{array} \right) \right)$ and the conditional random vector $\boldsymbol{Y} | \boldsymbol{X}$, such that 
$$
\boldsymbol{Y} | \boldsymbol{X}  \sim N_{2}\left( \left( \begin{array}{c} X_{1}\\ X_{1} + X_{2} \end{array}  \right), \left( \begin{array}{cc} 1 & 0\\ 0 & 1  \end{array}  \right)  \right).
$$

Determine the distribution of $Y_2 | Y_{1}$ and the distribution of $\boldsymbol{W} = \boldsymbol{X} - \boldsymbol{Y}$.</li><br><br>

<hr size="3" width="100%" noshade style="color:#440000" align="left" />

<li><b>Exercise 3</b><br> Consider a random vector $\boldsymbol{X} \sim N_{2}(\boldsymbol{\mu}, \Sigma)$, for $\boldsymbol{\mu} = (2,2)^\top$ and $\Sigma = \mathbb{I}_2$ (a unit matrix). Consider also matrices $\mathbb{A} = (1, 1)$ and $\mathbb{B} = (1, -1)$. Show, that the random variables $\mathbb{A}\boldsymbol{X}$ and $\mathbb{B}\boldsymbol{X}$ are independent.</li><br><br>

<hr size="3" width="100%" noshade style="color:#440000" align="left" />

<li><b>Exercise 4</b><br> Consider a random vector $(X, Y, Z)^\top \sim N_3(\boldsymbol{\mu}, \Sigma)$. Compute the values for $\boldsymbol{\mu}$ and $\Sigma$, if you know, that 

<ul>
<li>$Y | Z \sim N(-Z, 1)$;</li>
<li>$\mu_{Z|Y} = -\frac{1}{3} - \frac{1}{3}Y$;</li>
<li>$X | (Y, Z) \sim N(2 + 2Y + 3Z, 1)$;</li>
</ul>
Determine the conditional distribution of $X|Y$ and also $X | Y + Z$.
</li><br><br>

<hr size="3" width="100%" noshade style="color:#440000" align="left" />

<li><b>Exercise 5</b><br> Assume, you know the following: 
<ul>
<li>$Z \sim N(0,1)$;</li>
<li>$Y | Z \sim N(1 + Z, 1)$;</li>
<li>$X | (Y, Z) \sim N(1 - Y, 1)$;</li>
</ul>
Find the distribution of the random vector $(X, Y, Z)^\top$ and the conditional distribution of $Y | (X, Z)$. 
Consider the linear transformations $U = 1 + Z$ and $V = 1 - Y$ and find the joint distribution of the random vector $(U, V)^\top$. Calculate the value of $E(Y | U = 2)$.
</li>

</ul>
<hr size="3" width="100%" noshade style="color:#440000" align="left" />




<br><br>
<hr size="3" width="100%" noshade style="color:#440000" align="left" />
<hr size="3" width="100%" noshade style="color:#440000" align="left" /><br><br> 


## <font color = "#880000"><b>Homework Assignment</b></font>
### (Deadline: Lab session no. 4 / 08.03.2022)

<ul>
<li>Choose one theoretical example from the list above and solve it. Use the R simulations to empirically 
verify the results. Graphical outputs of the simulations are to be provided on the web site.  </li>

<li>As already mentioned above,  the joint normality of some random vector directly implies normality of all possible 
marginals. On the other hand, the normal distribution of all marginals does not necessarily imply the joint normality of the whole random vector. 

Try to propose a simple  example (using a two/three dimensional random vector $(X_1, X_2)^\top$  or $(X_1, X_2, X_3)^\top$ respectively)
showing that despite all normaly distributed marginals the whole random vector fails to follow the joint normal distribution. 

Think of some appropriate options to vizualize the whole problem. Graphical outputs are again to be provided on the web site. 
 </li>


<li> Provide the solution on you personal web page by <font color = "red"> Thuesday, 08.03.2022, 12:00 </font>at latest.</li>
</ul>




</td></tr></table>
<br><br><br>