class: title-slide <br> <br> .right-panel[ # Grid Approximation ## Dr. Mine Dogucu Examples from [bayesrulesbook.com](https://bayesrulesbook.com) ] --- class: middle ## Specifying the Posterior `\(Y | \pi \sim \text{Bin}(10, \pi)\)` `\(\pi \sim \text{Beta}(2,2)\)` -- Suppose `\(Y = 9\)` -- Then `\(\pi|(Y = 9) \sim \text{Beta}(11,3)\)` --- ## Step 1: Define a grid of 6 pi values ```r pi_grid <- seq(from = 0, to = 1, length = 6) pi_grid ``` ``` ## [1] 0.0 0.2 0.4 0.6 0.8 1.0 ``` ```r grid_data <- data.frame(pi_grid) ``` --- ### Step 2: Evaluate the prior and likelihood at each pi ```r grid_data <- grid_data %>% mutate(prior = dbeta(pi_grid, 2, 2)) ``` -- .pull-left[ ```r grid_data ``` ``` ## pi_grid prior ## 1 0.0 0.00 ## 2 0.2 0.96 ## 3 0.4 1.44 ## 4 0.6 1.44 ## 5 0.8 0.96 ## 6 1.0 0.00 ``` ] -- .pull-right[ <img src="slide-05a-grid-app_files/figure-html/unnamed-chunk-6-1.png" style="display: block; margin: auto;" /> ] --- ### Step 2: Evaluate the prior and likelihood at each pi ```r grid_data <- grid_data %>% mutate(prior = dbeta(pi_grid, 2, 2)) %>% mutate(likelihood = dbinom(9, 10, pi_grid)) ``` -- .pull-left[ ```r grid_data ``` ``` ## pi_grid prior likelihood ## 1 0.0 0.00 0.000000000 ## 2 0.2 0.96 0.000004096 ## 3 0.4 1.44 0.001572864 ## 4 0.6 1.44 0.040310784 ## 5 0.8 0.96 0.268435456 ## 6 1.0 0.00 0.000000000 ``` ] -- .pull-right[ <img src="slide-05a-grid-app_files/figure-html/unnamed-chunk-9-1.png" style="display: block; margin: auto;" /> ] --- ## Step 3: Approximate the posterior ```r grid_data <- grid_data %>% mutate(unnormalized = likelihood * prior) grid_data ``` ``` ## pi_grid prior likelihood unnormalized ## 1 0.0 0.00 0.000000000 0.000000e+00 ## 2 0.2 0.96 0.000004096 3.932160e-06 ## 3 0.4 1.44 0.001572864 2.264924e-03 ## 4 0.6 1.44 0.040310784 5.804753e-02 ## 5 0.8 0.96 0.268435456 2.576980e-01 ## 6 1.0 0.00 0.000000000 0.000000e+00 ``` --- ## Step 3: Approximate the posterior ```r grid_data <- grid_data %>% mutate(unnormalized = likelihood * prior) %>% mutate(posterior = unnormalized / sum(unnormalized)) grid_data ``` ``` ## pi_grid prior likelihood unnormalized posterior ## 1 0.0 0.00 0.000000000 0.000000e+00 0.000000e+00 ## 2 0.2 0.96 0.000004096 3.932160e-06 1.236472e-05 ## 3 0.4 1.44 0.001572864 2.264924e-03 7.122080e-03 ## 4 0.6 1.44 0.040310784 5.804753e-02 1.825311e-01 ## 5 0.8 0.96 0.268435456 2.576980e-01 8.103344e-01 ## 6 1.0 0.00 0.000000000 0.000000e+00 0.000000e+00 ``` --- ## Step 3: Approximate the posterior ```r grid_data <- grid_data %>% mutate(unnormalized = likelihood * prior) %>% mutate(posterior = unnormalized / sum(unnormalized)) ``` Confirm that the posterior approximation sums to 1 ```r grid_data %>% summarize(sum(unnormalized), sum(posterior)) ``` ``` ## sum(unnormalized) sum(posterior) ## 1 0.3180144 1 ``` --- ## The Grid Approximated Posterior .pull-left[ ```r ggplot(grid_data, aes(x = pi_grid, y = posterior)) + geom_point() + geom_segment(aes(x = pi_grid, xend = pi_grid, y = 0, yend = posterior)) ``` ] .pull-right[ <img src="slide-05a-grid-app_files/figure-html/unnamed-chunk-15-1.png" style="display: block; margin: auto;" /> ] --- ## Step 4: Sample from the Discretized Posterior ```r set.seed(84735) post_sample <- sample_n(grid_data, size = 10000, weight = posterior, replace = TRUE) ``` -- ```r post_sample %>% tabyl(pi_grid) %>% adorn_totals("row") ``` ``` ## pi_grid n percent ## 0.4 69 0.0069 ## 0.6 1885 0.1885 ## 0.8 8046 0.8046 ## Total 10000 1.0000 ``` --- ## Grid Simulation with the Posterior pdf .pull-left[ ```r ggplot(post_sample, aes(x = pi_grid)) + geom_histogram(aes(y = ..density..), color = "white") + stat_function(fun = dbeta, args = list(11, 3)) + lims(x = c(0, 1)) ``` ] .pull-right[ <img src="slide-05a-grid-app_files/figure-html/unnamed-chunk-19-1.png" style="display: block; margin: auto;" /> ] --- Let's increase the possible `\(\pi\)` values from 6 to 101. ```r pi_grid <- seq(from = 0, to = 1, length = 101) ``` --- ## Grid simulation with posterior pdf .pull-left[ ```r ggplot(post_sample, aes(x = pi_grid)) + geom_histogram(aes(y = ..density..), color = "white") + stat_function(fun = dbeta, args = list(11, 3)) + lims(x = c(0, 1)) ``` ] .pull-right[ <img src="slide-05a-grid-app_files/figure-html/unnamed-chunk-24-1.png" style="display: block; margin: auto;" /> ] --- ## Grid approximation <img src="slide-05a-grid-app_files/figure-html/unnamed-chunk-25-1.png" style="display: block; margin: auto;" /> --- ### Grid approximation Grid approximation produces a sample of `\(N\)` __independent__ `\(\theta\)` values, `\(\left\lbrace \theta^{(1)}, \theta^{(2)}, \ldots, \theta^{(N)} \right\rbrace\)`, from a __discretized approximation__ of posterior pdf `\(f(\theta|y)\)`. This algorithm evolves in four steps: 1. Define a discrete grid of possible `\(\theta\)` values. 2. Evaluate the prior pdf `\(f(\theta)\)` and likelihood function `\(L(\theta|y)\)` at each `\(\theta\)` grid value. 3. Obtain a discrete approximation of posterior pdf `\(f(\theta |y)\)` by: (a) calculating the product `\(f(\theta)L(\theta|y)\)` at each `\(\theta\)` grid value; and then (b) *normalizing* the products so that they sum to 1 across all `\(\theta\)`. 4. Randomly sample `\(N\)` `\(\theta\)` grid values with respect to their corresponding normalized posterior probabilities.