Probability

Draw random numbers uniformly from $[0,1]$, one at a time, and keep a running sum. Stop the moment the sum first exceeds 1. Let $X_N$ denote the last number drawn — the one that pushed the total over the edge, the proverbial last straw. The question:

$$
E[X_N] = \ ?
$$

At first glance this looks innocent. Every draw is $\mathrm{U}(0,1)$ with mean $\tfrac12$, so shouldn’t the answer be something boring like $\tfrac12$? It isn’t. The last draw is special: it was selected by the stopping rule, and that selection systematically favors larger numbers. Teasing out exactly how much larger takes us through one of the most elegant corners of elementary probability — and along the way, $e$ makes three separate appearances.

设

$$
P(x) \sim \mathcal{N}(\mu_1, \sigma_1^2), \quad Q(x) \sim \mathcal{N}(\mu_2, \sigma_2^2)
$$

定义

$$
D_{\mathrm{KL}}(P\lVert Q) = \int P(x) \log \frac{P(x)}{Q(x)} dx
$$

则有

$$
\boxed{
D_{\mathrm{KL}}(P\lVert Q) = \ln \frac{\sigma_2}{\sigma_1} + \frac{(\sigma_1^2 - \sigma_2^2) + (\mu_1 - \mu_2)^2}{2\sigma_2^2}
}
$$

Step 1. 分布比值

$$
\frac{P(x)}{Q(x)} = \frac{\sigma_2}{\sigma_1} \exp \left[ \frac{\sigma_1^2(x - \mu_2)^2 - \sigma_2^2(x - \mu_1)^2}{2\sigma_1^2\sigma_2^2} \right]
$$

因此

$$
\log \frac{P(x)}{Q(x)} = \ln \frac{\sigma_2}{\sigma_1} + \frac{(\sigma_1^2 - \sigma_2^2)x^2 + (2\sigma_2^2\mu_1 - 2\sigma_1^2\mu_2)x + (\sigma_1^2\mu_2^2 - \mu_1^2\sigma_2^2)}{2\sigma_1^2\sigma_2^2}
$$