Applied Bayesian Data Analysis
Bayesian
differential equations
Concepts:
- Ordinary differential equations
- Initial value problem
- Boundary value problem
David Tolpin, david.tolpin@gmail.com
Initival value problem
- First order IVP:
- Given $y'(t) = f(t,y(t)), y(t_0)=y_0$
- Find $y(t_n)$.
- Higher order IVP can be converted to
first order, for example
$$y'' = -y$$
can be converted to
\begin{align}
y' &= z \\
z' &= -y
\end{align}
Solving IVP
- Closed form: $y = A\sin(x) +
B\cos(x)$
- In most cases, no closed form
solution — numerical methods.
Numerical methods
- Euler method:
\begin{align}
t_0, t_1 & = t_0 + h, t_2 = t_0 + 2h,... \\
y_{n+1} &= y_n + hf(t_n,y_n)
\end{align}
-
Backward Euler method:
$$y_{n+1} = y_n + hf(t_{n+1},y_{n+1})$$
-
Multipoint methods (Runge-Kutta)
Runge-Kutta method
$$ y_{n+1} = y_n + \frac{1}{6}h\left(k_1 + 2k_2 + 2k_3 + k_4\right) $$
where
\begin{align}
k_1 &= \ f(t_n, y_n), \\
k_2 &= \ f\left(t_n + \frac{h}{2}, y_n + h\frac{k_1}{2}\right), \\
k_3 &= \ f\left(t_n + \frac{h}{2}, y_n + h\frac{k_2}{2}\right), \\
k_4 &= \ f\left(t_n + h, y_n + hk_3\right).
\end{align}
Bondary value problem
- Shooting problem:
- Given $y'(t) = f(t,y(t)), y(t_n)=y_n$
- Find $y(t_0)$.
- In general:
- Given $y'(t) = f(t,y(t),
\theta), y(t_n)=y_n$
- Find $\theta$.
Bayesian BVP
- Given:
\begin{align}
\theta & \sim Prior \\
y'(t) & = f(t, y(t), \theta) \\
z(t_i) & \sim Conditional(y(t_i))
\end{align}
- Find $\theta| z$
Challenges
- Stochastic differential equations
- Stiff problems
Stochastic differential equations
Stiff problems
- Differential equations for which explicit numerical methods are unstable.
- Example:$y'(t) = -15y(t)$
Lecture on stiff IVPs
Hands-on
- Ballistic trajectory
- Lotka-Volterra