(D) Let $I = \int_{0}^{\frac{\pi}{2}} \sin 2x \tan^{-1}(\sin x) dx = \int_{0}^{\frac{\pi}{2}} 2 \sin x \cos x \tan^{-1}(\sin x) dx$.
Let $\sin x = t$,then $\cos x dx = dt$.
When $x = 0, t = 0$ and when $x = \frac{\pi}{2}, t = 1$.
Thus,$I = 2 \int_{0}^{1} t \tan^{-1}(t) dt$.....$(1)$.
Using integration by parts for $\int t \tan^{-1} t dt$:
$\int t \tan^{-1} t dt = \tan^{-1} t \cdot \frac{t^2}{2} - \int \frac{1}{1+t^2} \cdot \frac{t^2}{2} dt$
$= \frac{t^2 \tan^{-1} t}{2} - \frac{1}{2} \int \frac{t^2+1-1}{1+t^2} dt$
$= \frac{t^2 \tan^{-1} t}{2} - \frac{1}{2} \int (1 - \frac{1}{1+t^2}) dt$
$= \frac{t^2 \tan^{-1} t}{2} - \frac{1}{2} (t - \tan^{-1} t) = \frac{t^2 \tan^{-1} t}{2} - \frac{t}{2} + \frac{1}{2} \tan^{-1} t$.
Evaluating the definite integral:
$\int_{0}^{1} t \tan^{-1} t dt = [\frac{t^2 \tan^{-1} t}{2} - \frac{t}{2} + \frac{1}{2} \tan^{-1} t]_{0}^{1}$
$= (\frac{1 \cdot \frac{\pi}{4}}{2} - \frac{1}{2} + \frac{1}{2} \cdot \frac{\pi}{4}) - (0) = \frac{\pi}{8} - \frac{1}{2} + \frac{\pi}{8} = \frac{\pi}{4} - \frac{1}{2}$.
Substituting back into equation $(1)$:
$I = 2 \times (\frac{\pi}{4} - \frac{1}{2}) = \frac{\pi}{2} - 1$.