SmartGame Library: Estimator weights in SgUctSearch

The weights of the estimators (move value, RAVE value) are chosen by assuming that the estimators are uncorrelated and modeling the mean squared error of estimator $ i $ by a function that depends on the number of samples and parameter constants, which represent the variance and bias of the estimator and need to be determined experimentally:

$w_i = \frac{1}{Z} \frac{1}{{\rm MSE}_i} \qquad Z = \sum_i \frac{1}{{\rm MSE}_i} \qquad {\rm MSE}_i = \frac{c_{\rm variance}}{N_i} + c_{\rm bias}^2$

Note that this formula is nearly equivalent to the formula suggested by David Silver on the Computer Go mailing list for the case of two estimators (move value and RAVE value) and used in newer versions of MoGo. However, MoGo uses the measured variance of the current RAVE value (for both the move weight and RAVE weight) instead variance parameter constants.

The formula is then reformulated to use different constants that describe the initial steepness and final asymptotic value of the unnormalized weight:

$Z w_i = \frac{c_{\rm initial}} {\frac{1}{N} + \frac{c_{\rm initial}}{c_{\rm final}}}$

with:

sample count of the estimator
$c_{\rm initial}$ Initial weight parameter; this is they weight if and $c_{\rm final} \gg c_{\rm initial}$
$c_{\rm final}$ Final weight parameter; this is they weight if $N \rightarrow \infty$

For the move value, $c_{\rm bias} = 0$ , and the variance can become part of the normalization constant, so the weight is simply $N_{\rm move}$ . If no estimator has a sample count yet, the first-play-urgency parameter is used for the value estimate.