Methods

The main steps in CorShrink are as follows

• Convert the correlation between variables \(i\) and \(j\), \(R_{ij}\) into Fisher z-score \(Z_{ij}\) by the following transformation.

\[ Z_{ij} = 0.5 \log \left (\frac{1 + R_{ij}}{1 - R_{ij}} \right ) \]

• Estimate from data, the standard errors (\(s_{ij}\)) of these Z-scores \(Z_{ij}\). This can be done in two ways in CorShrink. One approach uses an asymptotic normal approximation, where the standard errors are \(s_{ij} = \frac{1}{n_{ij} - 3}\), with \(n_{ij}\) being the number of complete observations between pair \((i,j)\) that generates the correlation \(R_{ij}\). The other approach performs a re-sampling of the observations for the \((i,j)\) pair and obtains a Bootstrap estimate of the standard errors from the re-sampled \(Z_{ij}\).

• Apply adaptive shrinkage (ash due to Stephens 2016) on the pairs \((Z_{ij}, s_{ij})\) either across all \(i\) and \(j\) pairs (matrix format) or along all \(i\) for one \(j\), or along all \(j\) for one \(i\) (vector formats).

\[ Z^{\star}_{ij} : = ash \; (Z_{ij}, s_{ij}) \]

The matrix format shrinkage is performed by the CorShrinkMatrix function while the vector format shrinkage is performed by the CorShrinkVector function.

• Reverse transform the posterior mean of the shrunk Fisher z-scores from the previous step.

\[ R^{\star}_{ij} = \frac{exp \; (2 Z^{\star}_{ij}) - 1}{exp \; (2 Z^{\star}_{ij}) + 1} \]

• If the user is attempting to estimate a correlation matrix ((R_{ij})) for a number of variables \((i,j)\), then a requisite condition is positive definiteness of the estimate. However, the \(((R^{\star}_{ij}))\) matrix obtained above may not be positive definite. We work around this problem by choosing a psoitive definite matrix nearest to the above estimate (see Higham 2002).

This R Markdown site was created with workflowr