Calculates PCA leverage or robust distance and identifies outliers.
clever(
X,
projection = "PCA_kurt",
out_meas = "leverage",
DVARS = TRUE,
detrend = TRUE,
PCATF_kwargs = NULL,
kurt_quantile = 0.95,
id_outliers = TRUE,
lev_cutoff = 4,
rbd_cutoff = 0.9999,
R_true = NULL,
lev_images = FALSE,
verbose = FALSE
)| X | Numerical data matrix. Should be wide (N observations x P variables, \(N >> P\)). |
|---|---|
| projection | Character vector indicating the projection methods
to use. Choose at least one of the following: |
| out_meas | Character vector indicating the outlyingness measures to
compute. Choose at least one of the following: |
| DVARS | Should DVARS (Afyouni and Nichols, 2017) be computed too? Default
is |
| detrend | Detrend the PCs before measuring kurtosis or computing
leverage or robust distance? Default: Detrending is highly recommended for time-series data, especially if there are many time points or evolving circumstances affecting the data. There are two reasons: first, temporal trends induce positive or negative kurtosis, contaminating the connection between high kurtosis and outlier presence. Second, trends tend to reduce the size of the in-MCD subset for the robust distance method causing many false positives. Detrending should not be used with non-time-series data because the observations are not temporally related. In addition to This option will not affect the PCATF projection, which is never detrended. |
| PCATF_kwargs | Named list of arguments for PCATF projection method.
Only applies if Valid entries are:
|
| kurt_quantile | What cutoff quantile for kurtosis should be used? Only
applies if |
| id_outliers | Should the outliers be identified? Default: |
| lev_cutoff | The outlier cutoff value for leverage, as a multiple of the
median leverage. Only used if
|
| rbd_cutoff | The outlier cutoff quantile for MCD distance. Only used if
The quantile is computed from the estimated F distribution. |
| R_true | The N x N correlation matrix, if known. Used for the bootstrap robust distance method. |
| lev_images | Should leverage images be computed? If |
| verbose | Should occasional updates be printed? Default: |
A clever object, i.e. a list with components
A list of all the arguments used.
The indices retained from the original SVD projection to make the variance-based PC projection.
The PC projection.
The indices retained from the original SVD projection to make the kurtosis-based PC projection. They are ordered from highest kurtosis to lowest kurtosis.
The PC projection. PCs are ordered in the standard way, from highest variance to lowest variance, instead of by kurtosis.
The indices of the trend-filtered PCs used to make the projection.
The PCATF result.
The leverage values for the PC_var projection.
The leverage values for the PC_kurt projection.
The leverage values for the PCATF projection.
The robust MCD distance values for the PC_var projection.
The robust MCD distance values for the PC_kurt projection.
The Delta percent DVARS values.
The DVARS z-scores.
The leverage cutoff for outlier detection: lev_cutoff times
the median leverage.
The robust distance cutoff for outlier detection: the
rbd_cutoff quantile of the estimated F distribution.
The Delta percent DVARS cutoff: +/- 5 percent
The DVARS z-score cutoff: the one-sided 5 percent significance level with Bonferroni FWER correction.
Logical vector idnicating whether each observation surpasses the outlier cutoff.
Logical vector idnicating whether each observation surpasses the outlier cutoff.
Logical vector idnicating whether each observation surpasses the outlier cutoff.
Logical vector idnicating whether each observation surpasses the outlier cutoff.
Logical vector idnicating whether each observation surpasses the outlier cutoff.
Logical vector idnicating whether each observation surpasses the outlier cutoff.
Logical vector idnicating whether each observation surpasses the outlier cutoff.
Logical vector indicating whether each observation was in the MCD estimate.
The scale for out-of-MCD observations.
Named numeric vector: c, m, df1, and df2.
Logical vector indicating whether each observation was in the MCD estimate.
The scale for out-of-MCD observations.
Named numeric vector: c, m, df1, and df2.
The scale value for out-of-MCD observations, and NA for
in-MCD observations. NULL if method is not robust distance.
The average of the PC directions, weighted by the unscaled PC scores at each outlying time point (U[i,] * V^T). Row names are the corresponding time points.
The PC direction with the highest PC score at each outlying time point. Row names are the corresponding time points.
The index of the PC direction with the highest PC score at each outlying time point. Named by timepoint.
clever will use all combinations of the requested projection and
out_meas methods that make sense. For example, if
projection=c("PCATF", "PCA_var", "PCA_kurt") and
out_meas=c("leverage", "robdist") then these five
combinations will be used: PCATF with leverage, PCA + variance with
leverage, PCA + variance with robust distance, PCA + kurtosis with leverage,
and PCA + kurtosis with robust distance. Each method combination will yield
its own out_meas time series.