Custom Search

Statistical tests are used to make inferences about a population based on data that are obtained from a sample of that population. Statistical tests evaluate the relationship between two or more variables that are measured in a sample. In the context of adverse impact, statistical tests assess the relationship between group membership (e.g., a particular race or sex) and decision outcome (e.g., pass/fail, hired, promoted). One does not know if adverse impact truly exists in some defined population. Therefore, we make the best decision we can based on the results obtained in a sample. Statistical test of adverse impact estimate the probability of obtaining the observed sample results assuming there is no relationship between group membership and outcome in the population. Statistical tests of adverse impact test the following hypothesis (or null hypothesis): There is no relationship between group membership and decision outcome (i.e., subgroups do not differ in decision outcome; there is no adverse impact); any observed difference is due to chance. Given this null hypothesis, there are four possible decision outcomes as shown in the table below:
Correct acceptance: Correctly accepting the null hypothesis. The truth (which is unknown) is that the population does not have adverse impact and it is decided based on the results of the statistical test that there is no adverse impact. Power: Correctly rejecting the null hypothesis. The truth (which is unknown) is that the population does have adverse impact and it is decided based on the results of the statistical test that there is adverse impact. Type I error:
Incorrectly rejecting the null hypothesis. The truth (which is unknown) is that
the population does not have adverse impact and it is decided based on the
results of the statistical test that there is adverse impact. This is sometimes
referred to as alpha error (or Type II error: Incorrectly accepting the null hypothesis. The truth (which is unknown) is that
the population does have adverse impact and it is decided based on the results
of the statistical test that there is no adverse impact. This is sometimes
referred to as beta error (or A statistically
significant result is one in which the probability of incorrectly concluding
that adverse impact exists (i.e., a Type I error) is less than a specified
level; this specified level is referred to as an alpha level (or The choice of alpha level is somewhat arbitrary
and it should be determined based on the question or relationship that is being
analyzed. However, behavioral scientists have historically and
consistently chosen an alpha level of .05. In addition, in the context of
adverse impact, an alpha level of .05 appears to be the level recommended by the
Uniform Guidelines ( One disadvantage
of statistical tests, compared to the impact ratio, is that statistical tests
only indicate the likelihood with which the differences are due to chance; they
do not describe the magnitude of the selection rate differences or describe how
meaningful the differences are (e.g., trivial differences can be significant
when the sample size is large, and meaningful differences can be non-significant
when the sample size is small; see Use |