P-value adjustments for Multiple Comparisons

* Ray
* I am sending you an improved version of the syntax for p-values
* adjustment algorithms. I have added some step-up methods and
* documentation for every algorithm.
* Marta Garcia-Granero 2002/04/17.

**************************
ADJUSTP SYNTAX:

DISCLAIMER

ADJUSTP is provided "as is" without warranty of any kind.
The entire risk as to the quality, performance, and fitness
for intended purpose is with you. You assume responsibility
for the selection of the program and for the use of results
obtained from that program.

DESCRIPTION

There are 8 p-value adjustment algorithms available. They include
one-step, step-down and step-up adjustment methods.

1. One-Step Bonferroni
2. One-Step Sidak [References 1 and 2]
3. Step-Down Holm [3]
4. Step-Down Sidak [1,2,4 and 5]
5. Step-Down Finner [4 and 5]
6. Step-Up Hommel [6 and 7]
7. Step-Up Hochberg [8]
8. Step-Up Simes [9]

All the results have been tested with Multiplicity program (2.0) ,
by Barry W. Brown, The University of Texas M D Anderson Cancer Center.

ALGORITHMS

The following nomenclature will be used:

n =     number of p-values
p(i)  = ith smallest p-value
p#(i) = adjusted value of p(i)

In one-step adjustment methods p-values are compared to a
predetermined value that is a function of alfa, the significance
level, and n, the number of p-values.

- One-step Bonferroni:
  p#(i)=n*p(i)

- One-Step Sidak:
  p#(i)=1-(1-p(i))^n

In step-down methods p-values are examined in order, from smallest
to largest. Once a p-value is found that is large according to a
criterion based on alfa and the p-value's position in the list, that
p-value and all larger p-values are accepted.

- Step-down Holm:
  p#(i)=(n-i+1)*p(i)

- Step-down Sidak:
  p#(i)=1-(1-p(i))^(n-i+1)

- Step-down Finner:
  p#(i)=1-(1-p(i))^(n/i)

In step-up methods p-values are examined in order, from largest to
smallest. Once a p-value is found that is small according to a
criterion based on alfa and the p-value's position in the list,
that p-value and all smaller p-values are rejected.

- Step-up Hommel:
  p#(i)=n*Cn*p(i)/i
  Cn=1+1/2+ ... +1/n

- Step-up Hochberg:
  p#(i)=(n-i+1)*p(i)

- Step-up Simes:
  p#(i)=n*p(i)/i

See [10] for a general reference on p-value adjustment algorithms.

REFERENCES

 1. Sidak Z (1967) "Rectangular Confidence Regions for the Means of
    Multivariate Normal Distributions" American Statistical Association,

    62, 626-633.
 2. Sidak Z (1971) "On probabilities of rectangles in multivariate
Student
    distributions: their dependence on correlations" Ann Math Statist,
42,
    169-175.
 3. Holm S (1979) "A Simple Sequentially Rejective Multiple Test
Procedure"
    Scandinavian Journal of Statistics, 6, 65-70.
 4. Finner H (1990) "Some New Inequalities for the Rnad Distribution
With
    Application to the determination of Optimum Significance Levels of
    Multiple Range Tests" Journal of the American Statistical
Association,
    85, 191-194.
 5. Finner H (1993) "On A Monotonicity Problem in Step-Down Multiple
Test
    Procedures" Journal of the American Statistical Association, 88,
920-923.
 6. Hommel G (1988) "A stagewise rejective multiple test procedure based
on
    a modified Bonferroni test" Biometrika, 75, 383-386.
 7. Falk RW (1989) "Hommel's Bonferroni-type inequality for unequally
    spaced levels" Biometrika, 76, 190-191.
 8. Hochberg Y and Benjamini Y (1990) "More powerful procedures for
multiple
    significance testing" Statistics in Medicine, 9, 811-818.
 9. Simes RJ (1986) "An improved Bonferroni procedure for multiple tests

    of significance" Biometrika, 73, 751-754.
10. Westfall PH and Young SS (1993), Resampling-Based Multiple Testing:
    examples and methods for p-value adjustment. New York: John Wiley
and Sons.

********************************************************************************

* BEGINNING OF SYNTAX

* Create dataset with p-values. Replace by your own *.
DATA LIST list / pvalue(f9.4).
BEGIN DATA
0.0728
0.0023
0.3829
0.0041
0.0101
0.4557
END DATA.

* SOME AUXILIARY VARIABLES *.
COMPUTE id=$CASENUM.
FORMAT id (F2.0).
SORT CASES BY  pvalue (A) .
COMPUTE pos=$CASENUM.
FORMAT pos (F2.0).
*>>>>> This section added by Ray Levesque <<<<<<<.
* Calculate the number of p values.
RANK pvalue /n into N /PRINT=NO.
* N contains the number of cases in the file.
*>>>>>> End of section added by Ray <<<<<<<.

* ADJUSTED P-VALUES *.
* (1) One step methods *.
COMPUTE bonferr=pvalue*n.
IF bonferr>1 bonferr=1.
COMPUTE sidak=1-(1-pvalue)**n.
* (2) Step-down methods *.
COMPUTE holm=(n-pos+1)*pvalue.
IF holm>1 holm=1.
IF (holm<LAG(holm,1)) holm=LAG(holm,1).
COMPUTE downsidk=1-(1-pvalue)**(n-pos+1).
IF (downsidk<LAG(downsidk,1)) downsidk=LAG(downsidk,1).
COMPUTE finner=1-(1-pvalue)**(n/pos).
IF (finner<LAG(finner,1)) finner=LAG(finner,1).
* (3) Step-up methods *.
COMPUTE cn=1/pos.
DO IF $casenum>1.
COMPUTE cn=cn+lag(cn,1).
END IF.
SORT CASES BY pos(D).
IF cn<lag(cn,1) cn=lag(cn,1).
COMPUTE hommel = cn*n*pvalue/pos.
IF hommel>1 hommel=1.
IF (hommel>LAG(hommel,1)) hommel=LAG(hommel,1).
COMPUTE hochberg=(n-pos+1)*pvalue.
IF (hochberg>LAG(hochberg,1)) hochberg=LAG(hochberg,1).
COMPUTE simes=n*pvalue/pos.
IF (simes>LAG(simes,1)) simes=LAG(simes,1).

* FINAL REPORTS *.
FORMAT bonferr to simes (f9.4).
VARIABLE LABELS id 'Nr.' /pvalue 'Original' /pos 'Rank'
        /bonferr 'One-step Bonferroni' /sidak 'One-step Sidak'
        /holm 'Step-down Holm' /downsidk 'Step-down Sidak'
        /finner 'Step-down Finner' /hommel 'Step-up Hommel'
        /hochberg 'Step-up Hochberg' /simes 'Step-up Simes'.
SORT CASES BY pos (A).
REPORT FORMAT=LIST AUTOMATIC ALIGN(CENTER)
  /VARIABLES=pos id pvalue bonferr sidak
  /TITLE "Original and one-step adjusted p-values".
REPORT FORMAT=LIST AUTOMATIC ALIGN(CENTER)
  /VARIABLES=pos id pvalue holm downsidk finner
  /TITLE "Original and step-down adjusted p-values".
REPORT FORMAT=LIST AUTOMATIC ALIGN(CENTER)
  /VARIABLES=pos id pvalue hommel hochberg simes
  /TITLE "Original and step-up adjusted p-values".

********************************************************************************

* END OF SYNTAX.