**************************************************************************** *** Meta-Analysis: Fixed and Random Effects Models *** Valentim R. Alferes (University of Coimbra, Portugal) *** valferes@fpce.uc.pt ** ** This syntax does a meta-analysis on a set of studies comparing two ** independent means. It produces results for both fixed and random effects ** models, using Cohen's d statistic, with or without Hedges' correction. ** ** The user has TEN MODES FOR ENTERING SUMMARY DATA (see PART 1): ** ** Mode 1 - Study No., N1, M1, SD1, N2, M2 SD2. ** Mode 2 - Study No., N1, M1, N2, M2 SD_POOL. ** Mode 3 - Study No., Direction of Effect, Difference, N1, SD1, N2, SD2. ** Mode 4 - Study No., Direction of Effect, Difference, N1, N2, SD_POOL. ** Mode 5 - Study No., DF, M1, SD1, M2 SD2. ** Mode 6 - Study No., DF, M1, M2, SD_POOL. ** Mode 7 - Study No., Direction of Effect, DF, Difference, SD1, SD2. ** Mode 8 - Study No., Direction of Effect, DF, Difference, SD_POOL. ** Mode 9 - Study No., Direction of Effect, N1, N2, T_OBS. ** Mode 10 - Study No., Direction of Effect, DF, T_OBS. ** ** There are no limits for the number of studies to be analyzed and the user ** can input data simultaneously in the ten modes or enter all the studies ** only in one mode. In the modes not used, the lines of data have to be ** cleared, but not the corresponding command lines. ** ** If the input are means, the program assumes that Group 1 is the ** experimental or focus group and Group 2 is the control or comparison ** group. ** ** If the input are differences between group means or observed Ts, they ** are registered in absolute values (DIF=|M1-M2| or T_OBS=|Tobs|) and the ** user specifies the direction of effect in a different variable (DIRECT): ** +1 (if the effect is in the expected direction: Group 1 mean greater ** than Group 2 mean) and -1 (if the effect is reversed: Group 1 mean lesser ** than Group 2 mean). ** ** When the input are degrees of freedom, the syntax asssumes equal Ns if df ** are even, and N2=N1-1 if they are odd. ** ** When the data are selected from two contrasting ANOVA treatments, the ** user can input them in modes 2 or 4 and let the pooled standard deviation ** (SD_POOL) equals the squared root of the original ANOVA MS Error. ** ** By default the measure of effect size is Hedges' correction to Cohen's d. ** If you want to use d statistic without correction, you can change the ** default in the corresponding command line. ** ** The OUTPUT is organized in nine tables: ** ** Table 1  User's data ** ** Table 2  Program imputations ** ** Table 3  Individual T Tests and observed power ** - N1, N2, degrees of freedom (DF), difference between group means (DIF), ** observed T (T_OBS), two-tailed probability (P_TWO), and one-tailed ** probability (P_ONE); ** - Alfa (ALFA), Harmonic N (N_HARM), noncentrality parameter (NCP), and ** observed power (OPOWER). ** [for algorithm, see Borenstein et al., 2001] ** ** Table 4  Measures of Effect Size and Nonoverlap ** Measures of effect size: ** - Cohen's d (D); ** [Cohen, 1988, p. 20] ** - Hedges' correction (D_H); ** [D_H = d, in Hedges & Olkin, 1985; D_H = d*, in Hunter & Schmidt, ** 1990; see Cortina & Nouri, 2000, p. 9]; ** - r point biserial (R); ** - Squared r point biserial (R2); ** - Binomial Effect Size Display (BESD_LO and BESD_UP). ** [see formulas in Rosenthal et al. 2000, pp. 8-19] ** ** Measures of nonoverlap: ** - U1 (percent of nonoverlap between the two distributions); ** - U2 (the highest percent in Group 1 that exceeds the same lowest ** percent in Group 2); ** - U3 (percentile standing = percentile of the Group 2 distribution ** corresponding to the 50th percentile of Group 1 distribution); ** [see formulas in Cohen, 1988, pp. 21-23] ** ** Table 5 - Non weighted effect size - Descriptive statistics ** - Number of studies (NSTUDIES), Cohen's d (D), and Hedges' correction ** (D_H) (minimun, maximun, mean, sem, and sd). ** ** Table 6  Fixed effects model ** - Weighted average effect size (EF_SIZE), VARIANCE, and standard error ** (SE); ** - z Test (z), two-tailed probability (P_TWO), and one-tailed probability ** (P_ONE); ** - Confidence level (CL), and lower (CI_LOWER) and upper (CI_UPPER) ** interval confidence limits. ** [see formulas in Shadish & Haddock, 1994, pp. 265-268] ** ** Table 7 - Chi-square Test for homogeneity of effect size: ** - Q statistic, degrees of freedom (K), and two-tailed probability ** (P_CHISQ) ** [see formula in Shadish & Haddock, 1994, p. 266] ** ** Table 8 - Random Variance Component ** - V0 [see formula in Lipsey & Wilson, 2001, p. 134]. ** ** Table 9  Random effects model ** - Weighted average effect size (EF_SIZE), VARIANCE, and standard error ** (SE); ** - z Test (z), two-tailed probability (P_TWO), and one-tailed probability ** (P_ONE); ** - Confidence level (CL), and lower (CI_LOWER) and upper (CI_UPPER) ** interval confidence limits. ** [see formulas and procedures in Lipsey & Wilson, 2001, pp. 134-135] ** ** For calculating observed power of individual studies, the syntax assumes ** alfa = 0.05. For calculating the confidence interval of weighted effect ** sizes, the syntax assumes confidence level = 95%. If you want, you can ** modify these values in the corresponding lines (see PART 2). ** ** After running the syntax, the user can have access to Tables 2, 3 and 4 ** in SPSS active file, so that he may handle the data for other meta- ** analytic procedures based on different effect size measures or exact ** probabilities (see other syntaxes in this site). ** ** In the example, we have 20 studies and we have used the ten input data ** modes. **************************************************************************** *** BEGIN OF THE SYNTAX. ** PART 1: ENTERING SUMMARY DATA. * Mode 1: Enter, row by row, Study No., N1, M1, SD1, N2, M2 SD2. DATA LIST LIST /Study(F8.0) N1(F8.0) M1(F8.2) SD1(F8.2) N2(F8.0) M2(F8.2) SD2(F8.2). BEGIN DATA 1 17 7.46 1.98 16 6.23 2.45 2 15 5.34 2.14 15 4.47 2.51 END DATA. SAVE OUTFILE=DATA1. * Mode 2: Enter, row by row, Study No., N1, M1, N2, M2 SD_POOL. DATA LIST LIST /Study(F8.0) N1(F8.0) M1(F8.2) N2(F8.0) M2(F8.2) SD_POOL(F8.2). BEGIN DATA 3 14 7.32 16 8.23 2.67 4 23 6.20 27 4.47 2.21 END DATA. SAVE OUTFILE=DATA2. * Mode 3: Enter, row by row, Study No., Direction of Effect, Difference, * N1, SD1, N2, SD2. DATA LIST LIST /Study(F8.0) Direct(F8.0) DIF(F8.2) N1(F8.0) SD1(F8.2) N2(F8.0) SD2(F8.2). BEGIN DATA 5 +1 1.04 10 3.04 11 2.98 6 -1 2.25 12 2.63 12 2.21 END DATA. SAVE OUTFILE=DATA3. * Mode 4: Enter, row by row, Study No., Direction of Effect, Difference, N1, * N2, SD_POOL. DATA LIST LIST /Study(F8.0) Direct(F8.0) DIF(F8.2) N1(F8.0) N2(F8.0) SD_POOL(F8.2). BEGIN DATA 7 -1 1.32 34 33 2.44 8 +1 1.25 20 20 3.09 END DATA. SAVE OUTFILE=DATA4. * Mode 5: Enter, row by row, Study No., DF, M1, SD1, M2 SD2. DATA LIST LIST /Study(F8.0) DF(F8.0) M1(F8.2) SD1(F8.2) M2(F8.2) SD2(F8.2). BEGIN DATA 9 34 7.46 1.69 6.33 2.98 10 33 5.34 2.94 5.46 2.31 END DATA. SAVE OUTFILE=DATA5. * Mode 6: Enter, row by row, Study No., DF, M1, M2, SD_POOL. DATA LIST LIST /Study(F8.0) DF(F8.0) M1(F8.2) M2(F8.2) SD_POOL(F8.2). BEGIN DATA 11 27 7.76 5.29 2.77 12 28 6.30 4.21 2.41 END DATA. SAVE OUTFILE=DATA6. * Mode 7: Enter, row by row, Study No., Direction of Effect, DF, Difference, * SD1, SD2. DATA LIST LIST /Study(F8.0) Direct(F8.0) DF(F8.0) DIF(F8.2) SD1(F8.2) SD2(F8.2). BEGIN DATA 13 +1 40 3.07 1.77 2.87 14 -1 37 2.11 2.62 2.21 END DATA. SAVE OUTFILE=DATA7. * Mode 8: Enter, row by row, Study No., Direction of Effect, DF, Difference, * SD_POOL. DATA LIST LIST /Study(F8.0) Direct(F8.0) DF(F8.0) DIF(F8.2) SD_POOL(F8.2). BEGIN DATA 15 -1 23 2.22 1.88 16 +1 34 3.17 1.94 END DATA. SAVE OUTFILE=DATA8. * Mode 9: Enter, row by row, Study No., Direction of Effect, N1, N2, T_OBS. DATA LIST LIST /Study(F8.0) Direct(F8.0) N1(F8.0) N2(F8.0) T_OBS(F8.2). BEGIN DATA 17 +1 20 20 4.74 18 -1 14 15 3.17 END DATA. SAVE OUTFILE=DATA9. * Mode 10: Enter, row by row, Study No., Direction of Effect, DF, T_OBS. DATA LIST LIST /Study(F8.0) Direct(F8.0) DF(F8.0) T_OBS(F8.2). BEGIN DATA 19 +1 54 5.46 20 -1 49 2.27 END DATA. SAVE OUTFILE=DATA10. GET FILE=DATA1. ADD FILES/FILE=*/FILE=DATA2/FILE=DATA3/FILE=DATA4/FILE=DATA5 /FILE=DATA6/FILE=DATA7/FILE=DATA8/FILE=DATA9/FILE=DATA10. EXECUTE. ** PART 2: SETTING ALFA AND CONFIDENCE LEVEL, CHOOSING EFFECT SIZE MEASURE ** AND RUNNIG META-ANALYSIS. * Enter alfa for computing observed power (by default, AFFA = 0.05). COMPUTE ALFA = 0.05. EXECUTE. SORT CASES BY STUDY(A). IF (M1>=M2) DIRECT=1. IF (M10) DIF=ABS(M1-M2). COMPUTE SDX=(((N1-1)*(SD1**2))+((N2-1)*(SD2**2)))/(N1+N2-2). IF (SD_POOL<=0 OR SD_POOL>0) SDX=SD_POOL**2. IF (SDX<=0 OR SDX>0) T_OBS=DIF/SQR(SDX*((1/N1)+(1/N2))). COMPUTE SD_POOL=SQR(SDX). COMPUTE T_OBS=DIRECT*T_OBS. COMPUTE DIF=DIRECT*DIF. COMPUTE TABS=ABS(T_OBS). COMPUTE P_TWO=(1-CDF.T(TABS,DF))*2. COMPUTE P_ONE=1-CDF.T(TABS,DF). COMPUTE D=T_OBS*SQR((1/N1)+(1/N2)). COMPUTE N_HARM=(2*N1*N2)/(N1+N2). COMPUTE NCP=ABS((D*SQR(N_HARM))/SQR(2)). COMPUTE T_ALPHA=IDF.T(1-ALFA/2,DF). COMPUTE POWER1=1-NCDF.T(T_ALPHA,DF,NCP). COMPUTE POWER2=1-NCDF.T(T_ALPHA,DF,-NCP). COMPUTE OPOWER=POWER1+POWER2. COMPUTE R=T_OBS/SQR((T_OBS**2)+DF). COMPUTE R2=R**2. COMPUTE D_H=D*(1-(3/(4*(N1+N2)-9))). COMPUTE BESD_LO=.50-(R/2). COMPUTE BESD_UP=.50+(R/2). COMPUTE U3=CDF.NORMAL(D,0,1)*100. COMPUTE U2=CDF.NORMAL((D/2),0,1)*100. COMPUTE U2X=CDF.NORMAL(((ABS(D))/2),0,1). COMPUTE U1=(2*U2X-1)/U2X*100. FORMATS P_TWO P_ONE ALFA N_HARM NCP OPOWER D D_H R R2 BESD_LO BESD_UP(F8.4) U1 U2 U3(F8.1). SUMMARIZE/TABLES=STUDY DIRECT N1 N2 DF M1 M2 DIF SD1 SD2 SD_POOL T_OBS /FORMAT=VALIDLIST NOCASENUM TOTAL/TITLE="Table 2  Program imputations" /CELLS=NONE. SUMMARIZE/TABLES=STUDY DIRECT DIF DF T_OBS P_TWO P_ONE ALFA N_HARM NCP OPOWER/FORMAT=VALIDLIST NOCASENUM TOTAL /TITLE="TABLE 3  Individual T Tests and observed power"/CELLS=NONE. SUMMARIZE/TABLES=STUDY DIRECT D D_H R R2 BESD_LO BESD_UP U1 U2 U3 /FORMAT=VALIDLIST NOCASENUM TOTAL /TITLE="Table 4 - Measures of effect size and nonoverlap"/CELLS=NONE. SUMMARIZE/TABLES=D D_H/FORMAT=NOLIST TOTAL/TITLE="Table 5 - Non weighted " +"effect size  Descriptive statistics: Cohens d and Hedges' correction" /CELLS=COUNT MIN MAX MEAN SEMEAN STDDEV. SAVE OUTFILE=META_DATA. * Choose the effect size measure (Cohen's d = 1; Hedges' correction = 2) * (by default, ES = 2). COMPUTE ES = 2. IF (ES=1) D=D. IF (ES=2) D=D_H. EXECUTE. COMPUTE V=((N1+N2)/(N1*N2))+((D**2)/(2*(N1+N2))). COMPUTE W=1/V. COMPUTE WD=W*D. COMPUTE WD2=W*D**2. COMPUTE W2=W**2. COMPUTE X=1. EXECUTE. SAVE OUTFILE=FOUTX. AGGREGATE/OUTFILE=*/BREAK=X/SUM_W=SUM(W)/SUM_WD=SUM(WD) /SUM_WD2=SUM(WD2)/SUM_W2=SUM(W2)/NSTUDIES=N. COMPUTE K=NSTUDIES-1. COMPUTE EF_SIZE=SUM_WD/SUM_W. COMPUTE VARIANCE=1/SUM_W. COMPUTE SE=SQR(1/SUM_W). COMPUTE Z=ABS(EF_SIZE)/SE. COMPUTE P_TWO=(1-CDF.NORMAL(Z,0,1))*2 . COMPUTE P_ONE=1-CDF.NORMAL(Z,0,1). EXECUTE. * Enter confidence level for interval confidence (by default, CL=95%). COMPUTE CL = 95. COMPUTE ZCL=IDF.NORMAL((1-(((100-CL)/100)/2)),0,1). COMPUTE CI_LOWER=EF_SIZE-ZCL*SE. COMPUTE CI_UPPER=EF_SIZE+ZCL*SE. COMPUTE Q=SUM_WD2-SUM_WD**2/SUM_W. COMPUTE P_CHISQ = 1-CDF.CHISQ(Q,K). COMPUTE V0 = (Q-K)/(SUM_W-SUM_W2/SUM_W) . EXECUTE. SAVE OUTFILE=FOUTY/KEEP=V0 X. FORMATS ALL(F8.4) VARIANCE SE(F8.5) NSTUDIES CL K(F8.0). SUMMARIZE/TABLES=NSTUDIES EF_SIZE VARIANCE SE Z P_TWO P_ONE CL CI_LOWER CI_UPPER/FORMAT=LIST NOCASENUM TOTAL /TITLE='Table 6  Fixed effects model:' +' Weighted average effect size, z test, and confidence interval' /CELLS=NONE. SUMMARIZE/TABLES=Q K P_CHISQ/FORMAT=LIST NOCASENUM TOTAL/TITLE= 'Table 7 - Chi-square test for homogeneity of effect size'/cells=none. GET FILE=FOUTX. MATCH FILES /FILE=*/TABLE=FOUTY/BY X. EXECUTE. COMPUTE V=V+V0. COMPUTE W=1/V. COMPUTE WD=W*D. COMPUTE WD2=W*D**2. COMPUTE W2=W**2. EXECUTE. FORMATS V0(F8.3). SUMMARIZE/TABLES=v0/FORMAT=NOLIST TOTAL/TITLE='Table 8 - Random variance' +' component'/CELLS=MEAN. AGGREGATE/OUTFILE=*/BREAK=X/SUM_W=SUM(W)/SUM_WD=SUM(WD) /SUM_WD2=SUM(WD2)/SUM_W2=SUM(W2)/NSTUDIES=N. COMPUTE K=NSTUDIES-1. COMPUTE EF_SIZE=SUM_WD / SUM_W. COMPUTE VARIANCE=1/SUM_W. COMPUTE SE=SQR(1/SUM_W). COMPUTE Z=ABS(EF_SIZE)/SE. COMPUTE P_TWO=(1-CDF.NORMAL(Z,0,1))*2 . COMPUTE P_ONE=1-CDF.NORMAL(Z,0,1). EXECUTE. * Enter confidence level for interval confidence (by default, CL=95%). COMPUTE CL = 95. COMPUTE ZCL=IDF.NORMAL((1-(((100-CL)/100)/2)),0,1). COMPUTE CI_LOWER=EF_SIZE-ZCL*SE. COMPUTE CI_UPPER=EF_SIZE+ZCL*SE. FORMATS ALL(F8.4) VARIANCE SE(F8.5) NSTUDIES CL K(F8.0). SUMMARIZE/TABLES=NSTUDIES EF_SIZE VARIANCE SE Z P_TWO P_ONE CL CI_LOWER CI_UPPER/FORMAT=LIST NOCASENUM TOTAL /TITLE='Table 9  Random effects ' +'model: Weighted average effect size, z test, and confidence interval' /CELLS=NONE. GET FILE=META_DATA/KEEP=STUDY DIRECT N1 N2 DF M1 M2 DIF SD1 SD2 SD_POOL T_OBS P_TWO P_ONE ALFA N_HARM NCP OPOWER D D_H R R2 BESD_LO BESD_UP U1 U2 U3. *** END OF THE SYNTAX. **************************************************************************** ** Note ** ** ** Beginning in line: ** ** COMPUTE W=1/V. ** ** with effect sizes (D) and variances (V) from original sources, this ** syntax was tested with data reported in Lipsey and Wilson (2001, p. 130, ** Table 7.1) and Shadish and Haddock (1994, p. 267, Table 18.2). ** ** Imputations procedures and Individual T Tests were tested in SPSS, ** comparing the results with outputs obtained from raw data examples. ** ** Power calculations are the same given by SamplePower (Borenstein et al., ** 2001) and measures of effect size and nonoverlap were tested with ** tabulated values and examples given by Cohen (1988) and Rosenthal et al. ** (2000). ** ** Feel free to use and modify this syntax as you wish. In case you want to ** refer it, the proper form is: ** ** Alferes, V. R. (2003). Meta-analysis: Fixed and random effects models ** [SPSS Syntax File]. Retrieved [Date], from [URL] **************************************************************************** ** References ** ** ** Borenstein, M., Rothstein, H., & Cohen, J. (2001). SamplePower 2.0 ** [Computer Manual]. Chicago: SPSS Inc. ** Cohen, J. (1988). Statistical power analysis for the behavioral ** sciences (2nd ed.). Hillsdale, NJ: Lawrence Erbaum. ** Cortina, J. M., & Nouri, H. (2000). Effect sizes for ANOVA designs. ** Thousand Oaks, CA: Sage. ** Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. ** Orlando, FL: Academic Press. ** Hunter, J. E., & Schmidt, F. L. (1990). Methods of meta-analysis: ** Correcting error and bias in research findings. Newbury Park, CA: ** Sage. ** Lipsey, M. W., & Wilson, D. B. (2001). Pratical meta-analysis. Thousand ** Oaks, CA: Sage. ** Rosenthal, R., Rosnow, R. L, & Rubin, D. B. (2000). Contrasts and ** effect sizes in behavioral research: A correlational approach. ** Cambridge, UK: Cambridge University Press. ** Shadish, W. R., & Haddock, C. K. (1994). Combining estimates of effect ** size. In H. Cooper and L. V. Hedges (Eds.), The handbook of research ** synthesis (pp. 261-281). New York: Russell Sage Foundation. ***************************************************************************.