REGISTER OF PACKAGE PROGRAMS FOR EXACT STATISTICAL INFERENCE VERSION 2.8 : 02-SEP-96 STATUS OF THIS DOCUMENT/FILE This document/file has been produced as an activity of the e-mail discussion group exact-stats based on the list-server mailbase@mailbase.ac.uk . If you distribute this material onwards, then you should mention the source and also include this statement of requirement. To obtain the latest version of this document/file, send the following 1-line e-mail message : to: mailbase@mailbase.ac.uk message: send exact-stats rtpackages.txt You can also access this file, and other exact-stats files, via World Wide Web or via anonymous FTP (site: ftp.mailbase.ac.uk) This document/file is organised to provide a list of package programs known to us at exact-stats which undertake exact randomisation statistics and related techniques, along with descriptions of the individual items provided by suppiers and contact details. You may find that the list of items is longer than the list of descriptions - it is hoped that this mismatch will dwindle as suppliers see the advantage of letting people read about their products and find time to provide descriptions. NAMES OF PACKAGE PROGRAMS : LogXact (C) NPSTAT PERMUSTAT (F) RANTEST RESAMPLING STATS (C) RT (C) SAS (C) SCRT SIMSTAT (S) SPSS Exact Tests (C) Statistical Calculator ('SC') (C) StatXact (C) StatXact Turbo (C) StatXact Version 3 for Windows (C) Testimate (C) Key : (C) = Commercially Marketed ; (F) = Free (non-commercial use) (S) = Shareware Distributed SUPPLIERS' DESCRIPTIONS LOGXACT : Availability : Supplier : Cytel Software Corporation, 675 Massachusetts Avenue, Cambridge, Massachusetts 02139, USA; Phone : 617-661-2011; Fax : 617-661-4405; E-mail : mehta@jimmy.harvard.edu Version: 1.1 (July 1993); Hardware and Software Environment : IBM PC and Compatibles, MS DOS, OS 2 and Windows Costs from $295 to $1495 (depending on product and network) Contact Person : Dr. Cyrus R. Mehta Product Review : Author Lemeshow, S (1994) : Epidemiology 5, pp259-260. Features : LogXact-Turbo: Software for Exact Logistic Regression : Release 1. LogXact is the only software package in the world supporting exact inference for logistic regression. The usual method of estimating the parameters of logistic regression is maximum likelihood. However for small or imbalanced data sets the maximum likelihood method is often inaccurate, and sometimes unable to produce any estimates at all. In 1970, D.R. Cox proposed an alternative method of inference based on the exact permutation distribution of the sufficient statistics for the regression parameters. The method yields exact p-values and confidence intervals no matter how small, sparse, or imbalanced the data are. Until recently the computational difficulties inherent in generating the exact permutation distributions precluded using this method. But new algorithms by Dr. Cyrus Mehta and his colleagues at Harvard University have made exact inference in logistic regression practical. The algorithms are published in J.American Stat.Assn. vol 82, pages 1110 to 1117, 1987, and Biometrics, vol 44, pages 803-814, 1988. For this work they received the 1988 George W. Snedecor award from the American Statistical Association. LogXact has evolved over a period of five years and incorporates these computational advances into a friendly software package. It computes both exact and asymptotic p-values and confidence intervals. In addition to the usual logistic regression model, it also handles conditional logistic regression for matched case-control studies or clustered binomial data. NPSTAT Availability: Contact : Richard May Address: 1217 E. Brickyard Road, #106, Salt Lake City, UT 84106, USA e-mail: rmay@utah-inter.net tel: 801-463-1839 Features: A primitive version (2.0) was described in _Behavior Research Methods, Instruments & Computers_ (1989, 21, 482-483). The current version (3.8) is accompanied by NPFACT 1.0 (December, 1993). These DOS programs are menu-driven. The programs carry out parametric as well as randomization tests, for several research designs including independent groups (equal or unequal n), repeated measures, and correlations. The user may obtain a copy of the randomization distribution if desired. Transformations are available including ranks which give access to more widely known tests (e.g., Wilcoxon-Mann-Whitney). There are also options for discrete data which include Fisher's exact test. Although NPSTAT 3.8 includes parametric analysis of 2-factor designs, NPFACT 1.0 carries out both parametric (F) and nonparametric (randomization) tests for 2-factor and 3-factor designs in independent (between), dependent (within), and mixed designs. A given set of data may be easily analyzed by a variety of analyses to suit the needs of the researcher. PERMUSTAT Contact: Andrew F. Hayes, University of New England, Department of Psychology, Armidale, NSW 2351, Australia. e-mail: ahayes@metz.uns.edu.au Current version: 1.2.1 (Version 2.0 expected in early 1997) Hardware/Software: Macintosh, 500K RAM, System 6.0 or later. DOS/Windows versions planned for release in 1997. PERMUSTAT is currently available free of charge for noncommerical distribution. Version 1.2 can be downloaded from the MacPsych archive by anonymous ftp to ftp.stolaf.edu/pub/macpsych. The files can be decompressed with Unstuffit,a shareware program available through the archive. PERMUSTAT is described in Behavior Research Methods, Instruments, and Computers, vol 28, 1996. Features: PERMUSTAT is a stand-alone Macintosh application that will compute approximate and exact p values via data permutation for Pearson and Spearman correlations and the difference between two means (independent or dependent). Fisher's exact test for 2x2 tables can also be conducted using the independent means option. Data can be entered manually or read from a disk file. No data editing features are included in version 1.2.x Output can be saved as text for printing. Plans for version 2.0 include analysis of one-way ANOVA-type designs (between and within), rank transformation of data, direct printing, and data editing capabilities. RESAMPLING STATS Availability: Contact: Peter Bruce Address: Resampling Stats Inc., 612 N.Jackson St., Arlington, VA 22201, USA email: stats@resample.com World Wide Web: http://www.statistics.com phone: 703-522-2713 ; fax: 703-522-5846. Hardware/Software requirements: Versions for DOS, Windows and Macintosh Price: $225 ; Academic Price : $125 ; Academic Site License (network installation for class use): $350 Features: Resampling Stats is a simple but powerful programming language designed to facilitate resampling methods, including randomization tests and bootstrap procedures. Commands like SHUFFLE, TAKE, SAMPLE and COUNT manipulate data vectors within repeat loops, enabling quick and flexible programming. Extensive online help (including a syntax prompt that helps you write programs) -- the program has been used successfully by 12 year olds. Comes with 80-page user guide. A companion text "Resampling: The New Statistics" is also available ($30). This is an introduction to statistics from a resampling / simulation perspective. Resampling Stats also carries stocks of most of the published books on resampling and randomization methods, including Efron and Tibshirani's "Introduction to the Bootstrap", Westfall and Young's "Resampling-Based Multiple Testing," Manly's "Randomization and Monte Carlo Methods in Biology," and more. See web site at www.statistics.com for a complete listing, including detailed book descriptions and prices. RT Availability: Contact Person: Bryan Manly Address: Centre for Applications of Statistics and Mathematics, University of Otago, P.O. Box 56, Dunedin, New Zealand Tel: 64-3-479-7775 Fax: 64-3-479-8427 or (in North America) Western EcoSystem Technology Inc., 1402 s. Greeley Highway, Cheyenne, Wyoming 82007 (Telephone 307- 634-1756, Fax 307-637-6981). Features: The computer program RT is based on the FORTRAN programs included in the book Randomization and Monte Carlo Methods in Biology, written by Bryan Manly and published by Chapman and Hall in 1991. It runs on IBM PC, XT, AT, PS/2, or compatible microcomputers with at least 640k of RAM memory and 700k of disk space. Additional desirable hardware features are a CGA, EGA or VGA graphics card and a maths coprocessor. The program includes modules for: * entering, editing, transforming and plotting data * calculating distance matrices * the paired comparison and Fisher randomization test on one sample * two sample comparisons of means and variances * analysis of variance with one to three factors, with randomization of observations or residuals * simple and linear multiple regression, with randomization of observations or residuals * Mantel's test with up to nine matrices of dependent variables * Mead's test and Monte Carlo tests for spatial patterns * tests for trend, autocorrelation and periodicity in time series * multivariate tests Improvements on version 1.01 of RT include: an extra time series test, output to a log file as well as the screen, and the option to randomize residuals with analysis of variance and regression. Version 1.02A is the same as version 1.02 except that some minor faults have been corrected. Statistical Analysis System (SAS) Availability: SAS Institute Inc., SAS Campus Drive, Cary, NC 27513. Many platforms. Features: SAS is one of the general statistical software packages that includes options for exact and Monte Carlo permutation tests. The software technology is largely provided by Cytel - producers of StatXact. 1. PROC FREQ: The EXACT option of the TABLES statement provides Fisher's exact test for contingency tables that are larger than 2x2. The CHISQ option of the TABLES statement also provides Fisher's exact test in addition to several chi-square tests if the table is 2x2. The Mehta-Patel network algorithm is used for efficient computation (more information in SAS/STAT User's Guide, Vol. 1: ACECLUS-FREQ, Version 6, 4th ed.). 2. PROC MULTTEST: This procedure handles data arising from a multivariate one-way ANOVA model, possibly stratified, with continuous and discrete response variables. You can perform a t-test for the mean for continuous data and the following statistical tests for discrete data: Cochran-Armitage linear trend test, Freeman-Tukey double arcsine test, Peto mortality-prevalence test, and Fisher's exact test. The following p-values adjusted for multiplicity are offered: Bonferroni, Sidak, bootstrap, and permutation. Nonsequential or sequential adjustments are possible. A simple Monte Carlo permutation t-test is obtained by selecting TEST MEAN(dependent) and specifying only one contrast: CONTRAST 'name' 1 -1 (more information in SAS Technical Report P-229 and in Westfall and Young, 1993). 3. PROC NPAR1WAY, PROC UNIVARIATE, AND PROC RANK: With these procedures the classical nonparametric rank tests can be performed. With PROC NPAR1WAY one can perform the Wilcoxon rank-sum test (Mann-Whitney U test), the Kruskal-Wallis test, the median test (Brown-Mood), the Van der Waerden test, and the Savage test. With PROC UNIVARIATE one can perform the sign test and the Wilcoxon signed rank test. Friedman's test and approximations to other nonparametric rank tests can be performed using PROC RANK before using the regular ANOVA or GLM procedures. (more information in SAS/STAT User's Guide, Vol. 2: GLM-VARCOMP Version 6, 4th ed., and in SAS Procedures Guide, Version 6, 3th ed.). Also see the entries for StatXact provided elsewhere in this listing. SINGLE CASE RANDOMIZATION TESTS (SCRT) Availability: Contact: Patrick Onghena Address (development and information): Katholieke Universiteit Leuven, Department of Psychology, Tiensestraat 102, B-3000 Leuven, Belgium Address (distributor): Interuniversity Expertise Center ProGAMMA, P.O> Box 841, 9700 AV Groningen, The Netherlands. Programmed by Geert Van Damme and Patrick Onghena. Hardware and software environment: IBM-compatible PC with 640 Kb RAM at least, running DOS 2.0 or higher. Useful supplementary (but not necessary) hardware are: a hard disk, a mouse, a RAM drive (for fast access to temporary files), a mathematical co-processor or 80386 or 80486 processor (for fast execution of computer intensive statistical analyses), and a printer. Features: SCRT is a software package to design single case experiments and to perform graphical and statistical analyses on data collected within such a design. All statistical tests of SCRT are based on the principle of randomization, and are implementations of the procedures described in journals like Behavioral Assessment and Behaviour Research and Therapy, and in the handbook on randomization tests by Eugene Edgington. The major types of single case experimental designs that are included as defaults are Alternating Treatments Designs, AB Designs and extensions (ABA, ABAB, etc.), and Multiple Baseline Designs, but a facility is provided to build and analyze any other tailor-made randomized design. One of the strengths of the implemented permutation procedures is that the test statistics are not restricted to those whose sampling distribution has been derived analytically. Therefore, in SCRT a Test Statistics Editor is provided where customized statistics may be built and selected as the criterion for the randomization test. For small-N designs or larger projects involving replications of single case experiments, a nonparametric meta-analytic procedure is included. The integration of single case designs and meta-analysis provides the researcher a bottom-up methodology for analyzing experimental data. In the usual top-down approach of group designs, all data are entered in an overall analysis first and afterwards the question of the generalizability of the mean effect to the individual is addressed. The bottom-up approach of single case meta-analysis allows the researcher to analyze the data on an individual level first and to integrate the results on a group level afterwards. Besides statistical tests, also descriptive statistics, graphical analyses, plots of the randomization distribution, and plots of the evolution of the p-value during the analysis can easily be obtained. Before the analysis, ordinal, Z, square root or other transformations are possible on the data. After each randomization analysis, a report with descriptive statistics and the results of the tests are generated. If the randomization test is based on random shuffling, 95% and 99% confidence intervals for the p-value are provided. The program can be mouse-driven or keyboard-driven. Options are selected from pull-down menus or in dialog windows. Analyses are performed by using hot-keys or pushing screen buttons. Data can be loaded from an ASCII-file or entered directly in the SCRT Data Editor. Output may be directed towards an ASCII-file or immediately to the printer, and graphs may be downloaded with a memory-resident screen capture utility. A 30 page manual and an extensive README-file explain all the details of SCRT and illustrate them with artificial and published data. SIMSTAT : Availability: Contact: Normand Pelandeau Address: 5000 Adam Street, Montreal, QC, Canada H1V 1W5 Version: 3.5 (September 1994); SIMSTAT is currently available as a Shareware program. It requires 356K of free RAM and 1Mbyte of disk space. The registered version can be obtained for $71 ($73 outside USA and CANADA). Educational license and quantity discounts are also available. Features: SIMSTAT v3.0 is an easy statistical and simulation program for DOS with pulldown menus, context sensitive help and mouse support. It can read various file formats such as ASCII, dBase, Lotus 1-2-3, SPSS/PC+, SPSS for Windows. The program provides a wide range of statistical analysis such as frequency, crosstabulation, breakdown, t-test, multi-way anova/ancova, linear, nonlinear & multiple regression analysis (standard, stepwise, forward, backward, and hierarchical), time-series, reliability analysis, and a wide range of nonparametric tests. The program also includes some analysis rarely found in other programs such as interraters agreement measures, multiple responses analysis, ROC curves analysis, and single case experimental design. SIMSTAT also provides an extensive bootstrap module with allow you to obtain the empirical sampling distribution of 27 different estimators. Typical bootstrap output includes descriptive statistics, nonparametric and bias corrected confidence intervals, high-resolution histogram, percentile table, and proportion of samples reaching various alpha levels. The ability to alter the bootstrap sample size gives the possibility to use the bootstrap procedure to perform nonparametric power analysis. The FULL ANALYSIS command also allows to do almost any analysis available within SIMSTAT on each bootstrap sample. SPSS Exact Tests Availability: Address: SPSS Inc., 444 North Michigan Avenue, Chicago, Illinois 60611. Version: New in SPSS 6.1 for Windows (April 1995); Hardware/Software Requirements: Many platforms. Features: SPSS is one of the general statistical software packages. The product described here is fully integrated into SPSS Version 6.1 for Windows. The software technology comes from Cytel - the producers of StatXact. This software supplements the long-established Crosstabs and Nonpar modules of the base SPSS package by providing exact p-values in place of the corresponding asymptotic ones. It comes with its own user-manual thoroughly documenting the statistical methods used to compute the exact tests. The level of documentation is high so that the manual may even be used to supplement textbooks for coureses on nonparametric statistics. The following exact statistical procedures are provided : 1. One-sample goodness-of-fit inference (Chi-square Test; Kolmogorov-Smirnov Test). 2. One-sample inference for binary data (Binomial Test; Binomial Confidence Intervals; Runs Test). 3. Paired-sample inference (Sign Test; Wilcoxon Signed-Rank Test; McNemar's Test; Marginal Homogeneity Test). 4. Two independent samples inference (Wilcoxon-Mann-Whitney Test; Kolmogorov-Smirnov Test; Wald-Wolfowitz Runs Test). 5. K related samples inference (Friedman's Test; Cochran's Q Test; Kendall's W Test). 6. K independent samples inference (Median Test; Kruskal-Wallis Test; Jonkheere-Terpstra Test). 7. Unordered RxC contingency tables (Fisher Exact Test; Pearson Chi-Square Test; Likelihood Ratio Test). 8. Singly-ordered RxC contingency tables (Kruskal-Wallis Test). 9. Doubly-ordered RxC contingency tables (Jonkheere-Terpstra Test). 10. Measures of association for ordinal data (Pearson Correlation; Spearman Rank-Order Correlation; Kendall Concordance; Kendall Tau; Somer D; Gamma). 11. Measures of association for nominal data (Contingency Coefficients; Goodman-Kruskal Tau; Uncertainty Coefficient). 12. Measures of agreement (Cohen Kappa). Also see the entry for StatXact elsewhere in this listing. STATISTICAL CALCULATOR (SC) Availability: Contact/Author: Tony Dusoir Address: Mole Software, 34 Greenville Rd., Bloomfield, Belfast BT5 5EP, N. Ireland. e-mail: ae.dusoir@ulst.ac.uk World Wide Web: http://www.shsj.ulst.ac.uk phone/fax: (+44) (0)1232 454640 Platforms: DOS, Protected Mode (PM) DOS, Unix, T800 transputer. Version: 1.3 (1995); Prices: (GBP.): Single User PM DOS 349; University Dept. PM DOS 699; Unix 5-user licence 699. No recurrent charges, free support. Features: SC is an extensible statistical environment, supplied with over 1200 built-in (compiled C) and external (written in SC's C-like language) routines. Exploratory, robust, and distribution-free statistics are particularly well represented. There are permutation-based methods for contingency tables (chisquare, likelihood, Kendall S, Theil U, kappa, tau, odds ratio), 1-sample inference (Wilcoxon & Fisher tests/intervals), >=2-sample comparisons (Mann-Whitney, Pitman,Friedman, Cochran, Kruskal-Wallis, Jonckheere,..), correlation (Pitman/Pearson,tau,Spearman), multivariate data (MV runs test, Boyett/Schuster, T-square,.), and for Anova and Regression designs. There are also many random number generators, sampling/shuffling/ assigning routines for easy building of other methods. SC's rich programming language makes it easy to use exact routines inside other procedures, e.g. CART (Classification & Regression Trees), or in Monte Carlo teaching demos.. There are specialised 'ready-made' bootstrap routines (e.g. for testing homoscedacity, detecting multimodality,..) + general bootstrapping and jack-knifing facilities. Robust M-estimation methods, and over 50 high-accuracy distribution-function routines, are also included. STATXACT Availability: Contact: Dr. Cyrus R. Mehta StatXact-Turbo: Exact Nonparametric Inference : Release 2 Supplier : Cytel Software Corporation, 675 Massachusetts Avenue, Cambridge, Massachusetts 02139, USA e-mail: mehta@jimmy.harvard.edu tel: 617-661-2011 fax: 617-661-4405 Hardware and Software Environment : PC 286, 386 or PC 486 with MS DOS, OS 2 or Windows; Sun Sparc Station with Unix. Costs: from $195 to $1495 (Depending on Product and Network) Product Review : Lynch, James C and Landis, J. Richard (1991) The American Statistician 45, pp151-154. Features : StatXact computes exact p-values and exact confidence intervals for nonparametric and contingency table data. The usual asymptotic methods of analysis are unreliable for such data, but the exact methods of StatXact, based on permuting the observed data in all possible ways that they could have arisen, provide absolute reliability, no matter how small, sparse or imbalanced the data are. StatXact is based on over ten years of research in the field of computational statistics by Dr. Cyrus Mehta and his colleagues at Harvard University. Their algorithms are published in journals like: J. Amer. Statist. Assn., Biometrika, Biometrics, and J. Comp. Graph. Stat. The software is installed at several thousand sites including the Food and Drug Adminsitration, the Centers for Disease Control, the National Institutes of Health, and numerous leading Universities all over the world. Statistical procedures include: one-sample tests, two-sample tests, rxc contingency tables, stratified 2x2 and stratified 2xc contingency tables, and exact confidence intervals for odds ratios. The exact results in StatXact are validated in an article by Strawderman and Mehta, Comp. Stat. and Data Analysis, 14, 2, 263-266, 1992. Two derivatives of StatXact, provided by agreement with Cytel, are available as modules within major general-purpose statistical packages. A special version, StatXact for SAS is also available for users who wish to remain within the SAS environment. A module is available for use within SPSS. See entries for SAS and SPSS in this list. Also see entry for StatXact Version 3 for Windows in this list. STATXACT VERSION 3 FOR WINDOWS Contact: Dr. Cyrus R. Mehta StatXact-3: Statistical Software for Exact Permutational Inference Supplier: Cytel Software Corporation, 675 Massachesetts Avenue, Cambridge, Massachusetts 02139, USA e-mail: mehta@jimmy.harvard.edu http://www.cytel.com tel: 617-661-2011 fax: 617-661-4405 Hardware and Software Environment: StatXact-3 runs on all Microsoft Windows platforms as a stand-alone product. In addition, a special version, PROC StatXact-3 for SAS Users, is available as external SAS procedures for both the Microsoft Windows and Unix operating systems., Costs: Product Review: Lynch, Landis and Localio (1991), American Statistician, 45(2):151-154 Features: StatXact-3 is a specialized software package for the exact analysis of small-sample categorical and nonparametric data with special emphasis on data in the form of contingency tables. The term "small-sample" applies equally to datasets with only a few observations, to large but unbalanced datasets, or to contingency tables with zeros and small cell-counts in some of the cells but large cell-counts in other cells. In these settings StatXact-3 produces exact p-values and exact confidence intervals instead of relying on possibly unreliable large-sample theory for its inferences. The inference is based on generating permutation distributions of the appropriate test statistics in a conditional reference set. For a discussion of the theory underlying exact inference, references to numerical algorithms that perform the computations, and several examples involving the analysis of biomedical data by StatXact, refer to Mehta (Statistical Methods in Medical Research, 3(2):135-156, 1994). StatXact-3, offers exact p-values and confidence intervals for one, two and K-sample problems, 2x2, 2xc, and rxc contingency tables, and measures of association. The data may be either unstratified or stratified. Both independent and blocked samples are accomodated. StatXact-3 has inference procedures that cater explicitly to binomial data, nominal categorical data, ordered categorical data, continuous complete data, and continuous right-censored data. TESTIMATE Availability: Testimate, Version 5.2 (December 1994) Contact: Dr. Volker W. Rahlfs Address (supplier): idv: Datenanalyse und Versuchsplanung, Wessobrunner Strabe 6, D-82131 Gauting, Munich, Germany; phone: 89.850.8001 fax: 89.850.3666; also, SciTech International, Inc., 2231 North Clybourn Avenue, Chicago, IL 60614-3011, USA; phone: 800.622.3345 or 312.472.0444; fax: 312.472.0472 Hardware and Software Environment: IBM PC and compatibles, MS DOS Costs: . Product Reviews: Kallam, M (1994): Social Science Computer Review, 12, pp. 149-152 Niculescu, SP (1994): Environmetrics, 5, pp. 87-88 Petticrew, M (1994): British Journal of Mathematical and Statistical Psychology, 47, p. 191 Features: Computes exact p-values and exact confidence intervals for nonparametric data, for contingency table data, for simple linear models (one-way ANOVA and ANCOVA), and for equivalence procedures in some cases (e.g., cross-over studies). The authors have written up some of their work in journals; a recent publication giving some flavor of their philosophy is: Rahlfs, VW and Zimmerman, H (1993), "Scores: Ordinal Data with Few Categories--How Should They Be Analyzed?", _Drug Information Journal_, 27, 1227-1240. Statistical procedures include: t-test (paired and unpaired), Fisher-Pitman test, Kolmogorov-Smirnov, Wilcoxon-Mann-Whitney, Median test, Mantel-Haenszel test, 2x2 table, 2xC table, all possible tests (Streitberg-Rohmel), Kruskal-Wallis, Multi-group median test, Mantel-Haenszel multi-group, Rx2 table, RxC table, Wilcoxon signed ranks, sign test, MeNemar's chi-squared, Friedman, one-way analysis of variance and analysis of covariance, logrank test and Crossover analysis. Documentation includes a separate 71 page examples booklet. (Some of this information was taken from an advertisement on p. 47 of the August-September 1994 _Amstat News_.)