Preface To Second Memorandum

This is my second memorandum examining statistical aspects of the CSAP tests. In my first document I provided background material having to do with the statistics of small samples and the fluctuations that arise therefrom and applied these concepts to an analysis of some of the 2001 CSAP tests. In this document I focus on the 2001 and 2002 3^rd grade reading tests at the level of our own Lewis-Palmer school district as well as at the state level.

While performing the analyses presented in this document I was looking for hints of non-statistical behavior. My goal has been to find means of detecting real improvement hidden in the statistical noise that is the natural consequence of our small sample sizes. Thus far my results have been negative. Statistical noise or fluctuations seem to dominate the year-to-year changes in CSAP scores.

Negative results, while somewhat interesting, represent a dead end. Positive results, that is developing an ability to sort out real changes from statistical fluctuations, would provide us with the possibility of developing useful and even predictive techniques of analysis. Progress can be made if we have positive results to work with.

After six months of work and, between these two memoranda, some 80 pages of text and figures documenting my researches, my conclusions are the following.

I have no doubt that much can be learned about the educational growth of individual students, by teachers, counselors, and principals by monitoring student performance in the various content from year-to-year.

On the other hand, the year-to-year performance of the ensembles of students that we call classes or grades and then schools seems, with few exceptions, to be within what are known as statistical confidence intervals about a mean. These are defined by the number of students in the group or ensemble.

There are clear differences in ability and performance between classes, schools, and entire school districts. Some of the causes are apparent and some are unknown. Even for known causal relationships quantification of the variables in the problem is difficult at best. This leaves us with a purely probabilistic description of CSAP testing performance, which has been the subject of these memoranda.