Joseph A. Sandford, Ph.D., BrainTrain
Please Do Not Cite Without Written Permission.

A study conducted at the Center for the Study of Special Populations (see note below) was designed to evaluate the effectiveness of commercially available computerized cognitive training (Captain's Log®) and neurofeedback (EEG brain wave biofeedback) treatment in helping to improve the functioning of children diagnosed as having an ADHD disorder. Cognitive training and neurofeedback are new techniques for helping people with ADHD which numerous clinicians and researchers have reported to be effective in reducing many of the characteristic symptoms associated with the disorder.

Previous research in cognitive training has shown it to be useful in improving memory, speed of mental processing, self-esteem and self-control for individuals with attention problems (e.g., ADHD, psychiatric and brain injured populations). Neurofeedback treatment has been reported in several research and clinical case studies to be effective in improving IQ test performance, sustained attention, response control and in reducing hyperactivity for people with ADHD. However, only a few studies for these two new promising types of interventions controlled for possible placebo treatment effects. The scientific approach requires that the efficacy of new treatment techniques be established by the replication of effects in several studies by different researchers. Accordingly, the purpose of this study was to carefully evaluate under controlled experimental conditions the possible effects of cognitive training and neurofeedback, comparing them with each other and to a no treatment group.

A total of 67 participants (85% males) between the ages of 8 and 11 completed the study. These volunteers had been diagnosed as having an ADHD disorder by either a physician or a psychologist and were randomly assigned to the following three groups: 1) Cognitive Training, 2) Neurofeedback or 3) No Treatment. Each person in the two treatment groups was provided 20 half- hour sessions over a 10 week period. No charges were made for any treatment.

To evaluate any significant cognitive, emotional and behavioral changes a battery of tests and parent rating scales were administered at the beginning and end of the study using test examiners who were "blind" to group affiliation. The majority of these individuals were taking medication for ADHD, but were tested prior to the administration of medication that day. Analysis of the experimental findings was based on a 3x2 ANOVA factorial design with subjects in each of the three conditions given a repeated pre-test and post-test battery. Thus, this design provided experimental controls for possible practice, learning and placebo effects.

Cognitive training using the Captain's Log® system was provided by graduate students at the University who followed a structured training protocol provided by this author. The complete Captain's Log® system consists of 33 programs divided into five functional areas. For the purpose of this study, training focused primarily on a wide variety of attention and response inhibition exercises. The emphasis of cognitive training exercises was on drill and practice with the goal being to advance as quickly as possible through the many pre-defined levels. The Captain's Log® cognitive training exercises utilized the capability of the computer to provide immediate non-judgmental feedback for self-regulation, individualized instruction, reinforcement of response inhibition, challenging tasks which require sustained attention, engaging and game-like stimuli and small "chunk-like" building block exercises. The overall goal of this cognitive training program was to promote efficient, self-regulated and self- confident learners.

The neurofeedback treatment was provided by a local, private clinical provider. The neurofeedback protocol for participants who were predominantly the impulsive/hyper-active type consisted of feedback when increases above a predetermined amplitude threshold of the sensory-motor rhythm (12-15 Hz) occurred, while the slower theta rhythm amplitude (4-7 Hz) was below a pre-set criterion level. Neurofeedback for individuals with predominantly inattentive type ADD symptoms differed only in that they were given feedback for the beta rhythm (15-18 Hz) instead of the sensory-motor. In addition, all neurofeedback trainees were given additional feedback to inhibit activity in the 22-30 Hz range. The feedback information was presented in a video game-like format which allowed the children to accrue "points" for changes in the desired direction.

Several memory tests from the WRAML were included in the evaluation to assess any improvements in attention and the encoding of information that may have resulted from the training programs. The ability to focus attention, actively process information and remember school-work type tasks which are not inherently entertaining is a recognized problem area of functioning for many children with an ADHD disorder. Three of the standardized memory tests given showed significant differences in pre and post scores among groups at the p<.01 level. The Cognitive Training group significantly improved on three memory tests. Only the Cognitive Training group showed improvement in the recall and reconstruction of complex visual designs and verbally presented stories. Both the cognitive training group and the neurofeedback group demonstrated significant improvements in another memory test which assesses the immediate recall of an aurally presented sequence of alternating numbers and letters. A significant decline in story memory was observed for the neurofeedback group, showing overall mixed results on memory tests. The No Treatment group did not show any significant improvements in memory abilities on any tests other than a practice effect on the Number/Letter memory test.

Many individuals with an ADHD disorder have problems with response inhibition and can be relatively slow in their mental processing speed on tasks which require them to inhibit an automatic response. Test tasks which require a person to "stop and think" before responding can be very difficult for many of these people. The Stroop test was used in this study to evaluate changes in mental processing speed when doing a task that requires response inhibition. Only the Cognitive Training group showed a significant improvement in mental processing speed on the Stroop test (p<.05).

Traditionally, many clinicians use behavioral rating scales in the diagnosis and assessment of medication or treatment effects for their patients who have an ADHD disorder. Several commonly used rating scales were included in this study to help quantify and statistically assess any emotional and/or behavioral changes which may have resulted from the treatments provided. An analysis of the differences between pre- and post-rating scales completed by the parents identified many significant changes among the groups at the p<.01 level. The meaning and interpretation of these significant findings is presented below for each of the three groups.

If implemented properly, working with computer programs that provide immediate feedback can help to improve self-esteem and feelings of success for most children. The Captain's Log computerized cognitive training exercises were developed as a "Mental Gym." Numerous levels are provided in a structured, behavioral hierarchy which requires sustained attention, perseverance, and reinforcement of response inhibition in order to perform well. Many children report that learning with the computer is fun and computer time is used as a reinforcer in many school settings. The significant improvements observed on the Child Behavioral Checklist scales of Depression and Hyperactivity and the Conners Learning scale are interpreted as reflecting positive changes in self-esteem and self-control. On these three scales the Cognitive Training group was found to be significantly less depressed, hyperactive, frustrated and distractible after treatment.

Based on the Conners Impulsive-Hyperactive scales, even though the groups were randomly assigned, the neurofeedback group was found prior to treatment to be significantly more impaired than the other two groups in terms of the following ADHD characteristics: excitability, bossiness and restlessness. After treatment, this group, based on the rating scales, significantly reduced these ADHD symptoms. Insignificant decreases in these symptoms were observed in the Cognitive Training group for the Impulsive-Hyperactive scale, but the mean of this group was initially not in the clinically impaired range. After treatment, the mean score of the neurofeedback group decreased to this same non-impaired level.

The value of having a No Treatment control group in this study is twofold. First, this group controls for the practice effects of re-testing on the cognitive tests. Second, this group made it possible to determine what happens during summertime for ADHD children when no structured intervention is provided. It could easily be hypothesized that with less demand to perform and an easier life style, children with ADHD would show less emotional and behavioral problems. In contrast, this study did not find this hypothesis to be true. While the treatment groups showed significant improvements in several areas, the No Treatment group's test results showed dramatic, global increases in behavioral and emotional problems. This group became overall significantly more depressed and hyperactive. The No Treatment group also was found on the Child Behavioral Checklist to become significantly more withdrawn. In addition, the No Treatment group's severity of home problems worsened significantly (p<.05). The pervasiveness of the increased problems identified in the No Treatment group were further documented by significant increases in the characteristic symptoms of ADHD comprising the Conners Learning and Impulsive-Hyperactive scales.

There were two Continuous Performance Tests (CPTs) administered in this study. An analysis of these data was not found to be possible at this time due to anomalies and inconsistencies in the data encoded. The cause of these problems was unknown. Several other tests (Kagen Familiar Figures, Underlining and Pegboard) were also administered as part of the test battery, but did not show significant results. These additional tests are not typically used in clinical evaluations of ADHD problems. No studies are known to this researcher which have shown ADHD children to have significantly impaired scores on these additional tests. Consequently, no change after treatment would be expected if these tests are not related to ADHD symptoms. Thus, the finding of non-significant differences on these additional tests was not considered relevant to this analysis.

In summary, objective non-computerized cognitive tests supported the efficacy of the Captain's Log® computerized cognitive training system in helping to significantly improve the ability to focus and sustain attention, to encode and retrieve visual and auditory information and to increase the speed of mental processing. This technique also significantly improved emotional and psychological functioning by apparently decreasing ADHD symptoms which interfere with efficient learning. One test rating scale of Hyperactivity also showed significant decreases in ADHD symptoms. On another scale of Hyperactivity no decreases were observed, possibly because this group's mean scale score to start was already below the clinical cut-off level for Impulsive-Hyperactive behaviors.

An article about the value of using computers to help children with ADHD appeared in Attention Magazine (Fall, 1994). The Captain's Log® system meets many of the criteria emphasized in improving learning and behavior discussed in this article. This article also presented information about the positive benefits of computerized training for ADHD children found in other research studies. Given the relatively low cost of computers and the growing body of research showing their efficacy, a "Mental Gym" such as the Captain's Log® should be seriously considered as a viable technique both for school and home settings to help increase self-esteem, self-control and learning efficiency for individuals with ADHD when used appropriately on a regular, systematic basis.

Neurofeedback treatment was found to be more specific in its benefits, and the evidence indicates that this treatment helped to reduce impulsive, hyperactive, and off-task behaviors for individuals who initially presented as significantly more impaired than the other two treatment groups in terms of Impulsive-Hyperactive behaviors. The feedback frequency range of brain waves selected for this study is recognized in the biofeedback literature as reducing hyperactivity, motoric restlessness and seizures. Thus, the rating scales findings did not reflect global changes, as might be expected if attributable only to placebo effects, but appeared to be directly related to neurofeedback treatment goals.

The important issue arises as to whether these observed changes have clinical significance. Clearly, the average improvement of 25% in memory observed in the Cognitive Training group could easily lead to better academic functioning. The positive treatment effects of those individuals in the Cognitive Training group of reduced hyperactivity, depression and impairments to learning are also recognized as valid therapeutic goals for traditional treatment techniques for ADHD. Other critical issues of generalization and resiliency were not addressed in this study. While it will take many studies to address fully all these issues, a relevant case study was presented at the American Psychological Association in 1994 which used Captain's Log® as the sole means of treatment. The young boy in this study had already been treated unsuccessfully using medication, group and individual psychotherapy. After 35 sessions of cognitive training, his behavior improved greatly and his grades improved from D's and F's to B's and C's. He has been followed for almost a year, takes no medication and has not regressed. This case study supports the clinical effectiveness of cognitive training, its generalization to school work and its lasting effectiveness after treatment stops.

Given the known variability of responding of ADHD children, it is often difficult to obtain significant results in clinical research unless a treatment effect is a relatively powerful one. Both cognitive training and neurofeedback are supported by this research as demonstrating positive meaningful effects in a real sense. For neurofeedback, three of 21 parent rating scales showed significant changes and these effects were congruent with specific treatment goals. Likewise, the Cognitive Training group was found to have significant improvements on three parent rating scales of emotional and psychological functioning, and these changes were congruent with the improvements found in the objective tests of mental processing speed, attention and memory. In contrast, the No Treatment group demonstrated the most changes (seven of 21) on these rating scales with the report of numerous increases in ADHD symptoms. While research has shown that ADHD rating scales can be influenced on retest by the factor of parental cognitive dissonance, the pattern of the results of this study do not show any support for this alternative explanation. The success of this study's findings can most likely be attributed to the power of these new treatments, the large number of individuals included in each group and a test design based on the comparison of pre- and post-test scores. Future researchers in this area may need to consider these factors so as to maximize the power of their study to detect meaningful changes.

Improvements in the neurofeedback group were also found to be clinically valid. At the end of treatment for this group of individuals, the mean score, which was initially clinically elevated based on the Conner's Impulsive- Hyperactive scale, decreased to below the clinical cut-off level. However, only the one area of memory related to an increased ability for immediate auditory recall showed improvement, and verbal recall of a story significantly declined. Thus, the neurofeedback treatment did not seem as reliable or broadly effective in improving attention, concentration and memory skills as cognitive training.

Based on this research, these new techniques show great promise as components in the multi-modal treatment of ADHD. Given the severity, chronicity and complexity of ADHD, a comprehensive multi-modal approach is often necessary for success. The traditional approaches of psychotherapy, behavior modification, school modifications, family therapy and parent education can all be integrated and combined with these new approaches. While our understanding of who will best benefit from cognitive training and neurofeedback and how these new techniques may best be used is limited at this time, it is hoped that these findings will encourage further controlled research in these areas.

It may prove beneficial to combine cognitive training and neurofeedback for some individuals. Taking one approach, integration of these two techniques could be implemented by first giving one treatment then the other one. For example, if a child is so restless and agitated that he cannot sit still, this study's findings indicate that it would be best to use neurofeedback first to reduce these symptoms. Next, this ADHD child may have problems remembering to do his homework or, if he does it, remembering to turn it in. In this case, based on this study's results, cognitive training would be the next best stage in the treatment protocol. Naturally, a clinician will have to make these types of decisions in the treatment protocol. Furthermore, cognitive training and neurofeedback can be simultaneously combined using multiple computers. This treatment approach naturally requires shifting and dividing attention. A typical protocol using this method would be for the client to focus on the neurofeedback screen for five minutes, then, while auditory EEG feedback continues in the background, perform a cognitive training task for five minutes. The EEG brain waves can continue to be recorded during the cognitive exercise, and the clinician can observe any interaction. The Captain's Log cognitive exercises cover a wide range of mental exercises, and tasks can be selected to require only simple attention or made extremely demanding when appropriate. The value and possible synergy resulting from combining these two techniques has not been proven, but does offer an exciting, promising area for further research.

Finally, of great interest is the surprising finding of this study that no treatment at all of children with ADHD during the summertime led to a pervasive worsening of their emotional and behavioral problems. The No Treatment group also showed no gains whatsoever in cognitive functioning on the objective tests. It is not clear to this researcher what kind of activities (e.g., camp, day care, summer school, vacations, etc.) these children participated in during the summer. However, the lack of regular tasks and exercises demanding focused attention, concentration, sitting still and increasing response inhibition clearly appears to be detrimental to the mental health and well being of ADHD children.

Note: This analysis is based on independently conducted research completed during the Summer of 1993 at the Center for the Study of Special Populations, California State Polytechnic University, Pomona, California, by Drs. Aubrey Fine and Larry Goldman. The interpretation and views expressed in this analysis of the study's findings are those solely of Dr. Sandford who has the other authors' permission to disseminate their findings and his own analysis of the results.

Dr. Bill Lynch
Program Chief, Brain Injury Rehabilitation Unit, Veterans Affairs Medical Center, Palo Alto, CA
November, 1993

"...With the variety of tasks available in the Captain’s Log® series, the clinician should have no difficulty in providing training exercises that are both deficit-relevant and challenging. Further, the numerous parameter adjustments available should permit even patients with considerable cognitive and/or motoric impairments to access several of the programs when working independently under the supervision of a clinician (e.g., doing "prescribed" homework or lab assignments).

The Captain’s Log® programs are well written, tightly constructed and virtually immune from "crashing" or locking up due to unanticipated user actions. They make good use of color and present critical information clearly. They permit alternative input methods, including keyboard, mouse, trackball, and head pointer. The programs’ menus are consistent in their appearance and easy to comprehend. There is sufficient variety among the program tasks to address the needs of most patients in a cognitive rehabilitation program.

All things considered, the Captain’s Log® series is a sensible choice both for clinician-assisted as well as independent cognitive training. It possesses that ideal combination of ease of learning and use, seamless movement among programs and modules, and comprehensive content choices."