How to Research Treatment Programs for LD and ADHD
Also available as a PDF download.
If you look on the Internet or watch television, you will see ads for programs that promise to “cure” students’ learning disabilities and or ADHD. Other programs promise to improve students’ cognitive skills. Before you spend money on a program, you may want to evaluate the claims of any treatment program in order to decide whether it is the right choice for your student, and make sure that the people providing the treatment have the qualifications you expect.
A few years ago, an acquaintance told me that a learning specialist had recommended that her daughter go through a program designed to enhance her cognitive processing. This program required the student to come for two or three sessions per week over a number of months and was quite costly. This program claimed to be “research-based,” so I checked the website page that listed the articles on which the program was supposedly designed. The article that seemed to serve as the basis for the frequency of sessions dated from the early nineteen-sixties, and it showed that socio-economically deprived students’ IQ scores rose when they were put in pre-school several days a week. While the results of this old study are not in question, the relevance of that study to the current’ program’s explanation for why sessions need to be so frequent was tenuous at best.
Before you spend money on any program, you may want to have a basic understanding of the terms used in the marketing materials for these programs. When a program says that it is “research-based,” it means that elements of a program (such as the kind of computer program used for students to practice a skill) are based on something used in a study. However, this is no guarantee that the program will produce the same results for your child, as the conditions may be different or your child may have a different learning profile of strengths and weaknesses than the students who were involved in the study.
There is nothing deceptive or wrong with companies saying that their programs are research-based. It just may not mean what you think it does.
Remember that the results of a study are particular to the conditions of that study. In other words, when those particular students worked with a particular program in particular conditions, their scores improved on a particular measure. One of the problems with putting too much stock in the research is that:
- this program may have worked well for the students in the study, but it might not work for your student;
- the program was delivered by people well-trained in the intervention used in the study; the people providing the program to you child might not have the same level of training;
- the time commitment may be unrealistic – sometimes interventions require that students come for several sessions a week;
- the tests used to measure skill improvement in the study might not have any carry-over to the academic skills you are hoping to improve.
The International Dyslexia Association’s journal, Perspectives on Language and Literacy, warned against this last assumption (see the Winter 2011 issue). Several programs claim that improvement in a particular skill lead to improvement in reading. But the research did not support their claims. Another thing to be careful of when looking at a program is companies’ claims that research has shown the effectiveness of their program. It can be helpful to know how to evaluate these statements.
Some advertisements will say that research has proven the effectiveness of a program. The gold standard for research to prove that an intervention works includes:
- a large sample size (i.e., it was tried on a large number of students);
- a control group that does not receive the intervention so that a comparison can be made between the experimental group’s progress and the control group’s. Some studies show that while the experimental group improved, the control group did too – sometimes to the same degree;
- subjects who stay enrolled in the study for the duration of the experiment;
- the results have been duplicated – in other words, another set of researchers obtained the same results when they repeated the experiment with the same conditions.
Most importantly, the study should be published in a peer-reviewed journal. This means that other professionals in the same field have looked at the experiment agree that it was carried out properly and that the results are valid.
If you look on the websites of some of these programs, you may see studies conducted by the corporation that sells the program. If you do not see that the studies are published in a peer-reviewed journal, this does not mean you have to discount the research results you see, but you should know that they have not been verified by anyone else, and you should consider how much stock you want to put in results that have not been examined by professionals who have no stake in the company.
Another element you need to consider is whether the peer-reviewed research shows long-term effects. Very few programs test their subjects a year or two years after the study is over to see whether the students have maintained their skill level without access to the treatment or program that produced the initial gains. This is important, as skills may deteriorate without constant practice in the intervention used in the study. Think of it this way – if you take tennis lessons every week, your stroke is likely to improve. If you keep playing tennis, you might continue to have a good stroke, but you also might fall back into bad habits. The same is true of academic skills. Programs should be measuring whether the techniques they use have a long-lasting impact in the absence of continued intervention (though even the good ones rarely do).
The concept of skill carry-over is one that warrants a little more explanation. What this means is that some programs target a particular skill (e.g. detection of short vowel sounds). While this skill may be one of the those that underlie a higher-level skill (such as reading), this does not necessarily mean that improving this discreet skill will lead to improvements in students’ reading. This is why it is important to look at the testing measures used. If pre- and post-testing was conducted with a normed, standardized reading measure (such as the Gray Silent Reading Test or the one of the Woodcock-Johnson reading measures), then you will be able to see that the improvement of the small skill led to improvement in the desired area. The research and promotional materials from a treatment program will not likely make this clear – this is why you want to check the research and make sure you don’t assume that a treatment will address the difficulty your student is having.
One of the best resources for checking a program’s research claims is the What Works Clearinghouse. Go to this page and type in the name of the intervention program you are considering. If WWC has reviewed it, you can see what they found in looking at the research studies done on that program.
In addition to checking the research claims, you want to make sure that you check the qualifications of whoever will be working with your child (not just those of the person who runs the treatment center). Franchises are opening that target disorders such as dyslexia and ADHD. Unlike your people who work in your public school district, the owners and employees of these centers are not required to have degrees in special education or any related field.
Instead, they may just get trained by someone from the company that owns the program they are using. While this is not necessarily bad, you have to consider whether you want to pay money to have your student treated by someone who has only received a few weeks’ or months’ worth of training on a particular program that is provided by a for-profit company, rather than a university program. Do not be shy about asking to see the credentials of anyone working with your student. If you do not see any degree or university-affiliated certificate in the history of that person, you simply have to decide whether having your student work with that person is worthwhile.
As a parent, you want to do the best you can for your student. If you are considering paying for a treatment program, tutoring, or intervention, do your homework to make sure that it is worth the investment of your student’s time and your money.