| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Clinical Forum |
Contact author: Sandra L. Gillam, Department of Communicative Disorders and Deaf Education, Utah State University, 1000 Old Main Hill, Logan, UT 84322-1000. E-mail: sgillam{at}cc.usu.edu
 |
ABSTRACT |
|---|
 TOP ABSTRACT THE SEVEN-STEP EBP DECISION...AN EXAMPLE OF AN...CONCLUSIONREFERENCES |
|---|
PURPOSE: The results of recent survey studies suggest that speech-language pathologists base most of their clinical decisions on information they were taught during their graduate programs, their clinical experience, and the opinions of colleagues (T. Wolf & J. Balderson, 2005; R. Zipoli & M. Kennedy, 2005). This is contrary to the principles of evidence-based practice (EBP), in which clinical decisions arise from the integration of scientific evidence, clinician experience, and client needs. Our field's interest in EBP is relatively young. Currently, there are no published committee-derived EBP guidelines for providing language intervention services for children with language disorders. Until national or international organizations publish recommendations from EBP guideline writing panels, clinicians will need to make personal evidence-based decisions with relatively little assistance from outside sources. The purpose of this article is to summarize a seven-step process for making local EBP decisions.
METHOD: The authors provide information about a method for forming clinical questions, finding relevant scientific studies, and evaluating those studies that requires relatively little time and few external resources. The authors also provide a system for integrating scientific evidence with their own expertise and training within the context of their employment settings. Finally, an example is provided to show clinicians how to use the evidence-based decision-making process to answer a clinical question about clinical methods for improving grammatical morphology in children with speech-language impairment.
CONCLUSION: It is possible for clinicians to make time- and resource-efficient evidence-based decisions that integrate scientific evidence, clinical expertise, and student–parent preferences.
KEY WORDS: evidence-based practice, language impairment, phonological awareness
Research-driven practices such as pharmaceutical treatments for acquired immune deficiency syndrome (AIDS) and surgical procedures for organ removal and transplantation have significantly impacted health care around the world. When physicians and surgeons make health-related decisions that are based on the careful and reasonable integration of knowledge obtained from published research, clinical experience, and patient needs, they are implementing a standard of care known as evidence-based practice (EBP; Porzsolt et al., 2003). Physicians employing EBP seek out the best research on the treatment of a problem and then make clinical decisions that are based on a synthesis of the research results, the patient's wishes, and their own clinical experience (Sackett, Rosenberg, Muir Gray, Haynes, & Richardson, 1996). Recently, there have been attempts to apply the principles of EBP in diverse fields such as education, clinical psychology, and speech-language pathology.
For years, speech-language pathologists (SLPs) have been taught to consider their client's wishes and their own experience in making clinical decisions. Similarly, clinicians have been encouraged to stay abreast of the current literature so that they can use research results to inform their clinical decisions. Until recently, there have been few resources that show clinicians just how to do that. The current emphasis on EBP differs from prior approaches to clinical decision making in that it provides clinicians with models for systematically collecting and reviewing research evidence and ways to make treatment decisions by integrating the best scientific evidence with personal experience, client preferences, and employer policies.
Evidence-based decisions can be made on global and local levels. At the global level, groups such as the Scottish Intercollegiate Guidelines Network (SIGN), the Campbell Collaboration, the Cochrane Collaboration, and the Academy of Neurologic Communication Disorders and Sciences (ANCDS) support the creation of guideline documents that contain recommendations for assessment and intervention practices that are based on a careful review of current evidence. Each organization employs its own process for developing practice guidelines. For example, the SIGN guideline development process begins with the formation of a representative guideline development group, which is composed of 15 to 20 professionals and lay persons. This group works through a series of 50 steps that involves specifying a clinical question, reviewing the literature related to the question, grading the evidence, synthesizing the evidence, and forming clinical recommendations, which receive widespread peer review. Even with the assistance of professional librarians and biostatisticians, this process takes approximately 2 years. The SIGN methodology and SIGN guidelines in many areas of health are available for free on the SIGN Web site (http://www.sign.ac.uk/index.html).
EBP guidelines that have been developed through careful and comprehensive reviews of the literature pertaining to a clinical question are extremely useful. Unfortunately, we know of no published guidelines pertaining to language intervention with school-age children. The American Speech-Language-Hearing Association (ASHA) has established the National Center for Evidence-Based Practice in Communication Disorders (N-CEP), which has begun to institute and support guideline writing panels. However, it will be years before comprehensive practice guidelines are published in the area of child language assessment and intervention. In the meantime, clinicians will have to make their own EBP decisions with the time and resources that are available to them. Unfortunately, most clinicians report that extra time and resources for EBP are in short supply (see Mullen, 2005). There is a need, then, for an efficient method for making EBP decisions.
In this article, we adapt Perzsolt et al.'s (2003) EBP decision-making process (see Table 1) to intervention decisions that must be made by school clinicians who treat children with language impairments and language-based learning disabilities. This decision-making process provides a way for clinicians to systematically integrate research evidence with their clinical knowledge and experience, parent–client preferences, and school district guidelines. One strength of this procedure is that it does not require an undue amount of time. We will work our way through the steps in the process and then answer a clinical question about intervention for grammatical morphology.
|
|
THE SEVEN-STEP EBP DECISION-MAKING PROCESS |
|---|
|
TOPABSTRACTTHE SEVEN-STEP EBP DECISION...AN EXAMPLE OF AN...CONCLUSIONREFERENCES |
|---|
Other clinical questions may be more specific. For example, the clinician may want to know whether she should replace the approach she usually uses to teach narration with a different intervention approach that parents are asking for. Specific clinical questions can be stated in a "PICO" format (Straus & Sackett, 1998) that is useful for formulating a question about a clinical case. Each letter in the acronym corresponds to a word or phrase that corresponds to one part of the clinical question.
The "P" refers to the patient or client characteristics and the problem that he or she is experiencing. For example, "P" might refer to preschool children with expressive language impairment or school-age children with deficits in receptive vocabulary.
The "I" represents the intervention program, approach, method, or technique that you are seeking to find evidence about. The "C" refers to the comparison treatment. For example, a clinician might be interested in comparing the language outcomes of children who received Fast ForWord—Language (FFW–L) (I) to the language outcomes of children who received traditional school therapy (C).
Outcome is represented by "O." Outcomes are usually assessed with a standardized measure of performance. For children with language impairment, the outcome measure might be improved performance on a measure of expressive and/or receptive language or vocabulary. Table 2 provides examples of PICO questions related to child language intervention.
|
Fortunately, the ASHA Web site is a valuable resource for gaining access to current research in our field. ASHA members can use the Web site to access articles that have been published in ASHA journals and in hundreds of other journals affiliated with High Wire Press. Members can reach the literature search area of the ASHA Web site by selecting "ASHA Full-text Journals" from the "ASHA and NSSLHA Members" menu. The Web site contains journal articles dating back to 1980 in .pdf format. ASHA and National Student Speech Hearing and Language Association (NSSHLA) members can download any number of articles for free. Clinicians can also find reference lists for research studies in child language at the Bamford-Lahey Web site (http://www.bamford-lahey.org/ebp.html).
One advantage of using the Web to perform literature searches is that a large number of articles can be located and downloaded quickly and conveniently. However, it is not always easy to find studies related to a particular topic. Recently, the authors asked two separate cohorts of students to use the procedures described in this article to find and review articles related to particular clinical questions. After working on the assignment for a week, a number of students reported that there was no evidence available on the topic they had been assigned. We were able to locate evidence on all of the students' topics. The problem that the students ran into was that they did not use a wide variety of search terms. Success in locating relevant articles often requires persistence and creativity in selecting appropriate key words. Simply typing in one series of terms rarely yields successful search results.
After reading the titles and abstracts of articles that are located, clinicians should review those studies that appear to be the most relevant to their clinical question. The review process is described in the next step.
Step 3: Determine the Level of Evidence and Critically Evaluate the Study
Speech-language pathologists (SLPs) want to base their clinical decisions on the best evidence that is available. They need to obtain the results of the highest quality studies that have been conducted (Dollaghan, 2004; Johnson, 2006). Generally, high-quality studies are those that follow research designs that are known to yield reliable results. One approach to selecting the best studies to review involves rating the research design (known as the level of the evidence) that a study employed. Then, the clinician assesses the quality and findings of just the highest ranked studies. There are a number of systems for rating levels of evidence. We have adopted the levels of evidence from the Oxford Centre for Evidence-Based Medicine (2001). As can be seen in Table 3, the Oxford system consists of five levels of evidence that comprise a continuum from highest (1) to lowest (5).
|
For example, an RCT could examine the efficacy of conversational recasts to facilitate grammatical morpheme use for children with specific language impairment (SLI) who are between the ages of 5;0 (years;months) and 7;0. Children may have been randomly assigned to one of two intervention programs or a comparison group (control) that received no intervention. One intervention program provided instruction through the use of conversational recasts; the other by having children practice using grammatical structures in an interactive computer program. All children were exposed to the same grammatical morphemes and spent the same amount of time in intervention daily and weekly. Performance on measures of the use of grammatical morphology obtained before and after intervention would be compared for students across the three groups. In this design, the effects of one intervention approach could be compared directly to the other intervention approach and against a control group who received no intervention.
Not all RCTs are designed and conducted equally well. That is why clinicians need to appraise the quality of the studies they select for review. Various aspects of the design of a study influence the reliability and generalizability of the results. Some important questions to consider in evaluating the studies that are selected for further review are listed and described in Table 4. We award one appraisal point for each "yes" answer to a question. A study that receives more "yes" answers to the questions is considered to be higher in quality, reliability, and generalizability than a study that receives fewer "yes" answers to the questions. The more appraisal points that can be applied to a study, the better the evidence, and the more faith a clinician can have in the results.
|
Level 2 external evidence includes comparisons of nonrandomized groups, quasi-experimental studies that evaluate changes over time (known as cohort studies), or systematic reviews of these two types of studies. In conducting a study with nonrandomized groups, a researcher may identify 10 children with language impairment and match them according to age and mean length of utterance (MLU) with 10 other children with language impairment. One group of children is assigned to participate in two, 30-min therapy sessions, twice weekly for 4 weeks. As the clinicians and children play with toys, clinicians expand the children's utterances to be more syntactically complete. The other group plays with a sibling for the same amount of time. Pre- and post-language samples are obtained from all 20 children, and their MLUs are compared. Results suggest that children who participated in the expansion program demonstrated longer MLUs than did children who did not participate in the program. Children are seen 6 months later and language samples are obtained and examined again. Findings reveal that the children who participated in the expansion intervention program continue to demonstrate significantly longer MLUs than the children who did not participate in the program.
Another type of Level 2 external evidence is a quasi-experimental cohort study, which evaluates change in two groups of children whose performance is measured before and after an intervention. For example, a study could examine the use of the key word strategy (mnemonic visualization strategy) for teaching new vocabulary to children with language impairments. Twenty children with language impairments attending a summer school program are assigned to two different classrooms by the school administration. The children receive a vocabulary pretest before the summer session begins. The SLP in charge of one classroom uses the key word strategy to teach vocabulary in stories that she reads aloud. The SLP in charge of the other classroom reads the same stories aloud but does not provide vocabulary instruction. Children are posttested at the end of the summer program. Success of the key word intervention program is measured by comparing the gains that the children in the two classrooms made on the vocabulary measure.
Most researchers in speech-language pathology also include multiple-baseline (or single-subject design) studies in Level 2. Multiple-baseline studies are designed so that each subject serves as his or her own control. This can be accomplished by measuring two or more behaviors across no-treatment and treatment phases. To illustrate, consider a study of two therapy approaches for improving grammatical morphology. Four children with language disorders are randomly assigned to one of two treatments: focused stimulation or imitation. In the baseline phase of the study, pronoun usage and auxiliary verb use are probed for five sessions before treatment is initiated. The probes demonstrate that none of the children use the target forms in more than 25% of their obligatory contexts. During the treatment phase, one child assigned to focused stimulation treatment and one child assigned to imitation treatment begin working on pronouns. Both pronoun and auxiliary verb usage are probed after each session. The focus of treatment switches to auxiliary verbs after a child makes 60% improvement on pronouns. The other child assigned to each type of treatment works on the targets in the opposite order: auxiliary verbs followed by pronouns. The researcher looks for steady improvements on the language skill that was the focus of treatment (but not on the other language skill), regardless of the order that the skills were treated in (pronouns before or after auxiliary verb usage).
Level 3 external evidence includes case-control studies of children who receive a particular treatment. For example, a researcher might report on the language and academic abilities of children who have been diagnosed with auditory processing disorders that did and did not improve on auditory processing measures across a 2-year period. The histories of the children would be examined for the co-occurrence of improvement on auditory processing tests and performance on measures of vocabulary, grammar, and phonological awareness that were collected 2 years before the outcomes were known.
Case studies in which the researcher provides a detailed description of a child and his or her treatment are included in Level 4. To illustrate this type of study, a researcher might describe the learning characteristics of a child with a language-based learning disability who has difficulty comprehending written language. Comprehension performance in text is assessed using comprehension questions and story recall. The investigator provides extensive and carefully collected background information on the child's academic performance and his speech, language, and hearing ability. Previous therapy is described in detail along with the outcomes of these treatments on passage comprehension. The researcher describes a treatment program for explicitly teaching comprehension strategies, and the results of the treatment are assessed by administering benchmark tests of comprehension that are part of the school district's curriculum.
School district data, including systematically collected archival records, may also be considered to be Level 4 external evidence. In the National Outcomes Measurement System (NOMS) project administered by ASHA, school districts participate in a data collection process that evaluates the outcomes of speech and language intervention. Clinicians use a rating system that evaluates communication ability in multiple speaking contexts at the beginning and end of the school year. This would not be on the same level as a case control study (Level 3) because all clinicians in a district do not use the same therapy approach with every child.
The lowest level of external evidence (Level 5) includes expert committee reports, conference proceedings, and/or opinions of respected authorities. ASHA technical reports and preferred practice patterns reflect the general consensus of committee members. These documents are considered to be the lowest level of external evidence because the therapy recommendations are not based on objective data. Clinicians should be cautious about implementing therapy suggestions from experts unless those suggestions are based on well-designed studies or systematic reviews of high-level evidence.
EBP does not require clinicians to use external research evidence as the only basis for clinical decisions. When making decisions about the value and desirability of a technique, clinicians should consider the external research evidence in light of internal factors related to student, parent, and clinician beliefs and opinions as well as the resources and policies of the agency that the clinician works for.
Step 4: Evaluate the Internal Evidence Related to Student–Parent Factors
A number of student–parent factors should be considered as part of the EBP decision-making process. A continuum similar to that for ranking research evidence can be used to weight student–parent factors. Our weighting system is only an example because the levels we suggest may or may not apply to a particular caseload. Each clinician should decide on a weighting system that best fits the students he or she treats and the particular features of the work setting. On the basis of our experience as school clinicians, we have included the following factors in our decision-making process: strong cultural values and beliefs, student activities, financial resources, level of student–parent engagement, and student–parent opinions.
The highest level of student–parent factors (Level 1) is strong cultural values and beliefs. We attach such importance to cultural values that we weighted them at the same level of importance as RCTs and systematic reviews of RCTs. In fact, we can imagine some situations where cultural values would trump all other evidence. For example, some children who are Native Americans are raised in environments in which it is inappropriate to ask adults certain kinds of direct questions (Joe & Malach, 1998). An approach to language therapy that involves instruction in questioning strategies would not be desirable for a child from that culture even if external evidence from systematic reviews and/or RCTs supports the efficacy of the therapy procedure.
The Level 2 student–parent factor is related to student activities and participation. Clinicians should select interventions that are associated with the kinds of activities that children find motivating and enjoyable. For example, we often use a therapy approach known as literature-based language intervention (Gillam & Ukrainetz, 2006). In this approach, language lessons are tied to the content of children's books. The effectiveness of this intervention has been demonstrated in an RCT (Gillam et al., in preparation). We believe this approach is useful because it is consistent with the types of classroom activities that children experience in school settings.
The third student–parent internal factor relates to the family's financial resources (Level 3). With the exception of public school services, families must bear the cost for many language intervention services. When two equally useful treatment approaches are compared, it makes sense that the least expensive of these be chosen.
Student–parent engagement (Level 4) refers to the amount of active participation that the child and his or her family are expected to demonstrate in the implementation of the program. Some language intervention approaches require a great deal of family participation. If the family will not participate at the level required to make reasonable progress in the program, it may not be time effective for the clinician or the client to pursue that approach. The choice of a similarly useful regimen that requires less time commitment on the part of the family might be more practical when the level of student–parent engagement is low.
The lowest consideration in deciding whether or not to use an intervention (Level 5) is to base a decision solely on a parent's or student's belief about a therapy that is not founded in the evidence. Parents sometimes request a particular therapy that they have heard about from mass media reports or from surfing the Internet. Other times, parents want their children to receive a treatment that they had. Suppose a parent believes that her child with language impairments should learn new vocabulary by looking words up in the dictionary. However, the clinician knows that the use of mnemonic devices has been shown to be more effective than traditional instruction (dictionary usage) for teaching academic vocabulary (Jittendra, Edwards, Sacks, & Jacobson, 2001). In this case, the clinician should explain to the parent that the evidence indicates that teaching mnemonic devices is more effective than dictionary training.
Step 5: Evaluate the Internal Evidence Related to Clinician–Agency Factors
The third tier in the decision-making process includes the consideration of clinician and agency factors. A continuum similar to that for ranking student–parent factors can be used to weight decisions based on factors that are specific to the clinician and to the agency or school district (Table 3). Clinicians should consider their education, the culture or policies of their agency or school district, data they have collected on the children they teach, their own theoretical orientation, and recommendations from other clinicians when making treatment decisions. Similar to the internal evidence related to student–parent factors, the system we suggest for assigning weights to clinician and agency factors may or may not apply to a variety of circumstances or caseloads. Clinicians should decide on a weighting system that best fits the settings they work in.
We do not believe that any clinician–agency factors should weigh as strongly as external evidence from RCTs, systematic reviews of RCTs, or a parent's strong cultural beliefs. But we do place a high value on prior knowledge and skills. Therefore, we placed education, the highest clinician–agency factor, at Level 2. Many clinicians are highly competent at implementing treatments that they have studied carefully, that they learned in graduate school, or that have been demonstrated to them by master clinicians. That is why clinicians should consider their own skills when they weigh treatment alternatives.
Agency and school district policies and financial resources (Level 3) often influence therapy decisions. For example, clinician schedules and class schedules often affect recommendations about the number and length of a child's treatment sessions. It is very difficult for clinicians to recommend that a child receive therapy for 1 hr per day if she is only scheduled to be at the child's school on Tuesdays and Thursdays. There are also situations in which clinicians are strongly encouraged to use a curriculum or therapy program that school district officials have purchased.
The data that individual clinicians collect on the children they treat is weighted as Level 4 evidence. We believe this is consistent with Level 4 external research evidence from single case studies. Over time, a clinician may collect data on 15 or 20 children who received the same kind of treatment. This data would provide a good indication of the range of outcomes that an individual clinician obtains. Clinician-generated outcome data from many children who received the same type of treatment would be consistent with studies of multiple cases (Level 3 external evidence). Whether clinicians have data from a few cases or many cases, they should compare their own outcome data to published outcomes to determine whether their intervention routinely yields results that are poorer than, similar to, or better than the results that were reported in a journal article.
We weighted a clinician's theoretical orientation as Level 5 evidence because it is consistent with expert opinion, which was placed at Level 5 of external evidence. Some clinicians have philosophical convictions that are consistent with or in opposition to a particular therapy approach. For example, a clinician who believes strongly in social interactional theories of language development might have difficulty implementing a behavioral therapy approach. If external evidence shows that two types of treatment yield similar outcomes, and if both treatments are consistent with a child's needs and interests (student–parent internal evidence), then the clinician should select the treatment that is the most consistent with his or her philosophical orientation.
Clinicians are also influenced by intervention suggestions made by other clinicians they respect (Level 5 internal evidence). Similarly, when a clinician adds a new child to his or her caseload, the intervention recommendations that have been made by the child's previous clinician (Level 5) can serve as a good starting point for intervention decisions.
Step 6: Make a Decision by Integrating the Evidence
The implementation of EBP is easiest when the beliefs and opinions of parents, students, clinicians, and agencies are consistent with (or are not in conflict with) external evidence. It is easy to decide on a course of treatment when results of well-designed studies support the use of therapy procedures that are consistent with the child's interests, the parent's wishes, and what the clinician knows from his or her training and experience (Rycroft-Malone, 2004). The integration of these factors is represented in Table 5.
|
|
Clinicians can individualize their outcome evaluations to their clients by noting instances in which target language structures are used correctly (without prompts or cues) in natural speaking situations. In addition, clinicians can obtain valuable information from parents, teachers, and other professionals who work with the children on their caseloads by asking questions about changes in the type and amount of the child's communication in daily activities.
As noted by McCauley (2001), school districts often place a great deal of value on the results of standardized tests. Clinicians who are required to administer standardized tests annually can use two techniques to evaluate the outcomes of their therapy. First, authors of research articles sometimes report the pretest and posttest standard scores for the participants in their studies. This enables clinicians to compare the average gains in standard scores from published research articles to individual gains in standard scores for the children on their caseloads.
Due to measurement error, standard scores are really estimates of performance. Clinicians can use confidence intervals to determine whether a gain in a standard score lies outside the typical range of error. This can by calculated by looking up the standard error of measurement (SEM) in the test manual. Imagine that a child received a pretest standard score of 81 on a test that had a SEM of 4. Theoretically, if the test was administered over and over during the same week, the child's score should fall between 77 and 85 (±1 SEM) 68% of the time. The child's score should fall between 73 and 89 (±2 SEM) 95% of the time. At posttest, if the child obtains a standard score of 90, the clinician can be 95% sure that the change is not due to test error alone. Generally, children have made noticeable gains if their posttest standard scores are three or more points above the upper range of the 95% confidence interval.
|
AN EXAMPLE OF AN EVIDENCE-BASED DECISION |
|---|
|
TOPABSTRACTTHE SEVEN-STEP EBP DECISION...AN EXAMPLE OF AN...CONCLUSIONREFERENCES |
|---|
Step 1
We posed the question, "What language intervention (I/C) approaches have been shown to be effective in increasing grammatical morphology (O) in school-age children with language impairments (P)?"
Step 2
In order to answer this general question, we first went to the search engine, Academic Search Premier, and typed in grammatical morphology, school age children, and specific language impairment. This search yielded no articles. We then changed the term school age children to children, keeping the terms grammatical morphology and specific language impairment. This search yielded 19 articles, none of which focused on intervention on grammatical morphology for children with SLI. Most of the articles compared children with SLI to typically achieving children, children with other types of disorders (dyslexia, autism, Down syndrome), or children learning English as a second language (Hebrew, French, Spanish). A few of the articles that were located were tutorials about the theoretical hypotheses about the cause of SLI (optional infinitive, general processing, specific processing) or diagnostic markers of SLI.
We then changed the search terms to intervention, grammatical morphology, and specific language impairment. This yielded no articles. When we changed the search terms to intervention, grammar, and specific language impairment, we located one article on the typology of SLI, one article concerning imitative production of past –ed, one article on the use of theoretical models in the development of language, and two articles that looked as though they might answer the clinical question. These articles included a paper by Leonard, Camarata, Brown, and Camarata (2004); a Level 2 data-based study; and a review paper by Fey, Long, and Finestack (2003).
Having located only one data-based study that reached Level 2 evidence, we decided to try the ASHA Web site. We went to the ASHA Web page and selected http://highwire.stanford.edu. This connected us to HighWire Press, which provides access to more than 900 journals and 1,000,000 full-text articles. We typed search number one as above (grammatical morphology, school age children, and specific language impairment) into the line "anywhere in text" and selected "all." We then checked "All (including PubMed)" with no limitation on the dates. This yielded 348 articles. After perusing all of the 348 titles and abstracts and eliminating articles that did not address the specific question of grammatical morphology in children with SLI, we located the same Leonard et al. (2004) and Fey et al. (2003) articles, plus two more articles: Proctor-Williams, Fey, and Loeb (2001) and Fey, Cleave, Long, and Hughes (1993). Then, we typed search number two (grammatical morphology, children, and specific language impairment) and received 431 hits. There was one additional article that had not appeared initially: Fey et al. (1993). Using the third set of terms (intervention, grammatical morphology, and specific language impairment) yielded references for 274 articles. There were no intervention studies that had not appeared in the earlier searches. Our last search, using the terms intervention, grammar, and specific language impairment, yielded 514 articles. There were some additional articles, including ones by Nelson, Camarata, Welsh, Butkovsky, and Camarata (1996); Camarata, Nelson, and Camarata (1994); Law, Garrett, and Nye (2004); and Ellis Weismer and Murray-Branch (1989). Other articles that were not relevant to our clinical question addressed the use of grammar facilitation for improving phonological performance, narrative intervention procedures, categorization of grammatical morphology problems into functional categories, grammatical development in bilingual populations, grammaticality judgments for the extended optional infinitive account, proposals for classification of children with SLI, problems that kids have with grammatical morphology, reading outcomes, and the validity of parent reports for measuring vocabulary and syntax.
Step 3
The next step in the process is to review relevant articles in order to rate the level of evidence and to assess their quality according to the appraisal points in Table 4. Our search yielded one systematic review of RCTs: Law et al. (2004). This study is a systematic review of intervention studies involving children with speech and language delays/disorders. Law and his colleagues located 33 RCTs, 25 of which were ultimately included in the review. However, only 13 articles were judged to be similar enough to be combined for statistical analysis. In terms of our clinical question regarding syntax, Law et al. reported that speech and language therapy may or may not be effective for children with expressive syntax difficulties. Four of the five studies on expressive syntax reported improvements resulting from the use of indirect language stimulation and variations in the use of focused stimulation techniques. None of the articles investigating improvement in receptive syntax were found to result in significant improvements. One of the articles that turned up in our search (Fey et al., 1993) was included in Law et al.'s systematic review. We will not review it further in this paper.
We decided to review the studies that had not been included in the Law et al. systematic review. Leonard et al. (2004) investigated the use of focused stimulation in facilitating the use of third-person singular –s and auxiliary is/are/was. Thirty-one preschool-aged children received focused stimulation with grammatical contrasts presented during story reading and conversational recasting. Children were assigned to an intervention group that focused on either third-person singular –s (3S) or auxiliary is/are/was (aux) but not both. Assignment to the intervention group was site specific but was not randomized. The clinicians who provided the intervention were not aware of the overall purposes of the investigation or of all of the specific targets that were being investigated.
The children in the 3S group received 12 weeks of intervention that resulted in a total of 48 contact sessions. Sessions were held two times daily, twice per week. Children were told stories (a total of 48, one per session) that contained a minimum of 12 exemplars of the target morpheme and then the clinician re-told the stories using a set of corresponding toys. Children were encouraged to play with the toys. During play, clinicians used conversational recasts of utterances produced by the child that contained the morpheme being targeted. When children did not provide utterances that were suitable for recasting, clinicians engaged children with the toys in the hope of eliciting utterances for use in recasting targets.
The aux group received treatment that was similar in terms of the use of focused stimulation stories and props to facilitate the use of target forms through conversational recasting. For this group, recasts were equally divided among is, are, and was rather than focusing on only one form, as was the case for third person –s. Fidelity of the treatments was controlled through prior training in the use of the programs, very structured implementation guidelines, and strict observation of 22 treatment sessions.
This Level 2 study satisfied six of eight appraisal points (Table 7). All of the children made similar gains regardless of the program they participated in. Children in both groups performed significantly better (p < .001) on third person –s and auxiliary is, are, and was than on past –ed or on the control forms (d = .84–1.19). Further, children demonstrated use of one non-target tense/agreement form to a degree that was significant in comparison to past –ed or the control forms. Thus, focused stimulation and conversational recasts were associated with gains in grammatical morphemes that were targeted, and some generalization of use to other forms was observed. The completed appraisal form for Leonard et al. (2004) is shown in Table 7.
|
Camarata et al. (1994) compared the use of imitative procedures to conversational recasts or "growth recasts" for improving grammatical morphology and complex syntax use in 21 children with SLI ranging in age from 4;0 to 6;7. Language samples were obtained and targets were selected based on grammatical morphemes and complex syntactic structures that were not used in obligatory contexts. Targets were randomly assigned to either the imitative or the recast condition.
In the imitative condition, children received a model and a prompt paired with a picture of object stimulus. A token economy was used to reward correct responses. Once the 90% criterion was met on a target, the model was faded out. A transfer phase followed, wherein children were provided multiple opportunities to practice new skills in structured activities during which models, prompts, and imitation were encouraged.
In the recast condition, the environment was structured to elicit child attempts at targets. That is, toys and activities were chosen that obligated certain structures while playing. When children produced an utterance and omitted a target, the clinician recasted the utterance in a way that retained basic semantic information and also included the target.
Spontaneous use of targets occurred more often in the conversational recast procedure than in the imitative procedure (d = .61). Similarly, imitative use of targets occurred more often under the imitative procedure (d = 2.31). This study meets Level 3 requirements and earned five of eight appraisal points.
An article by Nelson et al. (1996) reported the results of a study that expanded on their previous study of imitative and conversational recast procedures. This study included a control period in which no structures were targeted. Children with SLI and children with similar language abilities who were developing typically were compared in terms of their responses to these procedures. The results of this study revealed that language structures that were targeted under either imitative or conversational recast procedures were more often acquired to criterion in at least one generalized spontaneous production than were structures in the control condition. Further, for targets that were absent before intervention, the conversational recast procedures resulted in faster acquisition of morphemes than in the imitative condition for both typically developing children (d = 2.45) and children with SLI (d = .42). Notice that the effect size for children with SLI was much lower than the effect size for typically developing children. This study met six of eight appraisal points.
Proctor-Williams et al. (2001) investigated the relationship between parent recasts of copula and articles to preschool children with SLI or to language-matched typically developing children. Children's language was sampled before beginning the study, at the midpoint (4 months), and again at the conclusion of the study (8 months). Parents and children participated in free-play for 30 min each session, and these interactions were both video- and audio-taped.
Results indicated that parents did not differ in terms of the rate or percentage of recasts that they used with their children (typical and SLI). Rate of copula recasts by parents of children with typical language was significantly correlated with the rate of use (r = .63) and accuracy (r = .87). This was not the case for parents of children with SLI. Camarata, Nelson, and Camarata (1994) reported a rate of target-specific recasts that was 4.7 times greater than recasts produced by parents in the Proctor-Williams et al. (2001) study. Fey et al. (1997) suggested that children with SLI could benefit from recasts when they are presented at a much higher rate than is typical during ordinary conversations with children. This study met five of eight appraisal points and constituted Level 2 evidence.
The last study we reviewed was by Ellis Weismer and Murray-Branch (1989). These authors examined modeling versus modeling plus evoked production training in 4 children ages 5;5–6;11 with language delays. Children participated in a single-subject design treatment program with baseline phases. In the modeling procedure, models of the target form (i.e., possessives, plurals, past tense) were provided by the clinician without requiring a response from the child during the context of play activities. The modeling plus evoked production procedure involved use of models of the target form with intermittent opportunities for the child to respond and/or receive feedback about the accuracy of their utterances. The model plus evoked production was also provided within the context of play activities.
Results suggested that there were no significant differences between the two treatment approaches. That is, for the 3 children with expressive language problems, both treatment programs worked equally well. For the child with a delay in both expressive and receptive language, neither treatment was effective. No means or standard deviations were reported. This Level 2 study met three of the eight appraisal points. We should point out that the appraisal system was developed for assessing group studies. Therefore, single-subject design studies are, by their nature, at a disadvantage when they are appraised according to this system.
All of the studies reviewed were relevant to the population of interest (SLI) and all concerned intervention with preschoolers. In terms of levels of evidence, our search yielded three articles that were judged to be Level 1 evidence: the Law et al. (2004) systematic review; the study by Fey et al. (1993; included in the Law systematic review); and Fey et al. (1997). Both studies earned six of eight appraisal points and reported significant improvements in grammatical morphology using focused stimulation procedures. The findings from these studies would support the use of either clinician-directed or parent-directed intervention.
Step 4
The clinician should now evaluate the internal evidence related to student–parent factors. We can imagine a case in which the child lived in a single-parent home with multiple siblings. This parent may not be able to dedicate 2 hr per week for 12 weeks and then 1 hr per week for an additional 12 weeks to intervention because of employment and child care issues (Level 2 student–parent factor). Further, this child may be more interested and engaged when working with a clinician independently or in small groups (Level 4 student–parent factor).
Step 5
With respect to clinician–agency factors, a clinician who has been trained in the use of focused stimulation techniques would be more likely to use this procedure (Level 2). However, a clinician with a large individual caseload and a special education director who was concerned about the high time commitment and cost associated with the provision of clinician-directed therapy (Level 3 clinician–agency factor) may be more likely to use parent-directed procedures.
Step 6
The clinician must now make a decision by integrating the external and internal evidence. We have stated that the parent does not have the available time to commit to a parent-directed program and that the child responds well to clinician-directed intervention. Even though the clinician has a large caseload, it may be more beneficial for the child to receive clinician-directed intervention than parent-directed intervention in this hypothetical case.
It is important to note that research on syntax intervention has shown that contingent facilitation devices such as imitation, focused stimulation, expansions, and conversational recasts can be effective. The problem for clinicians who work with school-age children is that the evidence supporting the developmental effects of these procedures has been obtained from research with preschool children who are working on basic language structures such as past tense or the "be" system (copula and auxiliary). The difficultly for making EBP decisions in public school settings is that clinicians are faced with making decisions based on experimental literature that is not specific to the children they are treating or the environments they are working in.
Step 7
After implementation, the success or failure of the program would be evaluated in terms of specific linguistic outcomes as well as the levels of satisfaction expressed by the parents, teachers, and other professionals who work with the child. The clinician in our example could probe the child's use of the grammatical morphology targets during short language samples at the end of each week. After 4 or 5 months of therapy, the clinician should see an increase in the number of child utterances in which the targeted grammatical morphemes were used correctly. The clinician should also ask the child's parents and teacher whether they have observed any changes in the complexity of the child's language or in the amount of his or her communicative participation during daily activities.
|
CONCLUSION |
|---|
|
TOPABSTRACTTHE SEVEN-STEP EBP DECISION...AN EXAMPLE OF AN...CONCLUSIONREFERENCES |
|---|
We have presented a decision-making process that can integrate information from external research articles, student–parent factors, and clinician–agency factors to make client-specific evidence-based decisions. Searching for, reading, and evaluating external evidence is not a simple process. In our research for this article, we found that the HighWire Press search engine that is accessible from the ASHA Web site (which is free to all ASHA members) yielded a more comprehensive list of potential articles than "Academic Search Premier," which is a search engine that is available in many university libraries. We spent approximately 2 hr searching databases, reading 1,500 titles and/or abstracts, and making decisions about article relevance using the "All (including PubMed)" option on the HireWire Press search engine. We noted that in the largest of the four searches using this option (resulting in 514 hits), articles were often referenced more than once. It is important to note that all of the articles we were ultimately interested in were found in ASHA journals. In addition, it was never the case that we found a relevant article below #200 on the search list. When we tried using the same search procedures while restricting our search to ASHA journals only, we found the same set of articles that were appropriate for our review with fewer titles to scrutinize and less duplication in the citations. Further, the articles we were looking for appeared earlier in the citation lists.
The seven-step decision-making process described in this article provides a method for making evidence-based decisions that integrate research results, student–parent factors, and clinician–agency factors. Our suggestions for the level of internal evidence that were related to student–parent factors and clinician–agency factors were based on our experience as school clinicians. Clinicians may wish to re-weight these factors to be more consistent with their caseload, their employment settings, and/or their own values. Collecting and evaluating external evidence can be time consuming. The ASHA Web site provides clinicians with a helpful tool for finding relevant articles. However, EBP decision making will be much easier when peer-reviewed guidelines on assessment and intervention for a variety of communication disorders in school-age children are readily available. Until that time, clinicians can use procedures like the ones in this article to make evidence-based decisions that are specific to the children they serve and the clinical contexts they work in.
|
ACKNOWLEDGMENTS |
|---|
Received December 8, 2005
Accepted May 2, 2006
|
REFERENCES |
|---|
|
TOPABSTRACTTHE SEVEN-STEP EBP DECISION...AN EXAMPLE OF AN...CONCLUSIONREFERENCES |
|---|
Camarata, S., Nelson, K., & Camarata, M. (1994). Comparison of conversational recasting and imitative procedures for training grammatical structures in children with specific language impairment. Journal of Speech and Hearing Research, 37, 1414–1423.
Cohen, W., Hodson, A., O'Hare, A., Boyle, J., Durrani, T., & McCartney, E., et al (2005). Effects of computer-based intervention through acoustically modified speech (Fast ForWord) in severe mixed receptive–expressive language impairment: Outcomes from a randomized controlled trial. Journal of Speech, Language, and Hearing Research, 48, (3) 715–729.
Dollaghan, C. A. (2004). Evidence-based practice in communication disorders: What do we know, and when do we know it? Journal of Communication Disorders, 37, 391–400.[CrossRef][Medline]
Ellis Weismer, S., & Murray-Branch, J. (1989). Modeling versus modeling plus evoked production training: A comparison of two language intervention methods. Journal of Speech and Hearing Disorders, 54, 269–281.
Fey, M., Cleave, P., Long, S., & Hughes, D. L. (1993). Two approaches to the facilitation of grammar in children with language impairment: An experimental evaluation..
Fey, M., Cleave, P., & Long, S. (1997). Facilitation in children with language impairments: Phase 2. Journal of Speech, Language, and Hearing Research, 40, 5–19.
Fey, M., & Loeb, D. F. (2002). An evaluation of the facilitative effects of inverted yes–no questions on the acquisition of auxiliary verbs. Journal of Speech, Language, and Hearing Research, 45, 160–174.
Fey, M., Long, S., & Finestack, S. (2003). Ten principles of grammar facilitation for children with specific language impairments. American Journal of Speech-Language Pathology, 12, 3–16.
Gillam, R. B., Loeb, D. F., Hoffman, L. M., Brandle, J., Bohman, T., & Champlin, C., et al (in preparation). Comparison of language intervention programs.
Gillam, R. B., & Ukrainetz, T. M. (2006). Language intervention through literature-based units. In T. M. Ukrainetz (Ed.), & Literate language intervention: Scaffolding PreK–12 literacy achievement(pp. 59–94) Eau Claire, WI: Thinking Publications.
Hook, P. E., Macaruso, P., & Jones, S. (2001). Efficacy of Fast ForWord training on facilitating acquisition of reading skills by children with reading difficulties—A longitudinal study. Annals of Dyslexia, 51, 75–96.
Jittendra, A., Edwards, L., Sacks, G., & Jacobson, L. (2001). What research says about vocabulary instruction for students with learning disabilities. Exceptional Children, 70, 299–322.
Joe, J., & Malach, R. (1998). Families with Native American roots. In E. Lynch, & M. Hanson (Eds.), Developing cross-cultural competence(pp. 127–164) Baltimore: Brookes.
Johnson, C. J. (2006). Getting started in evidence-based practice for childhood speech-language disorders. American Journal of Speech-Language Pathology, 15, 20–35.
Law, J., Garrett, Z., & Nye, C. (2004). Speech and language therapy interventions for children with primary speech and language delay or disorder. The Cochrane Database of Systematic Reviews Issue 3. Art. No: CD004110. Available at http://www.cochrane.org/reviews/clibacress.htm.
Leonard, L., Camarata, S., Brown, B., & Camarata, M. (2004). Tense and agreement in the speech of children with SLI: Patterns of generalization through intervention. Journal of Speech, Language, and Hearing Research, 47, 1363–1379.
McCauley, R. J. (2001). Assessment of language disorders in children Mahwah, NJ: Erlbaum.
Mullen, R. (November 2005). Survey tests member's understanding of evidence-based practice. The ASHA Leader, 4, 14.
Nelson, K. E., Camarata, S. M., Welsh, J., Butkovsky, L., & Camarata, M. (1996). Conversational recasting treatment on the acquisition of grammar in children with specific language impairment and younger language normal children. Journal of Speech, Language, and Hearing Research, 39, 850–859.[Medline]
O'Connor, R. (2000). Increasing the intensity of intervention in kindergarten and first grade. Learning Disabilities Research and Practice, 15, 43–54.[CrossRef]
Oxford Centre for Evidence-Based Medicine. (2001). Levels of evidence and grades of recommendation Retrieved December 4, 2005, from http://www.Cebm.net/levels_of_evidence.asp.
Pokorni, J. L., Worthington, C. K., & Jamison, P. J. (2004). Phonological awareness intervention: Comparison of Fast ForWord, Earobics, and LiPS. Journal of Educational Research, 97, 147–157.[CrossRef]
Porzsolt, F., Ohletz, A., Gardner, D., Ruatti, H., Meier, H., Schlotz-Gorton, N., & Schrott, L. (2003). Evidence-based decision making: The 6-step approach. American College of Physicians Journal Club, 139, (3) 1–6.
Proctor-Williams, K., Fey, M., & Loeb, D. (2001). Parental recasts and production of copulas and articles by children with specific language impairment and typical language. American Journal of Speech-Language Pathology, 10, 155–168.
Rouse, C. E., & Kruger, A. B. (2004). Putting computerized instruction to the test: A randomized evaluation of a scientifically-based reading program. Economics of Education Review, 23, 323–338.[CrossRef]
Rycroft-Malone, J. (2004). The PARIHS framework—A framework for guiding the implementation of evidence-based practice. The Journal of Nursing Care Quality, 19, 297–304.
Sackett, D., Rosenberg, W., Muir Gray, J., Haynes, R., & Richardson, W. (1996). Evidence based medicine: What it is and what it isn't. British Medical Journal, 312, 71–72.
Straus, S. E., & Sackett, D. L. (1998). Using research findings in clinical practice. British Medical Journal, 317, (7154) 339–343.
Wolf, T., & Balderson, J. (March 2005). Selection of therapy techniques for voice therapy Poster session at the Midsouth Conference on Communication Disorders, Memphis, TN.
Zipoli, R., & Kennedy, M. (2005). Evidence-based practice among speech-language pathologists: Attitudes, utilization, and barriers. American Journal of Speech-Language Pathology, 14, 208–220.
This article has been cited by other articles:
|
|
 |
|
  P. Hargrove, M. Griffer, and B. Lund Procedures for Using Clinical Practice Guidelines Lang Speech Hear Serv Sch, July 1, 2008; 39(3): 289 - 302. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
G. L. Lof and M. M. Watson A Nationwide Survey of Nonspeech Oral Motor Exercise Use: Implications for Evidence-Based Practice Lang Speech Hear Serv Sch, July 1, 2008; 39(3): 392 - 407. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
N. J. Lass and M. Pannbacker The Application of Evidence-Based Practice to Nonspeech Oral Motor Treatments Lang Speech Hear Serv Sch, July 1, 2008; 39(3): 408 - 421. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
T. Brackenbury, E. Burroughs, and L. E. Hewitt A Qualitative Examination of Current Guidelines for Evidence-Based Practice in Child Language Intervention Lang Speech Hear Serv Sch, January 1, 2008; 39(1): 78 - 88. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
F. M. Cirrin and R. B. Gillam Language Intervention Practices for School-Age Children With Spoken Language Disorders: A Systematic Review Lang Speech Hear Serv Sch, January 1, 2008; 39(1): S110 - S137. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J. Tilstra and K. McMaster Productivity, Fluency, and Grammaticality Measures From Narratives: Potential Indicators of Language Proficiency? Communication Disorders Quarterly, November 1, 2007; 29(1): 43 - 53. [Abstract] [PDF]
|
|
|
|
|
|
M. E. Fey Commentary on "Making Evidence-Based Decisions About Child Language Intervention in Schools" by Gillam and Gillam Lang Speech Hear Serv Sch, October 1, 2006; 37(4): 316 - 319. [Abstract] [Full Text] [PDF]
|
|
| |||||
