The Return of the Dodo Bird
St. John's University
The field of psychotherapy has been rocked by the work of Wampold and Colleagues (Wampold & Imel, 2015) and their claim that all psychotherapies are equally effective.
This idea has been around since the 1930s and is called the Dodo Bird verdict. The term comes from Lewis Carol's book Alice's Adventures in Wonderland, where the Dodo bird judges a competition and concludes at the end that, "Everybody has won and all must have prizes."
Although this claim remains a hot topic of debate (See special issue Psychotherapy, 2014 Vol 51 issue 4), the Dodo bird is a resilient species who speaks to ABCT. The Dodo bird asks, "Are not the different types of CBT equally effective for all disorders?"
The general field of psychotherapy has been plagued by the proliferation of different schools of therapy. Currently, there are more than 300 different psychotherapy orientations. This proliferation of theories, however, has done little to expand our knowledge. CBT is just one subset of psychotherapy, but in CBT, we have had a similar proliferation of therapies, with new versions of CBT appearing each year. At the First National Conference on Cognitive Behavior Therapies in 1977, the field consisted of traditional behavior therapies (which included exposure, relaxation training, behavioral activation, and operant procedures), Beck's Cognitive Therapy, Ellis' Rational Emotive Behavior Therapy, Meichenbaum's self-instructional training, the social problem solving approach, and Kanfer's self-control therapies. All of these approaches have continued to prosper and have supporting research that demonstrates that their hypothetical constructs correlate with disturbance and that their interventions work.
As time has passed, new variants of CBT have appeared. We now have the following additional treatments falling under the CBT umbrella: Acceptance and Commitment Therapy, Cognitive Processing Therapy, Compassion Focused CBT, Meta-Cognitive Therapy, Mindfulness Focused CBT, Rumination Focused CBT, Schema Focused CBT, Trauma Focused CBT, and Trial Focused CBT. Few practitioners could master all of these therapies with a wide range of disorders. One approach has anticipated this problem and has provided a generalized set of procedures to use across disorders; that is Transdiagnostic Treatment approach. Each of these approaches was created by and practiced by caring people who are committed to reducing human suffering using scientifically validated principles.
Most of us practice one of these CBT variants. We believe our favorite intervention is the most theoretically elegant, and would have the most scientific support if the research would compare it to the other variants of CBT. If you read the requests for referrals that come through our organizational list serve, you will frequently see requests for a specific geographic location for a client with a specific problem, from a specific variation of CBT. The author of the post implies s/he refers only to members that practice her/his favored therapy. But what would the Dodo bird say? Do we have evidence for the superiority of any form of CBT? We are stuck in a multiple model stage of science and we could move forward to a more theoretically elegant and parsimonious paradigm. To do so, we need to test the forms of CBT and prune some of the branches so that we have a few broad principles to guide our interventions.
The Dodo bird asks three scientific questions of CBT at this time. The first question is "Do all versions of CBT have equal efficacy?" "Are some forms of CBT more efficacious for some problems than for others?" Researchers often design their studies on the their preferred variant of CBT and attempt to demonstrate that it is more effective than a control condition and sometimes explore whether it is more effective than another form of psychotherapy. Rarely do we see studies that compare two or more forms of CBT. Should we assume that the Dodo bird verdict applies within CBT? At a recent St. John's University colloquium, Deb Biedel, from the University of Central Florida, presented data demonstrating that exposure using virtual reality presentations of combat scenes was more effective in reducing PTSD symptoms in veterans than were other forms of CBT designed for PTSD. She directly confronted the issue of whether two successful forms of CBT were equally effective. This was a helpful presentation, and the results suggested that a cognitive behavioral approach was not equally effective to the exposure treatment. These results have implications for future research. More researchers need to follow the lead of Deb Biedel. We should not avoid comparisons within our own family; we all want to discover the best procedures to reduce human suffering.
The second question involves the relative contribution of different hypothetical cognitive constructs proposed to arouse emotions and emotional disturbance. Different theories MAKE different prose that different thoughts and beliefs lead to emotions. Research in this area usually involves measuring one type of thoughts, beliefs, and cognitions that one variant of CBT proposes to relate emotional disturbance or a disorder. As a result, we have research supporting that each of these different cognitive processes are linked to emotional disturbance. We know that negative automatic thoughts, underlying schema, irrational beliefs, attributional styles, a social problem solving variables, and cognitive flexibility each contribute to emotional disturbance. However, we do not know their relative contribution. Perhaps all these cognitive variables contribute to one latent variable that represents negative thinking. Changing any one of them could lead to a change on all of them and the disturbed emotions. Perhaps some cognitive constructs contribute unique variance over and above the others. Perhaps each construct provides unique variance, or they provide unique variance for some problem but not others. If we knew which negative cognitions provide unique variance, we could focus on that cognitive variable and leave the rest behind. Another possibility is that there are individual differences in the negative cognitions that appear in people who are disturbed. That is, the cognitive variable related to disturbance differs from person to person. Some people could become disturbed through poor social problem solving, yet not hold negative automatic thoughts or irrational beliefs. Perhaps only one type of negative though is necessary to become disturbed even if the others are in the healthy range. Such knowledge could lead to a more prescriptive form of CBT.
Graduates students working with me at St. John's University have started comparing different cognitive constructs. Avi Block looked at the relationship between Obsessive Compulsive Disorder and Body Dysmorphic Disorder with irrational beliefs from REBT and a measure of flexibility and diffusion hypothesized by ACT. He found clear evidence that the ACT flexibility measure was more predictive of OCD symptoms than the irrational beliefs. Irrational beliefs, however, appear to play a larger role in predicting BDD symptoms. Also, Brittany Soto has collected data on high school students comparing their anger scores with measures of cognitions including awareness of the negative consequence of anger, irrational demands, irrational frustration intolerance, automatic thoughts of overgeneralization, and attributions for hostile intent. Awareness of the negative consequence of anger (consequential thinking) accounted for the most variance in anger, followed by irrational demands. The other cognitive variables accounted for little if any unique variance. We are replicating this research with college students and older adults. The next step would be to compare interventions for anger that targeted these cognitions to see if they lead to more effective treatment than targeting the variables that did not emerge as contributing much unique variance.
Perhaps we are all too kind to each other. We do not design studies that could demonstrate the superiority of one CBT model over another. Science involves the generation of theories and the testing of theories and pruning back some of them. Presently, we generate good theories and good treatments. But we are stuck with multiple theories because we do not test multiple models in research that would allow us to give up some models in favor of others. Bill Chaplin who teaches statistics and research design in our department tells students that one should never test just one model. Most studies in CBT test only one model.
The third scientific question posed by the Dodo bird is, "What theory can account for all of the positive results in CBT?" Most theories in CBT focus on the theoretical constructs and techniques of one construct. Such theories do not expand our knowledge because they just add one more competing model to an already crowded field. A good theory would account for all of the positive results of all the models. Although I too have my favorite theory, I have to acknowledge that even if REBT has some evidence that it works, so does Cognitive Therapy, ACT, Problem-Solving and many more. A good theory could explain how these different models and different interventions work. If a theory only explains why one model works and ignores the positive results of competing models, we have incomplete theory.
CBT has come a long way, but we are stuck, and the proliferation of more models does not advance our field. Perhaps it is time to prune the branches of our tree. Our science will mature when we compare the existing theoretical models and interventions to see if they are equal or whether one model provides a better explanation of human suffering and leads to better outcomes.
If one model comes out better, we all benefit from the knowledge and we can all still be friends.
Publish Without Perishing: Advice for Students and New Faculty
Jon Abramowitz, Ph.D.
UNC Chapel Hill
Many ABCT members, including students and young professionals, are actively involved in research and in publishing their findings. I'm no newbie to this aspect of academia, but I still find it one of the most fulfilling experiences of my work. Of course, trying to publish can also be frustrating. Over a decade ago, some colleagues and I wrote two articles in the Behavior Therapist to help students and new faculty successfully navigate the publishing process (Taylor, Abramowitz, McKay, Stewart, & Asmundson, 2006; Taylor, McKay, Abramowitz, Asmundson, & Stewart, 2006). In this blog, I want to revisit and update what we wrote in 2006 based on my experiences as an author, editor, reviewer, mentor, and psychology department administrator. To be sure, there's no "best" formula for success in publishing, but perhaps these ideas will help-or at the very least, stimulate discussion among students and colleagues.
Allocating Your Time
Understanding the Local Environment
If you're in academia, you're probably involved in teaching, committees and other service, grant writing, and of course, conducting research. The degree to which these activities are weighted when it comes to decisions about tenure and promotion vary from department to department, across institutions, and perhaps with the preferences (or biases) of faculty committees responsible for hiring and promotion. This means it's important to understand the local conditions in which you work (or are planning to work). At some smaller colleges, for example, emphasis is placed on teaching and service rather than on publishing, and a faculty member may not be granted tenure on the basis of poor teaching evaluations from students. To avoid such problems, communicate with colleagues (or a department chair) at your institution and consult any faculty handbooks or department guidelines.
Grant-Writing vs. Publishing
Most research universities emphasize grant funding and the publishing of empirical research. But writing and (hopefully) obtaining a grant can be a painstaking and time-consuming process. In many cases, grants are a means to an end; a way to secure the time and resources to conduct and publish research. So, on the one hand, perhaps it is worthwhile only writing grants that you need to do your research. Yet the reality is that when it comes to promotion or salary bonuses, having grants might be more important than the number or quality of journal articles you've published. Bringing in grant funding increases the prestige of your department and university. You might even be required to fund your own salary, either by providing direct clinical service or by obtaining grants. Thus, it might be more important, at least in many academic settings, to secure grants than to publish research articles. The Catch-22 here is that publishing (and demonstrating your record as a successful researcher) is also necessary for success in the grant game.
Publish and Perish
Local conditions also define ways that one can publish and perish. Some departments, for example, emphasize empirical research and minimize the importance of "non-empirical" papers, such as review articles. So, devoting time to writing theoretically important review papers might make you less likely to obtain tenure and promotion. I personally believe a bias against review articles is ironic and unfortunate since such papers tend to have a greater impact on the field than do most empirical papers.
If you are a recent graduate working toward establishing yourself as an independent researcher, there are pros and cons to publishing with your advisor (or for that matter, any senior researcher in your field). The pros include (a) you are likely to learn important things about research and publishing when you work with senior researchers, and (b) your articles may be more likely to be read and cited by others. The cons include (a) the risk that people will incorrectly assume that the ideas in your paper came from your advisor or big-name co-author (rather than from you), and (b) the risk that members of hiring or promotion committees will be concerned about your ability to work as an independent investigator. One way to make it clear that you are the lead investigator and originator of important research ideas is to publish papers with your students, rather than with your advisor. Develop collaborations with colleagues, rather than with senior leaders in the field. And when appropriate, include footnotes in your papers acknowledging that you are the principal investigator.
Taking a "fox" rather than "hedgehog" approach to research and publishing can also harm your chances of academic success, especially in the early stages of your career. According to the ancient Greek poet Archilochus, the fox knows many things, but the hedgehog knows one big thing. One might adopt a largely fox-like approach, whereby you do research in many different areas and publish on a diverse range of topics. In comparison, one might adopt a hedgehog approach, in which you study one topic in great depth and publish primarily on that topic. Other researchers adopt of blend of fox and hedgehog approaches. While neither approach is inherently wrong, I believe the hedgehog-like approach is more advantageous if you are a new academic because it shows that you are pursuing a solid program of focused research instead of seemingly haphazard work. As you become more established in your academic career you might choose to diversify and become more fox-like, such as by pursing multiple lines of research and publication.
Publishing as Public Relations
Publishing your work is, to a large degree, the business of effective communication. The easier it is to read and understand what you have written, the more people will be drawn to your work. Becoming known as someone who communicates their ideas clearly is some of the best public relations you can get as a researcher, and publishing is an excellent opportunity to create an impression of yourself as thoughtful and articulate. Careful writing will increase the chances of your being asked to contribute chapters to edited books, articles to special issues of journals, papers in conference symposia, and to serve on journal editorial boards.
As an editor, I have encountered authors who consider good writing as mere window dressing. They submit poorly written work hoping that the reviewers will tell them how to fix their papers. But let me be clear: expecting reviewers to clean up your work is irritating to editors (not to mention reviewers) and thus a recipe for rejection. It is also a misuse of the peer-review process. The first thing editors and reviewers notice when they receive a submitted article is how clearly it is written. Clear prose is also important when revising and resubmitting a journal article. Your cover letter to the editor, in which you describe how you have addressed the reviewers' concerns, should be clear, concise, and non-combative.
Selecting a Journal to Publish in
The impact factor is perhaps the most widely used numerical index of a journal's prestige-it represents the average number of citations the average article in that journal receives per annum. Journals that publish mainly review articles tend to have higher citation indices than those that publish mainly empirical studies, since review articles tend to be more widely cited than empirical papers. Students and new faculty can use the impact factor for assessing the status of journals to which they might contribute their work, rather than relying on subjective impressions about the status of a journal. Committees responsible for hiring, promotion, and grant reviews may also use such indices as part of their evaluation. Of course, such indices are not universally used or valued. This underscores the importance of local environment, as I described earlier.
It's also important to consider the fit between your paper and the journal, guided in part by the aims and scope of the journal (usually provided on the journal's homepage). If your manuscript describes a ground-breaking piece of work, for example, it may be suitable for a high-ranking journal. On the other hand, research that is methodologically sound, but simply replicates other research studies usually ends up in a lower-tiered journal.
Sometimes it is difficult to determine whether to submit to a higher- or lower-ranking journal. Submission to a high-ranking journal can lead to rejection, delaying your article's publication in a respectable, but lower-ranking journal. But then again, you've got nothing to lose (but time itself) by aiming reasonably high. If in doubt about whether to submit your work to a high or lesser ranking journal, you might consider your paper along with a few comparable articles from the journal in which you are planning to submit. Ask yourself how your paper compares to the published articles. You might also ask colleagues or mentors for their advice. I often encourage students and new faculty to "aim high, but don't shoot for the moon."
You should consider the sorts of articles that appear in the journal as guided by the information for contributors. Some journals (depending on the editors) prefer creative papers that contain novel ideas while others prefer methodologically rigorous papers containing a minimum of speculation. Some journals are known for being quicker to review and publish articles, whereas others are known for being slow with this process. Again, talking with colleagues who have more experience and know the journal landscape in your area is always a good idea. You may wish, for example, to submit to the efficient journals and avoid the tardy ones. After all, if you are applying for a job or for a promotion, a paper that is "in press" carries more weight than one that is simply "submitted."
Watch Out for Bogus Predatory Journals
Given the importance of publishing, it is not surprising that an industry of bogus "predator" journals has established itself in the "service" of academics looking to publish their work. This exploitative business model involves charging publication fees to authors without providing the editorial and publishing services associated with legitimate journals. They are often labeled as "open access" journals, meaning that once published, your article will be openly available (online) to the public without a subscription. Some of these journals have names eerily similar to those of legitimate journals (such as the Journal of Depression and Anxiety, which is a predatory open access publication that is named similarly to the legitimate publication Anxiety and Depression), and their websites are set up to look just like those of well-known authentic publications. If you have received emails inviting you to submit articles to open access journals, or to join editorial boards, beware that these publications could belong to this group. Of course, I recommend never submitting your work to predatory journals and would in fact be highly skeptical of any journal offering to publish your work if you pay them. If you have doubts about a journal's legitimacy, seek advice from a more experienced colleagues, or check out this list of potentially predatory journals: https://scholarlyoa.com/individual-journals/.
Journal Articles vs. Book Chapters
How important is it to your academic career to publish book chapters? The answer depends on a range of factors, including the environment of your institution and the stage of your career. Although they show that other scholars have thought sufficiently highly of your work to invite you to submit a chapter, book chapters often do not have to meet the higher standards (e.g., peer review) for scholarship that is required for journal articles. Also, book chapters, unlike empirical journal articles, may make less of a contribution of new knowledge to the field. So, if you had to decide between spending your time on a journal article or a book chapter, I suggest the former.
To draw this to a close, let me point to an anecdote about Hans Eysenck, who is probably one of the most widely cited psychologists in the world. Despite his voluminous publication record, including many papers in high-ranking journals, Eysenck would be delighted each time one of his articles had been accepted for publication, even if the paper had been accepted in a low-tier journal. Eysenck was driven by curiosity and was an expert at the game of publishing, and evidently enjoyed celebrating his many publishing victories. There are many different reasons for doing research and publishing scholarly work. A sense of curiosity, meaningfulness, and enjoyment at tackling the various intellectual questions that one encounters can lead to a stimulating, fulfilling publishing career. Although academia emphasizes products (publications), it's the process that really counts - the process of doing personally meaningful work - if you want to have a fulfilling, productive career in writing and publishing… which might explain why I devoted my time to writing this blog instead of doing other things!
Diagnosing Personality Disorders in Children and Adolescents
Art Freeman, Ed.D.
In the mid to late 1990's, Mark Reinecke and I were talking about child and adolescent cases we had recently seen. I had recently seen Susie, * a 27 year-old woman who had dropped out of school, had three children (all in State care) fathered by three different men, was HIV positive, and was referred by her attorney who was looking for a "psych" defense to explain Susie having attacked a police officer. The woman lived in the town in which she was raised so I had access to her school records, including two evaluations (ages seven and twelve) by the school psychologists. The evaluations discussed Susie and the her home and classroom behavior. Susie was disinhibited, would frequently get into fights with boys and girls, and was dysregulated in her responses to feedback from teachers. Her frequent response was to verbally assault the offending teacher or school administrator who commented on Susie's need to better control herself in class and in the school yard. It was also reported that Susie was "promiscuous" and was sexually active since age eleven. She would perform fellatio on lines of boys during the lunch hour. She was also reported to be smoking, consuming alcohol, and making use of marijuana. These were, in large part, many of the same behaviors that she exhibited when I saw her at age 27. Susie's diagnosis at this point was Borderline Personality Disorder, having met seven of the nine DSM criteria.
Interestingly, Susie's childhood and adolescent diagnoses included ADHD, Bipolar Disorder, Oppositional Defiant Disorder, and Conduct Disorder. Both Mark and I wondered why Susie wasn't diagnosed at that point with Borderline Personality, inasmuch as she met the criteria.
It was at this point that we decided to develop a proposal for a volume on personality disorders in children and adolescents. We sent the proposal to several major publishers, not as a CBT book, but as a general book on personality disorders. The editors sent the proposal out for review and critique. The reviewers were, it seems, horrified. The reviews all concluded that this was a dangerous and impossible diagnosis. It was pointed out that the diagnosis could not be applied to children and adolescents. Given the universally negative reviews, the proposal was consistently turned down. Both Mark and I had other projects to which we could devote our energies, so this was just another interesting but bad idea. In 2001, we received a letter from John Wiley & Sons asking if we were still interested in doing the book. We were still interested and quickly revived the proposal. What made the difference? In 2000, Kernberg, Weiner, & Bardenstein published a volume for Basic Books on personality disorders in children, and in 2001, Bleiberg published a similar book for Guilford. Now that the cat was out of the bag, it was okay to do a book with childhood and adolescent personality disorders as the major focus. Our book, "Personality Disorders in Children and Adolescents," was published in 2007 by Wiley. Since that time there have been several more volumes with that focus (e.g., Aguirre, Borderline Personality Disorder in Adolescents, 2007, 2014; Sharp & Tackett, Handbook of Borderline Personality Disorder in Children and Adolescents, 2014).
The major complaint made by reviewers was that children or adolescents could not be diagnosed with having a personality disorder until they were aged 18. Where did this idea emerge? The quick answer was that DSM 1II-R (1987) forbid it. Look as we did, we could not find any hindrance to applying the diagnosis to children and adolescents. In point of fact, DSM III-R (1987) stated, "Personality Disorder diagnoses be applied to children and adolescents in those relatively unusual instances in which the individual's particular maladaptive personality traits arrear to be pervasive, persistent and unlikely to be limited to a particular development stage or an episode of an Axis I disorder.' (p. 631). This description continues unchanged through to DSM IV, DSM IVTR, and DSM 5. Further, DSM states, "The features of a personality disorder usually become recognizable during adolescence or early adult life." (DSM-IVTR, page 688). To summarize, the symptoms must be: pervasive, persistent, not consistent with a normal developmental stage (e.g., a two year-old holding onto mommy's hand would not be considered a dependent personality; a 42 year-old holding mommy's hand for safety, would), clearly recognizable, not better explained by an Axis I diagnosis, and identifiable for at least one year. If we take a talmudic view, the criteria sets for DSM 5 (2013) all state, "……beginning by early adult hood." It does not say, "….beginning in early adulthood i.e., age 18." This would imply that the pattern begins before early adulthood, and not at age 18 as is commonly, though mistakenly, believed. This requirement of age 18 is stated only for Antisocial Personality Disorder.
The complaints regarding the use of the diagnosis include, 1) Personality is still forming and that to label it as "disordered" prior to age 18 gives the impression that the personality of the child is fully formed and encased in stone. 2) "Inappropriate" early labeling with such a powerful diagnosis may lead to care-givers, therapists, and teachers giving up on the child, which might lead to the diagnosis being a convenient "cop-out" for therapists who are unsuccessful with a child or family. 3) The diagnosis will be applied in greater proportion and inappropriately to culturally different groups. 4) "Personality" does not exist absent the specific behaviors, therefore there can be no "personality disorder." And 5) Being identified as having a personality disorder would be too weighty a diagnosis for a child to carry through life and would negatively influence the child throughout life.
Our response would be that: 1) These children need to be diagnosed very early so that they can receive the best, most intensive, and most appropriate care and that early detection and intervention may limit the chronicity. 2) Since most adults with personality disorders can identify childhood and adolescent manifestations of the disorder, clinicians can take a more preventative stance. 3) We can initiate therapy for the child, request/require parental involvement, request school or agency intervention, and develop opportunities for postvention over the years. And, 4) Early identification and intervention may halt or limit traumatic experiences.
If we review the expressed concerns, i.e., 1) Personality is still forming. At what age is it "formed?" 2) "Inappropriate" early labeling with such a powerful diagnosis. There are many diagnoses that are at least as negative (ASD, cognitive loss, brain damage). It might lead to the diagnosis being a convenient "cop-out" for therapists who are unsuccessful with a child or family. A therapist or caregiver who chooses to give up may find a multitude of reasons. 3) The diagnosis will be applied in greater proportion and inappropriately to culturally different groups. This I believe to be true. 4) "Personality" does not exist absent the specific behaviors, therefore there can be no "personality disorder." Here, again, we are talking about angels dancing on the head of a pin. And 5) Being identified as having a personality disorder would be too weighty a diagnosis for a child to carry through life and would negatively influence the child throughout life. All diagnoses need to be reviewed and are related to the establishment of new data sets. As a child I could not see clearly so I sat at the front of the room. I was fitted with glasses but was still seated at the front as a visually challenged child even though with glasses my vision was 20/20.
We know that the contributing factors that are typically part of the picture can be identified, including basic skills deficits, such as failure to teach frustration tolerance, inappropriate child rearing or child management, and frequent and severe boundary violations. There are also parental factors such as parental values that support the behavior, parental psychopathology. Situational factors such as significant and prolonged psychosocial stressors (family or child), emotional, physical, sexual, and/or verbal abuse, parental neglect and rejection, and biological factors, such as child temperament.
For example, the compliant, dependent, obsessive-compulsive, hard-working, success-driven, demanding child may succeed at school and be seen by teachers as a hard worker and may be a superior student as measured by academic success. The results of the personality style in this instance may serve to enhance a child's school performance. The child may be rewarded by certificates, plaques, public recognition, or admiration of other students. The child may receive the approval of teachers, school administrators, and parents. After all, who wants their child to be an academic failure? The teacher earns acknowledgement for having a "good student" who "does well" and "brings credit to the school." Parents may get a bumper sticker. They would rarely be diagnosed as a disorder. It is when they are finished with formal education that the problems emerged in that they keep searching for the previously acquired markers of success, grades.
Or, the avoidant child may be overlooked by teachers and clinicians inasmuch as they are reserved in response, do not generally call attention to themselves, and generally cause little trouble in class. The fact that they play by themselves and stay on the outskirts of work and play groups will usually not bring them into conflict with schools as long as they do relatively well in school. Inasmuch as they may relate well to family and first degree relatives, the families may not see any problems other than the child is noted to be "kind of shy," and shyness and diffidence are not seen as negative in many cultures.
To avoid labeling and stigmatizing a child, I would recommend more tolerable descriptors with which most parents and school personnel could accept. The isolated and withdrawn child (Schizoid spectrum); the very careful and even suspicious child (Paranoid); the highly/easily irritable or extremely sensitive child (Borderline); the self-focused child (Narcissistic); the aggressive child (Antisocial); the attention-seeking child (Histrionic); the conscientious and careful child (Obsessive Compulsive); yhe needy child (Dependent); the shy child (Avoidant); the stubborn though apparently agreeable child (Passive aggressive); the sad child (Depressive).
For professionals who are reluctant to use the personality disorder diagnosis for children, we can work to de-stigmatize many of these terms. Whether the term or construct is used is another issue. Using less useful labels does not help garner the help that these children often need. As I teach my graduate students, "If it looks like a duck, walks like a duck, and quacks like a duck, it is most likely a duck."
* not her real name
Cognitive Behavior Therapy for Jealousy
Many couples struggle through anger, anxiety, depression, and hopelessness related to the issue of jealousy.
Indeed, jealousy is a major cause of domestic violence-especially domestic homicide-and often can continue for years, with people prone to jealousy experiencing a series of relationship losses that only confirm a sense of insecurity.
In my recent book, Emotional Schema Therapy, I describe some of the elements of a CBT approach that clinicians might use. In this posting, I will describe a brief outline of how to help patients and couples cope with jealousy.
- 1. Understanding Jealousy
I view jealousy as a universal emotion that has evolved based on attempts to protect one's genetic investment. According to Trivers' "parental investment theory" individuals will attempt to protect their genetic "investment" by providing resources and protection to their offspring. Accordingly, males have greater uncertainty about the paternity of the offspring and will drive off competitors who seek sexual access. Females, always secure about the genetic investment, will seek out emotional support, resources and protection from males. This sex difference is reflected in the fact that males are more likely to express sexual jealousy while females express emotional jealousy. The clinician can help patients understand that jealousy is one of many emotions that one has, that one is responding to issues that were adaptive in the evolution of our species (and other species), and that jealousy is something that can be treated effectively.
- 2. Angry agitated worry
Jealousy is characterized by threat monitoring, worry about future betrayal, and anger over humiliation and potential loss. Moreover, jealousy also involves rumination where the jealous partner dwells on past "evidence" attempting to interpret "clues" in the framework of betrayal schemes. There is a "confirmation bias" that seeks out evidence of betrayal. This includes selective attention to the partner's mood, appearance, interpersonal behavior, and unexplained absence as well as "clues" of the interest of competitors. Jealous individuals are often searching for evidence, but discounting countervailing evidence. The therapist can use worry time ("jealousy time") where jealous thoughts are delayed to another time during the day. The patient can evaluate if dwelling on these worries will lead to a productive outcome. Thought flooding (repeating the jealous thought) hundreds of times can lead to habituation. Patients seem to respond to this latter technique which I refer to as "the boredom technique". Learning how to accept uncertainty and lack of control while focusing on valued goals within and outside the relationship can assist patients in breaking loose from the hold of jealous thoughts and feelings.
- 3. Safety behaviorsAs with most anxiety disorders, jealous individuals use problematic behaviors that they believe will provide safety or warning of danger. These include the search for clues mentioned above, surveilling the partner, interrogating, reassurance seeking, and attempts to control the partner's free movements. The jealous individual fears giving up these safety behaviors because they believe that they risk getting caught off-guard. Using these behaviors only reinforces the belief that something must be controlled or surveilled. Ironically, these behaviors lead to greater conflict which undermines the very security that one is seeking. The clinician can assist the patient in gradually giving up these safety behaviors to evaluate the belief "everything will fall apart".
- 4. Cognitive Biases
Jealous individuals often utilize a wide range of cognitive biases. For example, these include the following: mind reading ("She finds him attractive" or "He is interested in having sex with her"), fortune-telling ("He's going to betray me" or "She will leave me for another man"), labeling ("He's a liar" or "She has no conscience"), overgeneralizing ("He's never reliable" or "She's always flirting"), catastrophic thinking ("The worst thing that could happen is if he betrayed me" or "I couldn't stand living alone"), discounting the positives ("Just because she hasn't cheated on me doesn't mean she won't" or "I know he loves me, but you can't base everything on that"), personalizing ("She finds him interesting which means that she is losing interest in me" or "He seems less interested in sex this week so something must be going on"), and negative filter ("Look at how he seems to find her attractive" or "She wasn't as affectionate this week"). The therapist can use a wide range of cognitive therapy techniques: examine the costs and benefits, vertical descent, evidence for and against, alternative interpretations of the partner's behavior, and role-playing against the negative thoughts.
- 5. Perceptions of emotions
Jealous individuals often do not experience much validation for their feelings, since their emotions are expressed as accusations or inquiries that lead the partner to become defensive. Often the jealous individual will fail to recognize the range of positive and neutral feelings that they have, they may believe that unless they "do something" their jealous feelings (anxiety and anger) will escalate, and that they cannot control their feelings. The therapist can assist patients in accepting that everyone is vulnerable to jealousy, that jealousy is often linked to the values of commitment and honesty, and that one can have a feeling of jealousy but not take action against the partner. The therapist can help the patient in challenging thought-action fusion by decentering from a thought or emotion and observing that it can exist and not lead to loss. Mindful detachment, acceptance, allowing the emotion to exist as a "compartment" in the relationship can replace the attempts to either suppress the emotion or "get the answers". One can accept having a feeling without undermining the relationship.
Jealousy is a relationship issue and clinicians need to be aware that both partners can contribute to the problem and to the solution.
Evidence-Based Therapies and Evidence-Based Practice: Addressing the Denizens of Antiscience Island
Dean McKay, PhD, ABPP
Professor, Fordham University
Island biodiversity is a curious specialty area of study for biologists. It serves as a unique model for understanding evolution. Studied naturalistically, we see examples of how organisms evolve to meet the highly specific and idiosyncratic demands of islands, where some adaptive features would not function well on other landmasses or islands. There are illustrations of evolution proceeding in ways that confer survival value for physical features that would lead to immediate extinction in virtually any other setting. The tortoises of the Galapagos would likely not survive in any other setting given their size and specific dietary demands. Many bird species on Guam have become extinct with the introduction of invasive snake species. These birds, which nest on the ground, never had predators like snakes to deal with, and so their unique evolution was no match for the snakes that arrived as stowaways on ships from North America and Europe (Quammen, 1998).
Adherents to different theories of psychotherapy could be said to live on their own islands, where their approaches evolve in idiosyncratic ways, and reach a peak of efficacy through a recursive feedback loop, unsullied by outside sources. This is all tossed out the window when some kind of invasive-species organisms lands on the shores of the island of a different theory. Take for example how CBT has taken hold in such exotic locales as New York City, long home to psychoanalytic models of treatment1. As consumers realized that therapy might include short-term symptom relief, suddenly demand for CBT increased, and dramatically. Like an invasive species, CBT started spreading throughout the greater NY area, where there is a long-standing culture that embraces psychotherapy. The existing therapy community had been comparably ill equipped for this new resident, and a rush was on to rapidly evolve to survive in the presence of this new resident.
How exactly has that been working out? Well, many more clinicians report that they practice CBT, or report employing CBT-techniques. This has not necessarily led to better delivery of care, and may even represent a threat to the survival of CBT through a loss of fidelity of treatment, and in turn a loss of confidence by consumers in our approach (see McKay, 2014, for a detailed discussion of this issue).
1 This is not to ignore the fact that some well-known CBT practitioners were based in NYC for years, such as Al Ellis or Leo Salter.
Who are the Denizens of Antiscience Island?
I was recently asked to write a comment paper in response to a critique of evidence-based therapies (Shean, in press). The arguments put forth in Shean's piece represent a kind of antiscience attitude that still exists (surprisingly) among a subset of practitioners. Specifically, the claim is that evidence-based therapies constrain a practitioner from exerting clinical judgment and that the application of rigorous tests of psychotherapy efficacy (through randomized controlled trials) results in a reification of protocols deemed empirically supported by dint of their surviving these specific tests, without regard for the real-world applications and whether these protocols work in the private office or clinic. These residents of Antiscience Island do not offer constructive recommendations that address their concerns, only claims that the evidence-based therapy adherents are destroying the art of psychotherapy by robbing therapists of the ability to creatively apply their craft when there may be limited science to guide their clinical actions. They fall back on the inherently idiographic nature of clinical practice (ignoring that other service delivery fields contend with this as well and yet are science-based; see, for example, virtually all of medicine). I won't restate here the points I am raising (co-authored, Brian Pilecki) in the comments to Shean, except to note that there are some misconceptions inherent in the arguments against evidence-based therapy that is commonly stated by the antiscience crowd.
These Antiscience Island residents, in an attempt to survive in the face of an existential threat to which they are ill-equipped to defend against, offer a litany of claims about the efficacy of existing approaches. They cry out "Common Factors!" (Wampold, 2007), "Interpersonal Relationship!" (Shedler, 2010) or "Therapist Allegiance!" (Shean, in press) as a justification for a status quo that most consumers simply do not want.
Evidence-based Therapy and Practice Misunderstood
The denizens of Antiscience Island may feel neglected by the emerging efforts to promote evidence-based therapy. Indeed, as defined by a taskforce commissioned by the American Psychological Association, evidence-based practice is intended to be a combination of best practices in assessment, case conceptualization, and treatment components derived from existing empirically-supported treatments (APA Presidential Task Force on Evidence-Based Practice, 2006). This leaves ample room for clinical judgment and creative, flexible interventions developed for specific clients with myriad conditions. However, for the denizens of Antiscience Island, this would mean change in how they practice, something that has not been fostered in the evolution of this island's residents.
Should we worry about the denizens of Antiscience Island? I should think not. Evolving practice standards would necessarily demand that they adapt or go extinct. There are still places where their form of practice lives on quite well, thank you. Buenos Aires, Argentina has a thriving psychotherapy community, where most residents are in treatment, typically psychoanalysis (Landau, 2013). Across the river in Montevideo, Uruguay, the training model emphasizes primarily seeing clients and receiving supervision in psychodynamic methods, and with a general rejection of empirically-supported interventions2. So there are geographic locales that would be receptive to residents of Antiscience Island. There are also options for residents of Antiscience Island to ply their craft without any relocation. For example, with the advent of high speed web camera communication, some practitioners of non-scientifically informed procedures have begun delivering treatment across continents and the international dateline, such as New York area psychoanalysts delivering treatment to people in China (Osnos, 2011).
2 I learned of this training model during my time as a visiting scholar to Universidad Catolica de Montevideo, in 2007, where a faction of the faculty were seeking to integrate empirically-based methods into their training curriculum, to the chagrin of their own Antiscience Island residents.
Evolving Practices and Leaving Behind the Denizens of Antiscience Island
The denizens of Antiscience Island will continue to find ways to survive and, possibly, thrive. There will continue to be a place for them in the market. It is unfortunate, however, that there remains a rejection of science when there have been calls for science on some of their own floodplains. For example, Bornstein (2005) famously offended many of his own colleagues in the psychoanalytic world by calling for nomothetic research approaches and for a measure of accountability through more effective communication of the core concepts inherent in the approach. And while the Antiscience residents seek safe haven in places where they can practice without consumers calling for interventions that are evidence-based, CBT continues to operationalize and empirically evaluate concepts that were previously not in their domain. For example, Chapman and Rosenthal (2016) have described, using scientifically-derived principles of behavior change, methods for addressing client behaviors intended to undermine the therapy process. As another example, therapy trials for CBT have specifically examine therapist alliance (cf. Accurso, et al., 2015; Raue, Goldfried, & Barkham, 1997).
Be aware that I am referring to residents of Antiscience Island as a particular breed of clinician. While it may appear that a specific theoretical approach has been singled out, this is not my intent. Antiscience Island is a place we might occupy in our minds, and anyone might devolve to become a denizen of the place. Antiscience Island should be avoided, for it can in some instances have a seductive appeal. CBT has a rich tradition of science informing practice, and so it is imperative that we remain firmly planted on a mainland to avoid the evolution traps that come with inhabiting an island. Our survival depends on it.
Accurso, E.F., Fitzsimmons-Craft, E.E., Ciao, A., Cao, L., Crosby, R.D., Smith, T.L., Klein, M.H., … , & Peterson, C.B. (2015). Therapeutic alliance in a randomized clinical trial for bulimia nervosa. Journal of Consulting and Clinical Psychology, 83, 637-642.
APA Presidential Task Force on Evidence-Based Practice. (2006). Evidence-based practice in psychology. American Psychologist, 61, 271-285.
Bornstein, R.F. (2005). Reconnecting psychoanalysis to mainstream psychology. Psychoanalytic Psychology, 22, 323-340.
Chapman, A.L., & Rosenthal, M.Z. (2016). Managing therapy-interfering behavior: Strategies from Dialectical Behavior Therapy. Washington, DC: American Psychological Association.
Landau, E. (2013). In therapy? In Argentina, it's the norm. http://www.cnn.com/2013/04/28/health/argentina-psychology-therapists/ (retrieved Nov 1, 2015).
McKay, D. (2014). "So you say you are an expert": False CBT identity harms our hard earned gains (President's Column). the Behavior Therapist, 37, 213-216.
Osnos, E. (2011). China on the couch. The New Yorker (http://www.newyorker.com/news/evan-osnos/china-on-the-couch) (retrieved Nov 1, 2015)
Quammen, D. (1998). Flight of the iguana: A sidelong view of science and nature. New York: Scribner.
Raue, P.J., Goldfried, M.R., & Barkham, M. (1997). The therapeutic alliance in psychodynamic-interpersonal and cognitive-behavioral therapy. Journal of Consulting and Clinical Psychology, 65, 582-587.
Shean, G. (in press). Psychotherapy outcome research: Issues and questions. Psychoanalytic Psychiatry.
Shedler, J. (2010). The efficacy of psychodynamic psychotherapy. American Psychologist, 65, 98-109.
Wampold, B.E. (2007). Psychotherapy: The humanist (and effective) treatment. American Psychologist, 62, 857-873.
Graduate Education in Clinical Psychology
Gerald C. Davison
Professor of Psychology
University of Southern California
"Education is the kindling of a flame, not the filling of a vessel." - Socrates
"There are these two young fish swimming along, and they happen to meet an older fish swimming the other way, who nods at them and says, 'Morning, boys, how's the water?' And the two young fish swim on for a bit, and then eventually one of them looks over at the other and goes, 'What the hell is water?'…. The immediate point of the fish story is that the most obvious, ubiquitous, important realities are often the ones that are the hardest to see and talk about." David Foster Wallace, Kenyon College Commencement Speech, May 21, 2005
"A fish doesn't know he's swimming in the water." Gerald C. Davison, Videotaped interview, ABCT Archives Project, November 15, 2001
Since I began university life in 1966 at Stony Brook, I've been fairly continuously involved in teaching at all levels - undergraduate courses in abnormal psychology and introduction to clinical psychology; graduate courses in psychosocial intervention, cognitive behavior therapy, and clinical interviewing; and clinical supervision of both Ph.D. students and postdocs. I've also spent many hours addressing education and training issues in various administrative capacities - as a director of clinical training, as a department chair, as an academic dean, and as a member of various committees at USC and in APA and ABCT. Throughout these many activities, I've had an opportunity to reflect on graduate education in clinical psychology, as complicated and daunting an educational enterprise as there is. I offer for your consideration some of the issues that have occupied my mind. Some of what follows is based on a piece I wrote when I was president of Division 12 of the American Psychological Association (Davison, 2006).
Education, not Training
I wish we would do away with the title "director of clinical training" and other similar designations that emphasize training rather than education. Training refers to teaching people specific skills, like how to score a Rorschach, time-sample behavior, assess intelligence with the WAIS, conduct a structured interview like the SCID, phrase empathic statements to patients, draw up an anxiety-hierarchy, challenge a biased way of processing information, encourage a patient to make alterations in maladaptive schemas, and ad infinitum. It is obviously important for our students - and for independent practitioners and teachers - to acquire these skills but....
I believe that doctoral programs in clinical psychology (including the more hard-nosed ones that call themselves clinical science), whether they award the Ph.D. or Psy.D. degree, should concentrate much more on education. By this term (which comes from the Latin, educere, to lead forth, draw out) I mean the encouragement of critical and independent thinking in the process of mastering and generating relevant bodies of knowledge. I routinely tell the first year clinical students at USC in my courses on psychological intervention and on clinical interviewing that one my teaching goals is to get them to worry. I don't mean that they should become diagnosable as suffering from GAD or otherwise be dysfunctional! I do mean that I want them to reflect carefully and critically on what they read, hear, and say; learn how to play today's science game well and to see its relevance as well as its limitations for responsible application; and approach widely touted innovations in assessment and intervention with a Missourian "show me" attitude. Not with a nihilistic cynicism but with a modesty and skepticism that befit a true scholar.
The Liberal Arts
These considerations have their roots in the liberal arts and the college experiences most of our graduate students have had. The liberal arts provide breadth of knowledge and perspectives, encouraging the student to reflect on and appreciate the complexity of human experience, on the role that the humanities (especially philosophy), the social sciences, and the natural sciences play in our efforts to understand our world. Most importantly, "Liberal education teaches the importance of tempering profound convictions with a measure of tolerance and a judicious sense of humility" (Freedman, 2003, p. 58). Such study, which I believe is of overriding importance in the education of clinical psychologists, requires students to engage in areas of inquiry they might not have elected voluntarily. It encourages students to become generally well-educated people who can think across a spectrum of disciplines, who question what is known, how it was learned, and why it might change over time.
These acquired abilities, these habits of careful, analytical critical thinking, are the keys to all kinds of intellectual and professional success, particularly at the higher levels of professional achievement, where judgment and imagination, analysis and synthetic thinking are prized over specific training in the customary ways of doing business. These habits of thought and temperament can also, I would suggest, facilitate creativity and wisdom because of the achievement of a broad historical and philosophical context for applied activities. For example, a civil engineer involved in building a bridge would do well to appreciate the history of the region and the various cultures and beliefs of the population, in addition to understanding soil mechanics and physical and engineering principles.
Doctoral Study and Breadth
In an article I published 10 years ago, I discussed the way our graduate students are educated and I offered some opinions on problems I see in how we select and educate them:
.... a liberal arts education provides undergraduate psychology majors - who account for the vast majority of applicants to our doctoral programs - with a suitably broad historical, social, and philosophical context for their specialty study of psychology. But ... when students apply to graduate psychology programs, the primary focus of admissions committees is, I believe, on statistics, research methods, psychology content courses, and especially involvement in psychological research to the virtual exclusion of non-psychology work and intellectual interests that can provide ... [a] broad context [for understanding the human condition] ....
Once they enter a doctoral program in clinical or counseling psychology, the de-emphasis on topics not tightly linked to psychology becomes even stronger. When Ph.D. programs required comprehensive examinations, including history and systems, there was some assurance that students would gain a modicum of exposure to the larger historical, social, and epistemological context of the study of the human condition. But [I believe that] students are not being encouraged or required to appreciate the macro factors that influence their subject matter (Davison, 2005, p. 1062).
I have long believed in the importance of a solid liberal arts education as the foundation for all fields of graduate and postgraduate specialization. Whether it makes the more hard-nosed amongst us uncomfortable or not, both researchers and clinicians - to the extent that there are sharp differences between them - have to be Menschenkenner, people who know and understand people, including themselves. I believe that a broad education -- in addition, no doubt, to some inborn abilities of empathy and interpersonal sensitivity - is essential to developing the ability to figure out the vagaries of human conduct and how most effectively to devise ethically proper and rigorously evaluated methods of change.
My concerns extend well beyond clinical psychology education and training. Increasing and earlier specialization in such domains as cognitive (neuro)science and behavioral neuroscience imposes more and more demands on students and makes it difficult for them to avail themselves of graduate-level coursework in areas such as social, clinical, and developmental psychology. Indeed, many of my nonclinical colleagues question the very relevance of study in these areas to the research and scholarly goals of their students. I disagree with this position. It is ironic that most of today's leading senior psychological neuroscientists are themselves products of doctoral programs that emphasized broad and general education in psychology as a useful context for their specializations. Who can say that the graduate level study of psychopathology, for example, could not play a heuristic role in the research of a nonclinical graduate student? The same can be said for coursework in areas such as social and developmental psychology. One cannot know from where good ideas may develop. Of course the importance attributed by clinical psychology to what is sometimes called "core psychology" has been integral to sound doctoral education and has been for many years central to APA standards for accreditation, but it has been my experience as a faculty member for half a century (gulp) that the importance of a broad philosophical and theoretical context is vitiated in doctoral programs especially as they become increasingly reductionistic.
Where Do Our Theories Come From?
The generation of a theory is perhaps the most challenging part of the scientific enterprise-and one of the least understood. It is sometimes asserted, for example, that a scientist formulates a theory simply by considering data that have been previously collected and then deciding, in a rather straightforward fashion, that a given way of thinking about the data is the most economical and useful.
Although some theory-building follows this course, not all does. Bold directions, paradigmatic shifts, if you will, go beyond what extant data and theorizing tell us. Aspects too seldom mentioned in science and application are creativity and the excitement of finding a novel way to conceptualize things. A theory sometimes seems to leap from the scientist's head in a wonderful moment of insight. New ideas suddenly occur, and connections previously overlooked are suddenly grasped. What formerly seemed obscure or meaningless makes a new kind of sense within the framework of the new theory or paradigm. How this happens has intrigued students of science and the arts for many hundreds of years. I believe that such fertility of imagination is not nurtured by a narrow study of a specialized domain of inquiry and application.
As Kurt Lewin said many years ago, there is nothing more practical than a good theory. And in the epistemology that most of us adopt, one characteristic of a good theory is that it is testable, and one characteristic of a good scientist is that he or she does not become so invested in a given theory or paradigm that the search for disconfirming information ceases and that the person spends their time either not thinking about the utility of their theoretical models or actively avoiding opportunities to find shortcomings in their conceptual frameworks and core assumptions. We should not fall in love with our theories and for me that goes back to my initial suggestion that we should not construe doctoral education in clinical psychology as primarily a training enterprise.
Culture and Values
As proposed in a 2005 report from the American Psychological Association on empirically based practices in psychology (www.apa.org/practice/ebpreport.pdf), an awareness of and sensitivity to cultural and ethnic differences are of critical importance in developing a comprehensive understanding of the person and how most effectively and ethically to intervene. Developmental factors also need to be borne in mind, such as the different ways patients at different stages of life construe and express their difficulties. I would add that the clinician needs to have an understanding of the very kinds of problems people have that vary significantly across the lifespan. Macro variables also must be incorporated into the clinician's understanding of the patient. For example, what are the economic conditions that might be influencing the patient in negative ways or that might provide a support for therapeutic change?
The methods of gathering assessment data on our patients and research subjects and the principles of change that we seek after/create in our efforts to improve the human condition are embedded in a larger context than the functionalist rules we work with as scientists. Said differently, the topography of our subject matter cannot be overlooked in our quest for knowledge.
And this brings me to the position I took in my ABCT (then AABT) presidential address in 1974. I believe that brief mention of it is relevant to the theme of this blog.
Not Can But Ought
As may be familiar to some readers, the essence of the argument that I put forward in my 1974 AABT presidential address was that the values and biases of therapists inevitably influence the way they construe problems and which goals they work towards; that goals are determined much more by the therapist than by the patient; that therapists never make decisions about goals outside of a political and moral context; and that changeof-orientation programs for gays should be stopped, even when patients request them, because prejudice and often physical attacks against homosexuals have made it highly unlikely that "voluntary" change requests are in fact self-determined. Several years later, I offered the following fantasy to try to encapsulate the situation of gays in therapy as of the 1970s (that this argument may seem belabored and unnecessary in 2015 speaks to how much things have changed in many segments of North American society):
"API (Apocryphal Press International).The governor recently signed into law a bill prohibiting discrimination in housing and job opportunities on the basis of membership in a Protestant Church. This new law is the result of efforts by militant Protestants, who have lobbied extensively during the past ten years for relief from institutionalized discrimination. In an unusual statement accompanying the signing of the bill, the governor expressed the hope that this legislation would contribute to greater social acceptance of Protestantism as a legitimate, albeit unconventional, religion.
"At the same time, the governor authorized funding in the amount of twenty million dollars for the upcoming fiscal year to be used to set up within existing mental health centers special units devoted to research into the causes of people's adoption of Protestantism as their religion and into the most humane and effective procedures for helping Protestants convert to Catholicism or Judaism. The governor was quick to point out, however, that these efforts, and the therapy services that will derive from and accompany them, are not to be imposed on Protestants, rather are only to be made available to those who express the voluntary wish to change. 'We only want to help,' he said (Davison, 2001)."
Becoming a clinical psychologist or other mental health professional makes great intellectual and emotional demands. As theory and research in cognitive neuroscience strongly suggest (cf. Damascio, 1994), there is always a blend of these two categories - "the mind" and "the heart", to use an old-fashioned distinction - when humans interact with the complexities of their physical and social world. This makes what we do and what we teach enormously challenging and rewarding. Not for the faint of heart and mind but exciting and, in our commitment to improve the human condition, even noble.
Damascio, A. (1994). Descartes' error: Emotion, reason, and the human brain. New York: Putnam Publishing.
Davison, G. C. (2005). Some critical observations on 21st century graduate education in clinical psychology. Journal of Clinical Psychology, 61, 1061-1066.
Davison, G.C. (2006). Graduate education in clinical psychology and the Empirically Based Practices in Psychology report. The Clinical Psychologist, 59 (Summer), 1-5.
Freedman, J.O. (2003). Liberal education and the public interest. Iowa City: University of Iowa Press.