Psychotherapy Research

By Alan Carr

[Alan Carr’s full research paper “The Effectiveness of Psychotherapy. A Review of Research prepared for the Irish Council for Psychotherapy” is available for viewing here]

ABSTRACT

In this paper research methods for studying the process and outcome of psychotherapy are described along with a summary of the major substantive findings of current psychotherapy research. Such research has indicated that about 2/3 of cases benefit from therapy. The provision of social support, the facilitation of belief system exploration and consulting to behavioural change are three factors common to many forms of psychotherapy that contribute to a positive outcome. These factors arc optimally effective at different stages of the change process, with support being important during the early transition from the precontemplation to the contemplation stage; a focus of belief system exploration being central to the transition from contemplation to planning; and consultation to behavioural change being critical during the action and maintenance stages of the change process. Specific therapeutic approaches that research has shown to be uniquely effective with various types of problems are also described with particular reference to depression, anxiety, alcohol and drug abuse, schizophrenia, marital problems and child-focused difficulties. Research findings on the contribution of client and therapist characteristics to therapy outcome are summarized. Work on the impact of therapy duration on outcome is also considered along with factors that contribute to deterioration. Finally, knowledge claims that may be made about the findings of psychotherapy research are considered.

THREE TYPES OF QUESTIONS

Questions about research on psychotherapy may be divided into three categories:

– Methodological questions like What methods may be used to find reliable answers to questions about if and how psychotherapy works?
– Substantive questions like Does psychotherapy work and if so how does it work?
– Philosophical questions like What knowledge claims may be made for the results of psychotherapy research?

In this paper questions in each of these categories will be posed and some answers offered. The answers to these questions are of interest to a variety of stakeholders in the wide; system of psychotherapy service development and practice. These include clients, therapists, psychotherapy trainers and super­visors, psychotherapy researchers and those involved with the development of health and social services in the public and private sectors.

METHODOLOGICAL QUESTIONS

A broad distinction may be made between psychotherapy outcome studies which aim to determine if therapy works and process studies where the central question is the way in which psychotherapy works (Kazdin, 1994; Beutler & Crago, 1991; Barker, Pistrang, & Elliott, 1995).

Outcome studies

Ina typical psychotherapy outcome study, clients with similar problems are randomly assigned to two groups. Clients in the first group receive treatment and those in the second group do not. Before and after the first group receive treatment clients’ problems are assessed using reliable and valid measures of client problems. Six months or a year following the end of treatment clients complete a follow-up assessment. If the treatment group show a greater reduction in problems over the course of therapy in comparison with the control group, it may be concluded that the gains were due to treatment rather than to the passage of time, and if these gains are sustained at six months or a year follow-up, it may be concluded that the therapy led to lasting benefits. The change in average levels of symptoms before and after treatment may be compared, and the statistical significance of this improvement determined. However, often such statistically significant improvement is of little clinical significance. Many researchers therefore also report the number of cases that showed clinically significant improvement. This basic design is the cornerstone of all psychotherapy outcome research and all other designs are refinements or modifications of it. A diagram of this design is presented in Figure 1 (below).

Refinements that increase the confidence with which conclusions may be drawn from such studies deserve mention. First, the homogeneity of the client groups may be enhanced by using explicit inclusion and exclusion criteria and widely accepted diagnostic systems such as the DSM-IV (APA, 1994). For example, in most outcome studies of cognitive behaviour therapy, cases with brain damage are excluded and all included cases meet the DSM IV diagnostic criteria for major depression. Second, specific measures of clients’ problems such as clearly defined individualised therapy goals in addition to general measures of client functioning such as the SCL-90 (Derogatis, 1983) are used to permit specific and general improvement to be assessed. Third, to prevent therapists’ expectations and biases from influencing the results of the assessments, the evaluation of clients’ problems and symptoms may be carried out by someone other than the therapist who is unaware of whether or not the client is in the treatment or control group. Fourth, to insure treatment integrity and purity, the precise nature of the therapy offered may be specified in a therapy manual and audiotapes of therapy checked by an independent researcher to determine the degree to which the therapist offered the type of therapy being tested in the study. For example, if a therapist is participating in a study of family therapy, a manual such as Positive Practice: A Step-by-Step Guide to Family Therapy (Carr, 1995) may be used and audiotapes of therapists’ sessions checked against a list of therapist and client activities such as maintaining a focus on interactional aspects of the problem, permitting all family members to contribute to the session and so forth.

The basic two group outcome design may be modified to answer questions other than whether cases receiving therapy improve more than those who do not. One important question is the degree to which improvements shown by a therapy group are due to some specific technical aspect of the therapy, over and above visiting a therapist and talking to him or her. To answer this question, an attention-placebo control group may be used. Here, clients attend a therapist who allows them to an opportunity to discuss their problems or other issues but does not actively engage them in therapy. Where clients in therapy groups have better outcomes than clients in attention placebo groups, it may be concluded, that some specific technical aspect of the therapy, over and above visiting a therapist and talking to him or her, led to improvement. Where the central question is which of two or more therapies are more effective for a particular problem, cases may be randomly assigned to two or more treatment groups and their outcomes compared. In other instances, the question may be with which of two or more problems is a particular treatment effective. Here, groups of cases with different problems may be offered the same therapy and their outcomes compared.

Process studies

While outcome studies are concerned with the overall effectiveness of therapy, process studies are concerned with identifying features of the client, the therapist and the interaction between clients and therapists that contribute to positive changes within therapy sessions, between therapy sessions and im­provement at the end of therapy (Kazdin, 1994; Beutler & Crago, 1991; Barker, Pistrang, & Elliott, 1995). In many of these studies, therapist characteristics (such as age, gender and so forth); client characteristics, and therapist-client interactions (such as provision of warmth, empathy and so forth); and struc­tural features of the sessions such as the duration of therapy are assessed and correlated with positive changes in the clients’ adjustment within sessions, after sessions or at the end of therapy. Traditional descriptive case study methods are typically used to generate hypotheses about important psycho­therapy processes which are subsequently tested in quantitative or qualitative process studies.

Meta-analysis

Thousands of psychotherapy process and outcome studies have been con­ducted. Meta-analysis is a quantitative method for synthesizing the results of these many studies (Cooper & Hedges, 1994). In its simplest form, meta-­analysis involves obtaining an average improvement rate from a large number of studies. Smith, Glass and Miller (1980) conducted the first major meta­-analysis of psychotherapy outcome studies and concluded that broadly speaking a person receiving psychotherapy was better off than 80% of those in con­trol groups who did not receive therapy. There have been many criticisms of meta-analysis, the most important of which is that only similar studies using similar client groups, similar treatments and similar measures should be combined in meta-analyses.

Methodology problems

Current psychotherapy studies are very sophisticated (Beutler & Crago, 1991). In most good studies, cases with specific diagnoses are randomly assigned to groups, independent assessors are used and manualized therapies are employed. However, in the US where most research is done, recruited rather than referred patients are treated often by inexperienced therapists in university rather than community based clinics in typical psychotherapy research studies (e.g. Weisz & Weiss, 1993). These factors may limit the generalizability of results to community settings. A second major problem is that the results of psychotherapy research are rarely translated into user-friendly terms and disseminated to therapists or those involved in service development.

SUBSTANTIVE QUESTIONS

Among the many important substantive questions, to which the psychotherapy research literature offers some answers are the following:

• Does psychotherapy work?
• What factors contribute to therapeutic change?
• What factors common to all forms of therapy are appropriate at various stages of the change process?
• What factors or techniques specific to particular types of therapy are appro­priate for particular problems?
• What impact do client or therapist characteristics have on therapy outcome?
• What impact does the duration of therapy have on therapy outcome?
• Can psychotherapy be harmful?

In subsequent sections tentative answers to these questions will be offered based on the psychotherapy research literature much of which has been reviewed and summarized in the following sources: Bergin & Garfield, 1994; Beutler & Crago, 1991; Pinsof & Wynne, 1995; Weisz & Weiss, 1993; Giles, 1993; Norcross & Goldfried, 1992.

Effectiveness of psychotherapy

Results of meta-analyses show that a person receiving psychotherapy shows greater improvement in presenting problems at the end of therapy than 80% of the cases in control groups who received no treatment and about 66% of adults presenting for outpatient psychotherapy benefit from the experience (whereas only about 34% of those in waiting list control groups show some im­provement) (Lambert & Bergin, 1994). In most of the hundreds of studies underpinning these conclusions the principal problems were anxiety, depress­ion or interpersonal difficulties and the principal treatment techniques were therapies arising from the cognitive-behavioural, psychodynamic, humanistic and family therapy traditions. There is a growing body of evidence which suggests that a majority of clients maintain treatment gains at follow-up. However, relapse rates are high for particular problems such as alcohol and drug abuse or recurrent episodic depression (Lambert & Bergin 1994).

These broad conclusions, while widely accepted by psychotherapy re­searchers of various theoretical orientations, continue to be disputed by a minority of scientists, notably Eysenck, who first argued that there was no difference between improvement due to psychotherapy and improvements due to spontaneous remission over 40 years ago (Eysenck, 1952). The interested reader may wish to consult Eysenck’s 1993 paper for an update on his position.

Common factors that contribute to therapeutic change

From Figure 2 (below), which is adapted from Lambert’s (1992) literature review, it may be seen that 40% of change during therapy may be attributed to factors outside treatment and the remainder to features of the therapy situation. While specific therapeutic techniques (such as graded exposure to feared situations) and clients’ expectations of improvement each account for about 15% of change, non-specific therapeutic factors are by far the most important determinants of therapeutic change and these account for about 30% of improvement during therapy. These non-specific factors fall into three broad clusters:

• provision of social support
• facilitating exploration of belief systems about the problem and its impact
• consulting to behavioural change

With respect to the provision of emotional support, it has been found that the quality of the therapeutic alliance, in particular accurate empathy, respect, warmth, genuineness and a collaborative approach are positively related to outcome (Orlinsky, Grawe & Parks, 1994). With respect to belief system ex­ploration, the following specific factors have been found to be associated with a positive outcome: clients’ openness; clients’ level of affective experiencing within therapy; clients’ insight experiences within therapy; clients’ exploration of life problems and significant relationships within therapy; therapists’ con­frontation of inconsistencies in clients’ accounts and experiences and thera­pists’ interpretation of clients’ accounts (Orlinsky, Grawe & Parks, 1994). Therapists’ encouraging exposure to feared intrapsychic material or external situations; therapist modelling of adaptive behaviour; and therapists giving corrective feedback on practice are the features of consulting to behavioural change that have been found to be associated with positive outcome across a range of therapeutic models (Orlinsky, Grawe & Parks, 1994; Lambert, 1992).

Common factors and the stages of therapy

Prochaska and DiClement (1992), from an analysis of 24 schools of therapy, identified five stages of therapeutic change. These were: pre-contemplation, contemplation, preparation, action and maintenance. They also found by surveying clients and therapists engaged in therapy, that specific techniques were maximally effective in helping clients make the transition from one stage of change to the next in the manner set out in Figure 3 (below). The list of a dozen techniques identified by Prochaska and DiClement fall into the categories of support, belief exploration and consulting to behavioural change which have been mentioned in the previous section. In the precontemplation stage, the provision of support by the therapist provides a climate within which clients may ventilate their feelings and express their views about their problem or life situation. Such support may help clients move from the pre­contemplation phase to the contemplation phase. By facilitating an exploration of belief systems about the evolution of the problem and its impact on the client’s life, in addition to providing support, the therapist may help the client move from the contemplation to the planning stage. In the transition from planning to action, the most helpful role for the therapist to adopt is that of consultant to the clients’ attempts at problem solving. In addition to providing support and an opportunity to explore belief systems, the therapist helps clients examine various action plans that may help disrupt destructive behav­ioural patterns. The therapist also facilitates the development of an emotional commitment to change. This role of consultant to the clients’ attempts at behavioural change is also an appropriate one for therapists to adopt in the transition from the action phase to the maintenance phase where the central concern of the client is relapse prevention. Prochaska and DiClement (1992) noted that different schools of therapy place particular emphasis on one or more of these common change processes, but most schools of therapy have something to say about each and in practice therapists tend to move through a progression from support to belief-system exploration to behavioural change consultation with their clients.

Specific factors

In Figure 2, it was noted that while 30% of client change may be attributed to common factors, 15% may be attributed to specific factors: that is, to particular therapeutic techniques. A central idea within the psychotherapy integrationist movement is that modules which research has shown to be effective with particular problems may be incorporated in most forms of psychotherapy practice (Norcross & Goldfried, 1992). What follows is a summary of some of the more important research on specific therapeutic approaches that have been shown to be effective with particular presenting problems.

For depression, cognitive therapy or interpersonal therapy coupled with antidepressant medication have been shown to be more effective than other treatments (Hollon & Beck, 1994). Cognitive therapy helps patients to alter their mood by providing them with skills for changing their negative thinking style and weaken the link between negative thoughts and low mood. Inter­personal therapy aims to provide clients with the skills to understand and change those aspects of their relationships which maintain their negative moods. Both forms of psychotherapy are as effective as antidepressant medica­tion in alleviating depressive symptoms but in conjunction with antidepressant medication are far more effective in preventing relapse than medication alone.

For anxiety that is associated with specific stimuli, various behavioural procedures where clients undergo controlled exposure to feared stimuli are maximally effective (Emmelkamp, 1994). These stimuli may be external (as in the case of agoraphobia) or internal (as in the case of trauma related memories). In vivo exposure for phobias is more effective than imaginal exposure and the longer the exposure and the more therapist support is provided the more effective the treatment tends to be. For obsessive compulsive disorders expos­ure to the stimuli that lead to anxiety (such as dirt) must be coupled with response prevention, so that the person learns to tolerate anxiety without en­gaging in compulsive rituals.

For alcohol and drug problems, the most effective treatment is a multimodal programme which includes the following elements: brief client-centred motiva­tional counselling to reduce denial and facilitate treatment engagement; self-control training to help develop skills necessary to manage high risk situations where alcohol or drug abuse are probable; social skills and assertiveness train­ing to help clients develop skills for making and maintaining supportive inter­personal relationships; and family therapy to enhance family communication and problem-solving skills and increase the rate at which the family reinforce the client for reducing drug and alcohol consumption (Hodgson, 1992; Edwards & Steinglass, 1995; Liddle & Dakof, 1995).

For schizophrenia, family education and therapy coupled with neuroleptic medication is the treatment most likely to reduce a pattern of chronic relapsing (Goldstein & Miklowitz, 1995). Florid positive symptoms may be controlled with neuroleptic medication. Family education should focus on the multi­factorial nature of the disorder and the importance of stress reduction in the prevention of relapse. Family therapy should aim to help family members reduce their expression of negative or intrusive emotions towards the person with schizophrenia. This in turn will reduce the exposure of the client to un­necessary stress which might precipitate a relapse.

For marital problems, marital therapy in which communication and problem-solving skills training occurs, where behavioural exchange skills are learned; and in which couples gain insight into the factors that contribute to their marital difficulties is the most effective treatment (Bray & Jouriles, 1995). For psychosexual problems such as vaginismus or erectile dysfunction behav­ioural interventions designed specifically to reduce sexual anxiety coupled with marital therapy focusing on enhancing communication is the most effective intervention (Emmelkamp, 1994).

For child focused problems, particularly internalizing and externalizing behavioural problems, individual work with the child coupled with family interventions which actively involve both parents and the child is treatment of choice (Estrada & Pinsof, 1995; Chamberlain & Rosicky, 1995). Effective family interventions aim to improve parent-child communication and provide parents with the skills to work together in supporting their children and consistently monitoring and explicitly discouraging antisocial behaviour while encouraging and rewarding prosocial behaviour.

Client and therapist characteristics

In addition to the common and specific factors in therapy, various character­istics of clients and therapists have an important bearing on outcome. Clients with more severe symptomatology and more comorbid diagnoses respond less well to therapy (Garfield, 1994). For both adults and children, clients from families where there is excessive criticism or overinvolvement or family prob­lems are more likely to make poorer therapy gains and to relapse (Kavanagh, 1992). For married couples, marital conflict has a negative impact on therapy outcome for other problems such as depression or alcohol abuse (Alexander, Holtzworth-Munroe & Jameson, 1994). For children, parental mental health problems, parental discord, stress, and lack of parental support are all assoc­iated with poor outcome (Chamberlain & Rosicky, 1995).

A variety of demographic and psychological characteristics of therapists have been found to contribute to a positive therapeutic outcome (Beutler, Machado & Neufeldt, 1994). Where therapists are similar to clients in terms of age, gender and ethnicity, it is more likely that clients will stay in therapy rather than drop out. Where clients perceive therapists to have a high level of expertise, to be trustworthy and to be attractive, they are more likely to have a positive outcome following therapy. Where therapists and clients place similar value on wisdom and honesty a positive therapeutic outcome is more likely. In contrast if clients place less value on interpersonal intimacy in friendship than their therapists, greater therapeutic progress occurs and this often involves change in relationship related values. Interpreting these findings, Beutler et al (1994) argue that humanistic values (such as valuing wisdom and honesty) may lead clients to engage in therapy to solve problems associated with difficulties in making and maintaining psychologically intimate relationships and over the course of therapy there may be a convergence of therapist and client values in the relationship domain.

Therapist level of professional experience influences both the process and outcome of therapy in quite specific ways (Beutler, Machado & Neufeldt, 1994). In comparison with less experienced therapists, those with more experience tend to develop better working alliances; to help more of their clients reach more favourable outcomes; and to have fewer clients who deteriorate. Fewer clients drop out of therapy when their therapist is experienced, and more severely distressed clients make therapeutic gains with experienced compared with inexperienced therapists.

The use of detailed structured therapy manuals and adherence to the guide­lines set out within the manuals is associated with positive outcome, although differing manualized therapies such as cognitive behaviour therapy or inter­personal therapy have been found in some studies to have similar effects (Beutler, Machado & Neufeldt, 1994). It seems to be that adherence to a coher­ent structured approach, rather than specific aspects of the approach that is important for a positive outcome.

Therapy duration and dropout

Within the psychodynamic tradition which has probably had the most widespread influence on our way of thinking about the therapy process, psychotherapy evolved into a long-tern intervention. However, surveys of outpatient clinic attenders show that most adult and child clients attend for only 6-10 session for individual and family or marital therapy (Garfield, 1994; Carr 1991; Weiss & Weisz, 1993). This is not surprising given that by 8 sessions 50% of adult clients have shown improvement and by 26 sessions 75% have made significant gains (Lambert & Bergin, 1994). Thus, it is clear that earlier sessions (the first 8-10) make a major contribution to improvement and there­after, the contribution of each session to improvement decreases.

In order for clients to benefit from the critical first 8-10 sessions, they must be engaged in therapy. Not all clients engage in therapy with equal facility. Clients from lower socio-economic groups, with lower educational levels and inaccurate treatment expectations are more likely to drop out of therapy (Garfield, 1994). Clients who participate actively in early sessions by talking freely and clients who have a positive emotional response to the therapeutic situation are more likely to stay in therapy. Therapists can prevent dropout by explicitly stating what the client and therapist’s roles entail, by giving a clear rationale for adopting these roles and by offering a brief a therapy contract for 8-10 sessions, which may be reviewed in the light of progress (Orlinsky, Grawe & Parks, 1994).

Harm caused by therapy

Approximately 10% of patients are harmed by therapy. Client characteristics, therapeutic techniques and therapist characteristics which contribute to deter­ioration have been identified (Lambert, 1994). Clients with diagnoses of border­line personality disorder or schizophrenia are particularly susceptible to deter­ioration. For clients with these problems, deterioration is more likely to occur in therapy which focuses on breaking down habitual coping strategies and defenses. People with schizophrenia or borderline personality disorder tend to benefit from supportive structured approaches to therapy rather than approaches involving confrontation. Coldness, disrespect and lack of empathy on the part of the therapist have been shown consistently to lead to deterioration.

PHILOSOPHICAL QUESTIONS

Two main viewpoints concerning the knowledge claims that may be made about findings from research on psychotherapy may be identified (Gergen, 1994). These are the positivist position which argues that the accumulated results of studies lead to the development of a grand narrative which has universal truth; and the social constructionist position which argues that the findings of psychotherapy research projects provide no more than local knowledge shared by highly specific communities. Ken Gergen (1994) has persuasively argued that the latter position is probably the most tenable. Within the field of psychotherapy quantitative and qualitative, process and outcome research will not lead to a grand narrative about how change always occurs in therapy or which therapies will always lead to problem resolution. However, this type of research may lead to many useful conclusions about which types of therapies work with which types of clients in particular socio-cultural contexts. The more rigorous this research is, the more useful and the less ambiguous the local narratives will be. Evaluation of therapeutic change from multiple-perspectives including the views of clients, therapists, referring agents and so forth is an important emphasis for such research. This is because it may throw light on how constructs such as improvement or deterioration are socially constructed by the community of people who are involved in the social venture we call psychotherapy. This community includes clients, therapists, insurance companies, service managers and so forth.

THE FUTURE

Psychotherapy outcome research provides useful information about the proportion of clients that benefit from therapy under particular circumstances. In Ireland today this information is critical in making a case for the develop­ment of a public psychotherapy service. Psychotherapy process research pro­vides information about how therapy works. This information is of particular interest to practitioners who wish to refine their practice so that they may offer their clients the type of therapy that is most likely to lead to improvement.

Conducting large scale psychotherapy research projects, requires funding and an extraordinary amount of co-ordination and co-operation among therapists, clients, researchers, and funding agents. In Ireland today, with the recognition that the majority of the people that consult their GP do so because of psychological problems (McKeown & Carrick, 1991); the recent publication of the first major book on psychotherapy in Ireland (Boyne, 1993); and the recent publication of the Psychotherapy Register, the climate is just about right for such work to blossom.

Figure 1. Basic two group design for a treatment outcome study

Figure 3. Common therapeutic factors which facilitate movement through the stages of change

References

Alexander, J., Holtzworth-Munroe, A., Jameson, P. (1994). The process and outcome of marital and family therapy. In A. Bergin & S. Garfield (eds.), Handbook of Psychotherapy and Behaviour Change (pp. 595-630). New York: Wiley.

American Psychiatric Association (1994) DSM – IV Washington, DC: APA.

Barker, C, Pistrang, N. & Elliott (1995). Research Methods in Clinical and Counselling Psychology. New York: Wiley.

Bergin, A. & Garfield, S. (1994). Handbook of Psychotherapy and Behaviour Change. New York. Wiley.

Beutler, L. & Crago, M. (1991). Psychotherapy Research. An International Review of Programmatic Studies. Washington: American Psychological Association.

Beutler, L. Machado, P. & Neufeldt, S. (1994). Therapist variables. In A. Bergin & S. Garfield (eds.), Handbook of Psychotherapy and Behaviour Change (pp.229-269)New York. Wiley.

Boyne, E. (1993). Psychotherapy in Ireland. Dublin. Columba Press.

Bray, J. & Jouriles, E. (1995). Treatment of marital conflict and prevention of divorce. Journal of Marital and Family Therapy, 21, 461-473.

Carr, A. (1991). Milan Systemic Family Therapy: A review of 10 empirical investigations. Journal of Family Therapy, 13, 237-264.

Carr, A. (1995). Positive Practice: A Step-by-Step-Guide to Family Therapy. Reading, UK: Harwood.

Chamberlain, P. & Rosicky, J. (1995). The effectiveness of family therapy in the treatment of adolescents with conduct disorders and delinquency. Journal of Marital and Family Therapy, 21, 441-459.

Cooper, H. & Hedges, L. (1994). The Handbook of Research Synthesis. New York: Sage.

Derogatis, L. (1983). SCL-90. Scoring and Procedures Manual for the Revised Version. Baltimore: Clinical Psychometric Research.

Edwards, M. & Steinglass, P. (1995). Family therapy treatment outcomes for alcoholism. Journal of Marital and Family Therapy, 21, 475-509.

Emmelkamp, P. (1994). Behaviour therapy with adults. In A. Bergin & S. Garfield (eds.), Handbook of Psychotherapy and Behaviour Change (pp. 379-427). New York: Wiley.

Estrada, A., Pinsof, W. (1995). The effectiveness of family therapies for selected behavioural disorders of childhood. Journal of Marital and Family Therapy, 21, 403-440.

Eysenck, H. (1952). The effects of psychotherapy: An evaluation, Journal of Consulting Psychology,  16, 319-327.

Eysenck, H. (1993). Forty years on: The outcome problem in psychotherapy revisited. In R. Giles (ed.), Handbook of Effective Psychotherapy (pp. 3-20). New York: Plenum

Garfield, S. (1994). Research on client variables in psychotherapy. In A. Bergin & S. Garfield (eds.), Handbook of Psychotherapy and Behaviour Change (pp.190-228)New York. Wiley.

Gergen, K. (1994). Realities and Relationships. Soundings in Social Constructionism. Cambridge, MA: Harvard University Press.

Giles, R. (1993). Handbook of Effective Psychotherapy. New York: Plenum

Goldstein, M. & Miklowitz, D. (1995). The effectiveness of psychoeducational family therapy in the treatment of schizophrenic disorders, Journal of Marital and Family Therapy, 21, 361 – 376.

Hodgson, R. (1992). Alcohol dependence treatments: The Research and Policy connections. In British Psychological Society Division of Clinical Psychology (eds.), On advising Purchasers. Part 2. Section II. Reprints (pp. 78-86). Leicester: BPsS.

Hollon, S. & Beck, A. (1994). Cognitive and cognitive behavioural approaches. In A. Bergin & S. Garfield (eds.), Handbook of Psychotherapy and Behaviour Change (pp. 595-630). New York: Wiley.

Kavanagh, D. (1992). Recent developments in expressed emotion and schizophrenia. British Journal of Psychiatry, 160, 601-620.

Kazdin, A. (1994). Methodology, design and evaluation in psychotherapy research. In A. Bergin & S. Garfield (eds.), Handbook of Psychotherapy and Behaviour Change (pp.19-71). New York. Wiley.

Lambert, M. & Bergin, A. (1994). The effectiveness of psychotherapy. In A. Bergin & S. Garfield (eds.), Handbook of Psychotherapy and Behaviour Change (pp.143-1 90). New York. Wiley.

Lambert, M. (1992). Psychotherapy outcome research: Implications for integrative and eclectic therapists. In J. Norcross & M. Goldfried (eds.). Handbook of Psychotherapy Integration (pp. 94-129). New York: Basic.

Liddle, H. & Dakoff, G. (1995). Efficacy of family therapy for drug abuse: Promising but not definitive, Journal of Marital and Family Therapy, 21, 511 -543.

McKeown, P. & Carrick, S. (1991). Public attitudes to depression: A national survey. Irish Journal of Psychological Medicine, 8, 116-121.

Norcross, J.  & Goldfried, M.  (1992). Handbook of Psychotherapy Integration. New York: Basic.

Orlinsky, D., Grawe, K. & Parks, B. (1994). Process and outcome in psychotherapy- Noch Einmal. In A. Bergin & S. Garfield (eds.), Handbook of Psychotherapy and Behaviour Change (pp. 270-238. ). New York: Wiley.

Pinsof, W. & Wynne, L (1995). The Effectiveness of Marital and Family Therapies, Journal of Marital and Family Therapy Volume 2. Special Edition. Washington, DC: AAMFT.

Prochaska, J. & DiClement, C. (1992). The Transtheoretical approach. In J. Norcross & M. Goldfried (eds.). Handbook of Psychotherapy Integration (pp. 300-334). New York: Basic.

Smith, M., Glass, G. & Miller, T. (1980). The Benefits of Psychotherapy. Baltimore: Johns Hopkins University Press.

Weisz, J. & Weiss, B. (1993) Effects of Psychotherapy with Children and Adolescents. London: Sage.

Wells, R. & Gianelli, V.  (1990). Handbook of the Brief Psychotherapies. New York: Plenum.