With treatment costs rising and mental health budgets dropping, intensive psychological treatments must be considered carefully and used only when they are seen as effective.  This can, be done in multiple ways.
 
The McMaster Family Assessment Device (FAD) is a short, self-report measure of family functioning that describes emotional relationships and functioning within the family.  The FAD has been validated with a number of clinical populations including a low-functioning population receiving in-home family therapy (Clark, Barrett, & Kolvin, 2000; Pfendler, Sharrow, Slattery, & Bean, 1997).  Kabacoff, Miller, Bishop, Epstein, and Keitner (1990) have assessed outcomes with a single administration of the FAD at termination from therapy.
 
Ridenour, Daley and Reich (1999) recommend using the General Functioning scale as a summary score of family functioning.  Epstein, Baldwin, and Bishop (1983) recommend using family rather than individual means and looking at each scale.  We compared three approaches to assessing outcomes using family means on FAD scales: assessing the statistical significance of change on outcome measures, identifying the magnitude of its change, and determining the percentage of families below clinical cut-offs at termination from therapy.

Participants
Thirty-eight families seen in the Family Links of Clarion County program since May of 1996 with both intake and exit scores on the FAD participated in this study.  Slattery, Hollis, Bean and Graham (1997) reported most families were under the official poverty level (57% earning less than $10,000 a year and 85% earning below $20,000 a year).   Pfendler and his colleagues (1997) reported that at least one parent frequently has a psychiatric diagnosis (36%), or a criminal history (37%).  The families are often headed by a single parent (48%), but also include blended (38%) and intact (14%) families.  Children in this population have a moderately high rate of prior psychiatric hospitalizations (M = 1.55) or foster care placement (M = .35), and were still judged to be at risk of being removed from their homes.  All identified patients ranged between two and 17 years of age.
 
Identified patients and their families were seen at Family Links of Clarion County, which is an in-home family therapy program.  Teams of professional and paraprofessional therapists saw families at least weekly. In general, families were seen for no more than six to eight months before being discharged or transferred to a less intensive service.  Therapy tended to focus on family structures, rules, roles and communication, although a range of techniques - both traditional mental health services and some less traditional ones - was used.

Procedure

The original 160 cases were reviewed to identify those with both intake and termination FAD scores.  The 38 complete cases were entered into StatView.   Successful change on the FAD over the course of therapy was assessed in three ways.  (1) Intake and exit scale scores were compared to determine whether a statistically significant change occurred.  (2) Scale scores were compared to clinical cut-offs. (3) Changes between intake and exit were categorized according to size: large (.50 or greater), moderate, (.10 to .49), none (+/- .10), or worse (less than  -. 10).

Statistically significant changes between intake and exit were observed on five of the seven FAD scales (Problem Solving, Communication, Roles, Behavior Control, and General Functioning) using a paired t-test.

Means of three of the seven scales (Problem Solving, Affective Responsiveness, and Behavior Control) dropped below clinical cutoffs by termination. Although means on these four scales remained in the clinical range, each improved.  The percentage of families moving below clinical cutoffs increased between intake and termination on each scale. Average percentage change between intake and termination was 18.4%.  The greatest change observed was on Behavior Control (44.7%); the smallest percentage was seen on General Functioning (7.8%).
 
An average of 77.1% of families improved more than .10 points across FAD scales. The greatest observed change was on Behavior Control (84.2%), the smallest observed change was on Roles (65.8%).  The average percentage of families making "large" change was 23.3%, however again we saw the greatest change on Behavior Control, with 57.9% of families making "large" change.  On the other hand, an average of 18.4% of families got worse (greater than .10).  The largest percentage of families got worse on
Roles (28.9%).

These results suggest that this type of family therapy was successful even with this difficult population and, further, support the use of the Family Assessment Device in assessing this change.  We used three ways of evaluating change: statistical significance of changes observed, percentages dropping below clinical cut-offs and the magnitude of change between intake and termination. Regardless of how change was assessed, this approach to therapy appears to be successful.
 
It must be noted, however, that this sample was not a random sample of the families seen in therapy at Family Links.  This paper only includes data from families who completed the FAD both at intake and exit since May of 1996.  Some families dropped out of treatment.  In other cases, therapists failed to administer the FAD at either intake or exit.  While there may be some bias affecting this sample, superficial inspection of these data suggests that there were no consistent differences between these groups, with one consistently doing better or worse than the other.

         Clark, A. F., Barrett, L., & Kolvin, I.  (2000). Inner city disadvantage and family functioning.  European Child and Adolescent-Psychiatry, 9, 77-83.

          Kabacoff, R. I., Miller, I. W., Bishop, D. S., Epstein, N. B., & Keitner (1990). A psychometric study of the McMaster Family Assessment Device in psychiatric, medical, and non-clinical samples. Journal of Family Psychology, 3, 431-439.

         Pfendler, B. A., Sharrow, W.W., Slattery, J. M., & Bean, T. (1997, April).  Validation of the McMaster Family Assessment Device (FAD) in an intensive family therapy program.  Poster presented at the annual meetings of the Eastern Psychological Association, Washington, D.C.

         Ridenour, T. A., Daley, J. G., & Reich, W. (1999). Factor analyses of the Family Assessment Device.   Family Process, 38, 497-510.

         Slattery, J. M., Hollis, M. L., Bean, T., & Graham, A.  (1997).  [Predictors of family therapy outcome.]  Unpublished data.
handout
 
 

        Slattery, J. M., Smith, W. N., Krapf, M. L., Buchenauer, E. L., & Bean, T. (2001, April).  Measuring improvement in family therapy using the Family Assessment Device.  Poster presented at the annual meetings of the Eastern Psychological Association, Washington, D. C.