Use of Non-Obvious Indicators for Screening of DWI Offenders

Thomas H. Nochajski, Brenda A. Miller, Diane K. Augustino and Robert J. Kramer

Research Institute on Addictions, 1021 Main Street, Buffalo, New York 14203-1016, USA

ABSTRACT

This study examined the effectiveness of non-obvious indicators for detecting individuals that may have potential problems with alcohol or drugs. The Research Institute on Addictions in conjunction with the New York State Department of Motor Vehicles and the Governor's Traffic Safety Committee has developed a screening instrument (RIASI) for use with the Drinking Driver Programs in New York State. The samples in this study consisted of DDP participants from Onondaga county, where all DDP participants are given clinical assessments prior to participation in the DDP. One sample consisted of 246 individuals that were administered the MAST, while the second sample consisted of 125 individuals that were administered the RIASI. Referral rates for clinical re-evaluation were basically the same for the MAST (56%) and RIASI (54%). However, the percent of false positives, those identified as potentially having alcohol or drug problems but not needing treatment, was significantly lower for the RIASI (25%) than for the MAST (41%). For the false negatives, individuals identified as needing treatment but missed by the screening procedure, the RIASI (6%) was again significantly lower than the MAST (38%). These results indicate that use of non-obvious indicators helped increase the accuracy of the screening procedure using the RIASI relative to the procedure using the MAST. Continued work on validation of subscales within the RIASI points toward even greater efficiency of the screening procedure in the DDPs.

INTRODUCTION

The Research Institute on Addictions and the New York State Department of Motor Vehicles recently implemented the RIA Self Inventory (RIASI), a new alcohol/drug problem screening instrument, as the screening instrument for use in the New York State Drinking Driver Programs (DDP). Screening is a quick and inexpensive approach for identifying individuals who require more professional (and higher cost) assessments. With good screening methods only the individuals who are most likely to require treatment are sent for the more intensive (and expensive) diagnostic evaluation. While the inclusion of information on the severity of the problem is both a desirable and feasible feature for a screening instrument, screening is not meant to take the place of a diagnostic evaluation.

For a screening instrument to be of genuine value, it should be able to accurately identify the problem group, an attribute known as criterion validity. Two important measures of a screening test's performance are sensitivity (the proportion of individuals with the problem who are identified) and specificity (the proportion of individuals without the problem who are identified as not having a problem). Sensitivity and specificity need to be balanced to ensure that the rates of false negatives (individuals who screen negative but actually have an alcohol or drug problem) and false-positives (individuals who screen positive but do not have an alcohol or drug problem) are within acceptable limits.

Prior to the implementation in the DDPs of the RIASI as the sole screening instrument, programs used either the Michigan Alcoholism Screening Test (MAST; Selzer, 1971) or Mortimer-Filkins Questionnaire (Mortimer et al., 1971). The MAST and Mortimer-Filkins both tend to focus on factors associated with late-stage alcohol dependence which may be particularly inappropriate for DWI offenders, since they tend to be younger and exhibit less severe problems with alcohol. Further, the obvious nature of the alcohol problem items in the MAST are susceptible to false responses by individuals wishing to avoid detection (Nochajski et al., 1993; Otto and Hall, 1988). In addition, because of some confusion over certain questions in the MAST, a high rate of false positive screens may occur. A misclassification rate as high as 58% has been reported for the MAST (Zung and Ross, 1980). The Mortimer-Filkins Questionnaire, can be burdensome and the complex weighted scoring procedures do not appear to be justified (Mischke and Venneri, 1987).

The RIASI assesses a variety of proximal (current consumption, alcohol beliefs, preoccupation with alcohol) and distal characteristics (hostility, sensation seeking, depression, anxiety, interpersonal competence) that are highly correlated with alcohol or drug problems. The basic rationale for development of an instrument that uses non-obvious indicators for screening purposes, such as the RIASI, comes from the literature that underscores the need for such an instrument. The premise is that direct problem related questions will be too explicit and result in an under-identification of individuals being sent for further evaluation.

The current study compares false negative and false positive rates for the RIASI and MAST in samples of convicted drinking-drivers from a DDP in New York State.

METHODS

Participants

The participants in this study were from a DDP in New York State. The DDP is part of a secondary prevention process for convicted drinking-drivers. The primary target of this program is the first-time offender, however, second offenders, and sometimes third offenders, are eligible if the prior offenses occurred more than 5 years antecedent to the current offense. The DDP is a voluntary 16 hour 7 week educational program. As an incentive, potential participants are offered a conditional license that allows them limited driving privileges. Participants chosen for this study attended a program in Onondaga County. Onondaga county was chosen because as part of a plea bargaining process all first offenders must get a clinical evaluation and most attend the DDP. Additionally, as part of the state laws all second offenders are also mandated to get clinical evaluations. These individuals are also screened for alcohol or drug problems as part of the DDP program. Since the individuals are evaluated prior to the screening process, determination is already made as to who needs treatment and who does not. This means that the false positive and false negative rates for the respective screening instruments can be evaluated and compared. In April of 1994 the screening instrument being used by the DDP site from Onondaga County changed from the MAST to the RIASI. The participants in the current study represent 7 classes prior to the change (n = 97) and 7 classes following the change (n = 113).

Instrument

The RIASI is a 49 item instrument that measures distal (hostility/aggression, sensation seeking, depression, anxiety, interpersonal competence, childhood risk factors, criminal history, health-related) and proximal factors (current drinking habits, preoccupation with alcohol, alcohol beliefs, use of alcohol to alleviate problems, and family history) associated with alcohol or drug problems. A total of any 10 positive responses would require the individual to go for the more intense clinical evaluation. The RIASI was developed specifically for use in the DDP, with the cutoff of 10 positive responses based on the baseline rates of problem-drinkers/drug users in the DDP population. While the belief is that the use of non-obvious indicators will work with other populations, the cutoff points may change as the baseline rates for problem-drinkers/drug users changes.

The samples used for development of the RIASI were large enough to allow for examination of subgroups based on age, race, gender and region of the state. As a result of using the proximal and distal factors and the evaluation of subgroups, the RIASI is more capable of dealing with the heterogeneous nature of the DWI population. Furthermore, the RIASI is easily administered, has a simple scoring procedure, does not need a clinician for administration and interpretation, and is capable of distinguishing between various problem groups. Finally, the RIASI is seen as a dynamic document, meaning that the instrument will continue to be monitored and changed as the DWI population changes.

Information concerning internal consistency shows that the magnitude of the Cronbach s alpha coefficient remained relatively stable across the different samples of convicted drinking-drivers. Cronbach s alpha was .819 for a 1992 statewide sample of 5,059 DDP participants; .819 for a 1993 sample of 1,024 DDP participants from Erie County in Western New York; .807 for a 1993 sample of 209 convicted drinking-drivers from New Jersey; .814 for a 1994 sample of 1,477 DDP participants from New York State; and .808 for a 1994 sample of 100 high risk convicted drinking-driver offenders from Erie County. These compare to the .893 found for the current sample of 113 individuals from an Onondaga county DDP. For more in depth information on the development, reliability, and validity of the RIASI, see Nochajski and Miller (1995).

RESULTS

There were no significant differences between the MAST (M) and RIASI (R) groups for the breakdown of gender (M = 17.9% female vs. R = 13.1% female), refusal of breath and/or blood test for alcohol content (M = 10.9% vs. R = 16.1%), and percent with a Breath Alcohol Content (BAC) of 200mg% above (M = 9.8% vs. R = 15.2%). However, the number of repeat drinking-driver offenders was significantly greater in the MAST classes than in the RIASI classes (26% vs. 15.7%, p<.05). While not significant, there was also a tendency for the individuals in the MAST sample than those in the RIASI sample to have been referred for treatment (55.2% vs. 44.4%). This latter finding reflects the higher rate of repeat offenders.

The results for the comparisons between the RIASI and MAST for false positive and false negative rates are shown in Table 1. Of particular interest is the comparison with the MAST cutoff of six positive responses, since that was the one used by the DDPs. The false negative rates for the RIASI were significantly lower than for the MAST (p<.0001).

Table 1
Comparison of MAST and RIASI across False Negative and False Positive Rates

In addition, comparisons with a MAST cutoff of three positive responses also showed the RIASI to be significantly lower on the false negative rates (p<.01). However, when using a cutoff of six for the MAST, the false positive rates show slight differences initially in favor of the MAST. This latter finding is not surprising given that the MAST would send only 16% for an evaluation in contrast to the 46% sent by the RIASI. Thus, the MAST has poor sensitivity, resulting in high false negative rates but because it does not send very many individuals for evaluations, it does fairly well in terms of specificity, yielding lower false positive rates. In contrast, the RIASI does equally well at identifying the problem and non-problem cases, with both overall correct rates above 80%.

Since repeat offender status is cause for automatic referral for evaluation in the DDP, further analyses were performed using repeat offender status in combination with the respective screening instruments. With respect to the false negative rates, the RIASI still performed significantly better than the MAST, whether six (p<.0001) or three positive responses were used as the cutoff. As a final test, the RIASI and repeat offender status were compared against a MATRIX system involving the MAST, BAC, and repeat offender status (Table 2). This is the system for referral that was used by the DDPs in New York state prior to the implementation of the RIASI. Individuals were referred if the BAC was 180mg%, they were repeat drinking-driving offenders, or they had six or more positive responses on the MAST. Referral also occurred if the individuals had a MAST score between three and five plus a BAC between 100mg% and 179mg%. Even with the addition of BAC, the screening process using the RIASI and repeat offender status still was significantly better on the false positive rates (p<.05) and marginally better on the false negative rates (p <.10).

Table 2
Comparison of MAST and/or Repeat Offender with the RIASI and/or Repeat Offender Across False Negative and False Positive Rates

CONCLUSIONS

High false negative rates found for the MAST attest to the falsification that occurs when using a direct method in screening a criminal justice population. The non-obvious indicators in the RIASI helped reduce the number of false negatives substantially, while not significantly increasing the number of false positives. What this indicates is that the detection of problem individuals can be greatly improved by inclusion of non-obvious indicators. Furthermore, since the false positive rates remain relatively low, this improvement can occur without overburdening the treatment system.

The results for this study are specific to the DDP population. Given the method used in determining the cutoff points for the RIASI, extension of its use to other populations would need to include investigation of baseline rates for alcohol or drug problems within that population. Currently, the RIASI is being pilot tested with samples of convicted drinking-drivers that are on probation with plans to extend this to other criminal justice populations. In addition, work continues on the development of subscales that may help improve the efficiency of the RIASI and assist evaluators and treatment providers.

ACKNOWLEDGEMENTS

This research was a cooperative venture with the New York State Department of Motor Vehicles Division of Traffic Safety Services and was supported by the Governor's Traffic Safety Committee of New York State with funds from the National Highway Traffic Safety Administration.

REFERENCES

Mischke, H. D., & Venneri, R. L. (1987). Reliability and validity of the MAST, Mortimer-Filkins questionnaire and CAGE in DWI assessment. Journal of Studies on Alcohol, 48(5), 492-501.

Mortimer, R. G., Filkins, L. D., Lower, J. S., Kerlan, M. W., Post, D. V., Mudge, B., & Rosenblatt, C. (1971). Court procedures for identifying problem drinkers: Report on Phase I. (DOT HS-800 630). Washington, DC: U. S. Department of Transportation.

Nochajski, T. H., & Miller, B. A. (1995). Training Manual for the Research Institute on Addictions Self Inventory (RIASI). Buffalo, NY: Research Institute on Addictions.

Nochajski, T. H., Miller, B. A., Wieczorek, W. F, & Parks, K. A. (1993, June). The utility of non-obvious indicators for screening of DWI offenders. Paper presented at the annual meeting of the Research Society on Alcoholism, San Antonio, Tx.

Otto, R. K., & Hall, J. E. (1988). The utility of the Michigan Alcoholism Screening Test in the detection of alcoholics and problem drinkers. Journal of Personality Assessment, 52, 500-505.

Selzer, M. L. (1971). The Michigan Alcoholism Screening Test: The quest for a new diagnostic instrument. American Journal of Psychiatry, 127, 89-94.

Zung, B. J., & Ross, M. (1980). Factor structure of the Michigan Alcoholism Screening Test (MAST) among acutely disturbed psychiatric patients. Journal of Clinical Psychology, 36, 806-812.