Home » About MHCC » Research and Planning »  Research Studies and Publications » An Extended Journey To Assist At-Risk Students

An Extended Journey To Assist At-Risk Students

An Extended Journey To Assist At-Risk Students

Beginning in the late 1970s, budget constraints and accountability concerns led to a re-assessment of the mission of community colleges and their effectiveness. These issues were later articulated in influential writings, such as Parnell's The Neglected Majority (1985) and the Roueche, Baker and Roueche 1987 article on the "open door or revolving door." Demonstrating that community colleges were truly meeting the needs of at-risk students required the development of sound assessment practices (Ewell, 1983; Kreider and Walleri, 1988).

The development of Mt. Hood Community College's (MHCC) assessment program was influenced by national, state and local factors. At the federal level, the Carl Perkins Act placed particular emphasis on assisting at-risk students in the funding of vocational programs (Searcy, 1980). The evaluation requirements under the Perkins Act was a significant factor behind the deans of students of the thirteen Oregon community colleges decision in 1979 to form a consortium and initiate a project focused on the problems of at-risk students. In addition, the late 1970s was a period of retrenchment in higher education. This condition put a premium on demonstrating effectiveness before state legislatures. In the case of MHCC, national and state factors interacted with local conditions to bring about an institutional focus on assessment of student outcomes and institutional effectiveness. For the first time since its founding in 1966, MHCC was experiencing significant enrollment declines combined with a decline in both state and local funding. Thus, the focus on at-risk students, and especially retention of these students, was motivated by both enlightened self-interest and a genuine desire to improve institutional effectiveness.

In 1980-81, a Student Success Task Force was formed, under the leadership of the vice-president, with a charge from the president to review all institutional policies affecting student progress and to make recommendations for improvements that might increase the prospects for student success (Japely, Kennedy and Walleri, 1987).

In response to the presidential charge to find ways to promote student success, initial activities identified underprepared students and addressed their needs. Efforts also were made to assess student persistence in terms of students' varying intentions and goals in attending the college. These activities were later integrated into a broader institutional effectiveness program.

Guided Studies Program and Student Intentions. An analysis of transcripts of at-risk students revealed that some students were spending an extended period of time in developmental education without making any significant progress. Other students were enrolling in college-level course work despite basic skills deficiencies identified in placement tests. At-risk students had very high attrition, and those who were retained generally exhibited poor academic performance. As a consequence, a mandatory testing and placement system for entering students was initiated (Japely, Kennedy and Walleri, 1987; Walleri, 1996). A Guided Studies program was implemented for underprepared students to ensure that they did not enroll in college-level courses until deficiencies were addressed. In addition, time limits were placed on students in developmental education unless the student was making progress and formal exits requirements were put in place.

A student intent information system was established to enable faculty to analyze and understand student persistence. This system allows faculty and staff to track and assess progress based on the students' initial or subsequently defined goals, such as to secure a job, to transfer to a four-year institution, to upgrade job skills, or to engage in personal enrichment (Walleri, 1990).

From Student Success to Institutional Effectiveness. Beginning in the mid-1980s, an effort was initiated by the college president to translate the focus on student success into an institutional effectiveness program. The idea was to build an integrated systems approach to organizational effectiveness (Gray, 1993; Gratton and Walleri, 1993). The first important element in this approach was the assessment of student success in relation to student intentions. This element was then integrated with an instructional program improvement process based on the relationship between student outcomes (retention, job placement, transfer success) and institutional cost. The results of the program review provided input to the third element, strategic planning and resource allocation. The fourth element was staff and organizational development to assist the faculty and staff in better serving students and encouraging continuous improvement.

The initial activities produced modest improvement in the success of at-risk students. In terms of implementing an institutional effectiveness program, the initial activities were important first steps in achieving information-based decision making and gaining faculty and staff participation in the process. Nevertheless, the success of these efforts was limited in scope.

Success/Limitations of Student Success Initiative. The Guided Studies program touched about 40% of entering degree-seeking students. Subsequent cohort tracking showed that the persistence rate and level of academic performance of at-risk students equaled that of entering students whose placement tests indicated readiness to pursue college-level course work. However, this initiative did not expand beyond the Guided Studies program. There was no general increase in student success, especially as measured by retention rates. Less than 50% of entering degree-seeking students are retained through two years, and even fewer achieve a significant record of academic accomplishment as measured by earned credits.

Despite awareness of other strategies, such as an early warning system and automated degree audit, that could enhance student success, MHCC managers and faculty did not pursue initiatives beyond the Guided Studies program. One obstacle was ongoing budgetary constraints. A second reason was a generally held perception that the causes of attrition were beyond the control or influence of the college (financial problems, conflict with work schedule, etc.). This belief undoubtedly influenced resource allocation decisions.

Success/Limitations of the Program Improvement Process. The program improvement process produced comparative information on student outcomes, program effectiveness and costs that proved useful to the administration and board of education. As a result, the decision-making process became more information-based rather than dependent on intuition and anecdotal material.

The process did not succeed fully due to the fact that a majority of the faculty never became confident in the integrity of the process. The limitations of the measures and data were part of the problem. In addition, the faculty were not willing to accept balancing student outcomes against institutional costs for a program. That is, most of the faculty simply did not believe that cost per FTE was a relevant factor in evaluating programs. Another issue was equity, the failure to have a counterpart review process for the college and instructional support areas.

Success/Limitations of Strategic Planning and Resource Allocation. The major value of the strategic planning process was the high degree of faculty and staff participation. Each budget unit had a plan with identified needs. Cross-campus communication was significantly enhanced by the dissemination of area goals and strategies. The process helped in identifying areas of duplication and encouraged more collaboration. For example, several of the instructional units were independently initiating "classroom research" efforts to enhance student involvement. Through the planning process and with the assistance of Staff and Organizational Development, faculty from the different areas were able to collaborate and thus, enhance their projects. However, the process failed to identify institutional direction and priorities. The strategic plan was a compilation of area goals and resource requirements, totally several million dollars more than any realistic expectation for the college's budget. As a result, it was very difficult to make a logical connection between the strategic plan and subsequent resource allocation decisions.

Despite the many efforts aimed at improving student success and institutional effectiveness, the efficacy of these efforts probably was limited and inhibited by problems that are systemic to the institution. General institution-wide support for the emphasis on student success and service to the community is undermined by the perception of staff that they are working toward these goals on their own. Staff generally feel that while individuals and their units make extra efforts to assist campus visitors and constantly look for ways to improve the service they provide, the various units of the campus are unable or unwilling to work collectively (Ewell, 1996). The apparent lack of effective cross-unit communication was first identified in the 1987 accreditation review and continues to be an issue at the college.

The lack of communication throughout the organization provides a partial explanation for why student attrition has continued to be a concern at MHCC. Retention rates in the general student population have remained lower than desired and have shown little or no improvement from one cohort year to the next. Although various units provide excellent service to students and make changes with the hope of improvement, the implementation of such changes and their potential benefits to students have not been effectively communicated to the greater college community. The result has been a variety of piecemeal efforts that lack the collaboration among and the support of campus units needed to make a coordinated impact on student success.

Better communication is also needed among top level administrators and unit managers, and among the academic and non-academic units. In particular, meaningful two-way communication needs to take place to clarify both the specific needs of various units and the reasons behind administrative decisions regarding resource allocations in terms of strategic priorities and assessment results.

Demands for accountability have intensified. External demands include efforts such as Oregon's Shared Information System (SIS) that requires all state workforce development agencies, including the community colleges, to report on performance in moving students/clients to self-sufficiency. SIS is a unit record data system that allows for matching agency records by SSN to report on employment and wages and the success of community college students transferring to the state's four-year colleges and universities. As a result of property tax limitation measures approved by the voters, the state now directly provides the vast majority funding for education, and the Governor and Legislature will be using the SIS results in allocating scarce funds among the various state agencies.

Another external demand is the recently adopted emphasis on student outcomes by the Northwest Association of Schools and Colleges for the accreditation process. Whereas most of the regional accrediting groups have ten or more years of experience in student outcomes assessment guidelines, the first explicit effort by the Northwest Association did not take place until the 1994 accreditation handbook.

Internal demands have also increased because a statewide property tax limitation has necessitated budget cuts. More than ever before it has become paramount to base resource allocation on reliable data that guide sound planning.

Factors Facilitating Progress to Date. Through the 1980s and into the early 1990s, the driving force behind development of the institutional effectiveness program had been a single individual. He moved from vice president to president in 1985. After this move, the vice-president position was left vacant. Although the various systems that had been previously set-up were sustained, it became increasingly difficult for the president to address identified weaknesses because of competing demands on his time and organizational inertia. The lack of any leadership replacement in the area of assessment, combined with budget constraints, limited efforts to maintaining existing systems and the status quo. In 1992, the vice president position was filled and the new person became a catalyst for change that allowed the institution to achieve the following improvements.

Reform of the Strategic Planning Process. In response to an appraisal of the limitations of the initial strategic planning process, the college's planning council, including the new vice president, conducted a review and recommended changes. Recommendations included simplifying the process and incorporating TQM techniques into strategic planning. A faculty member with expertise in TQM was granted release time to assist the council in designing a new planning system. These changes led to the identification of ten institutional goals. Every year these goals are re-assessed and prioritized. Each unit or division of the college revisits its area strategic plan on an annual basis and must clearly link its plan to the institutional goals.

Development of Program/Function Review Process. One of the ten institutional goals calls for the allocation of resources based on a program/function review process in concordance with priorities derived from the college strategic plan. The task force charged with developing the process included staff from a broad cross-section of areas on campus, including faculty representation, in the hope of promoting campus-wide cooperation with and confidence in the process. Program/function review is more equitable than its predecessor in that all instructional programs and administrative and student support areas participate in the review.

The process is ongoing and consists of two phases. The first is based on a standard set of indicators that are primarily quantitative. Phase two complements phase one by requiring individualized and qualitative assessment, which includes satisfaction surveys (administered to students and other college staff) and external peer review. The overall review is ongoing and integrated with other related activities such as the accreditation self-study.

Title III Grant Support for At-Risk Students. A five-year Title III grant has provided an opportunity to build on previous student success initiatives and it emphasizes assessment. The result is that a comprehensive review of both student services and instruction is underway. The assessment efforts include implementing a new orientation program for entering students, replacing a paper and pencil placement test with a computerized adaptive test, conducting curriculum reviews, developing linked courses, and a proposed integrated studies program, to name a few. The heart of these efforts is an intensive intervention program for at-risk students. Counselors and instructors, acting as student advisors, make multiple contacts with individual students each term. The expectation is that retention and success can be improved by demonstrating an interest in a student's well-being and progress.

Assessment Matrix. An assessment matrix and activity calendar have been developed in response to the perceived flurry of assessment activities on campus (Walleri and Stoering, 1996). The intention in developing these tools was threefold: illustrate the integration and overlap of the assessment activities; promote communication and understanding of them; and by doing so, foster participation in them.

Role of Information Technology. An information technology project has provided additional support for the above initiatives. A campus network system has improved communication and facilitated group work. A new management information system will allow for greater access and improved information, especially in the area of student tracking.

Teaching & Learning Center. The professional and organizational development component has been strengthened through a private grant which has allowed the college to establish a Teaching & Learning Center (TLC). The TLC provides a setting for formal professional development activities, such as training in the instructional uses of computer technology, and as a place where faculty can informally share and explore issues of concern. For example, the TLC is currently sponsoring a review of the student evaluation of instruction system.

Problems Blocking Progress to Date. Although recent activities have overcome many of the limitations of earlier initiatives, there are still issues that need to be addressed. First, staff cooperation with and participation in the activities need to be maintained. Faculty commitment to the intensive intervention with at-risk students is especially important to cultivate. Second, a stronger system for linking budget development directly to program/function review data and area strategic plans needs to be put in place. As it now stands, program/function review data are used primarily for affirmation that programs/areas are doing well, and in some cases, to make improvements. More needs to be done to follow through with difficult decisions about resource allocation based on the review process and to make clear to campus constituencies the explanations for the decisions. Finally, improvements to the student tracking system would facilitate assessment efforts. More accurate, specific reports and a faster turn around time aligned with the budget development schedule would give the campus more current and reliable information with which to work. This in turn would yield stronger campus support and a stronger tie to resource allocation.

Many of the problems that have impeded progress to date may decrease with time because of continued efforts to revise and refine institutional effectiveness at MHCC. The college is also experiencing large-scale changes in administration and technology and at the same time the pressure of constantly increasing budget constraints, all of which may have an impact on student success and institutional effectiveness efforts.

In 1996 both the dean of instruction and the president retired and the vice president resigned. New leadership was brought to the college. Staff are exercising guarded optimism that the new leadership will yield positive changes for the college, yet at the same time, all are aware that budget constraints continue to increase. Most are adopting a "wait and see" attitude to the new administration, reserving judgment until more time has passed.

Another major challenge is to fully utilize new technologies effectively in supporting the institutional effectiveness program. Again, it is too early to tell how much the new computer network system will facilitate the availability of high quality information and better dissemination of information, but as the college goes through the many stages of implementation, this continues to look promising.

The college is scheduled for accreditation review in the fall of 1997. Faculty and staff are currently involved in the preparation of the self-study document. The accreditation review will be a major test of the institutional effectiveness program and an opportunity to reflect on progress of the past ten years.

R. Dan Walleri is director of research, planning and computer services at Mt. Hood Community College in Gresham, Oregon. He is a past president of the National Council for Research & Planning, a council of the American Association of Community Colleges. He has written numerous articles on student outcomes and institutional effectiveness.

Juliette M. Stoering is research associate in the office of research and planning at Mt. Hood Community College. She holds a Master's in Psychology from the University of Victoria.