Home » About MHCC » Research and Planning »  Research Studies and Publications » Accountability Through Collaboration: The Oregon Shared Information System

Accountability Through Collaboration: The Oregon Shared Information System

This article traces the collaborative efforts of the Oregon community colleges to respond to state level accountability demands. The colleges participated in the development of the Oregon Community College Unified Reporting System (OCCURS), and also identified key effectiveness indicators upon which their performance could be evaluated. Finally, the colleges participated in an even broader accountability system involving 11 other state agencies -- the Oregon Shared Information System (SIS).

As the 1990s come to a close, state-level accountability in the form of outcomes assessment has become prevalent across the United States. Rather than a passing fad as some might have hoped, assessment programs have become even more important with the advent of performance-based funding (Alfred & Carter, 1997; Gaither, 1997; Ewell, 1997; Roueche, Johnson & Roueche, 1997). Postsecondary institutions must now compete with social and workforce development programs for scarce dollars as governors and legislators attempt to achieve significant reforms without increasing the general tax burden. Public colleges and universities are no longer considered separate entities with unique purposes, but rather part of a system of organizations that are evaluated on the basis of their ability to position states to thrive in unforgiving competitive national and global economies.

The 17 Oregon community colleges have sought to meet this challenge through collaboration with each other and with other state agencies. Locally governed but increasingly dependent on state funding, the colleges have been able to retain their traditions of responsiveness to community needs while at the same time contributing to the achievement of state goals and objectives. Rather than simply reacting to state level mandates, the colleges have actively participated in building the effectiveness agenda. As a result, the Oregon community colleges represent a model of self-directed action aimed at serving both community and state interests.

The story of how this came to be has three key parts. First, the context is set by a series of state workforce development and education reform initiatives. The second part involved the colleges' voluntary decision to implement a state level reporting system, including the identification of key effectiveness indicators aligned with those developed by the State of Oregon in its economic development plan. Third and finally, the colleges have worked with eleven other state agencies to build a Shared Information System (SIS), designed to measure and assess outcomes for state level accountability.

In the late 1980s, the State of Oregon embarked upon an ambitious goal of building a "work force that is measurably the most competent in America by the year 2000, and equal to any in the world by 2010" (Oregon Shines, 1989, p 1). Under the leadership of then-Governor Neil Goldschmidt, an economic development plan was prepared to guide the state in achieving this goal. The Oregon Progress Board was established to identify and monitor specific objectives to be achieved in five year increments through the year 2010 (Progress Report, 1990). Known as the Oregon Benchmarks, many relate to education and training. For example, one benchmark describes the "percentage of work force that has received at least 20 hours of education related to work skills and knowledge within the past 12 months," while another deals with the "percentage of adult Oregonians demonstrating prose, document and quantitative literacy" (Commissioner's Task Force, 1992, p 9). Each benchmark has been operationalized, and the Progress Board has led an effort to compile the needed information to assess achievement (Oregon Shines II, 1997).

In 1991, the Oregon legislature affirmed the work of the Progress Board and passed the Oregon Education Act for the 21st Century. This law initiated one of the leading educational reform efforts in American today (Oregon Report Card, 1996). Designed to replace "seat time" with demonstrated proficiencies for advancement, the law basically eliminated the traditional high school diploma and replaced it with the Certificate of Initial Mastery (CIM), to be achieved by the end of the 10th grade, and the Certificate of Advanced Mastery (CAM), to be achieved before a student could leave high school.

To date, the focus of the educational reform effort has been to develop standards. "Content standards describe what students must know and be able to do in nine academic areas. ... Performance standards describe the number, type and minimum scores required to meet benchmarks at grades 3, 5 and 8, the Certificate of Initial Mastery at grade 10 and the Certificate of Advanced Mastery at grade 12" (Oregon Report Card, 1996, p. 8). The State Board of Education is expected to give final approval to these standards in the near future.

Articulation between K12 and Community Colleges. The Education Act for the 21st Century also had a significant impact on the community colleges and the Oregon State System of Higher Education (OSSHE). High school students would no longer be bound by age and grade, but may advance to postsecondary institutions upon completion of the CAM. The educational reform effort was driven by a vision of "seamless" education which may involve structured work experience and on-the-job training as well as formal postsecondary education. Colleges and universities have had to re-examine entrance requirements and place greater emphasis on articulation arrangements, such as TechPrep and 2+2+2 programs.

Budget Constraints Impede Progress on Education Reform. During the 1990s, two statewide initiatives were enacted by the voters reducing property taxes, shifting the primary funding source for the K12 system and community colleges away from the local level to the state. Revenue sources for the state are limited to the income tax and lottery proceeds because there is no sales tax in Oregon. Even though the state economy has been and continues to be very strong, the property tax limitations have severely limited the ability of the state to provide adequate funding for a range of services. A large prison expansion program and the needs of other programs have increased the competition for scarce funds, resulting in most school districts, the community colleges and OSSHE experiencing severe fiscal constraints.

The convergence of state economic development efforts, education reform and fiscal austerity placed the community colleges in a precarious situation. With demands for accountability high among both policy makers and the general public, and with funding increasingly dependent on performance, greater state control and mandates appeared inevitable.

In 1991, then-Commissioner of Oregon's community colleges, Mike Holland, recognized the need to respond to accountability demands and established a Statewide Data Collection Committee made up of representatives from the community colleges. In the same year, Dale Parnell became Commissioner of Community Colleges and appointed a Task Force on Accountability, which in turn, absorbed the data collection committee. The Task Force followed two parallel tracks, one focused on the original data collection effort and the other on effectiveness indicators.

Task Force members studied existing federal and state reporting requirements, how each of the community colleges was responding to accountability demands, and the systems used in other states for data collection and reporting. Representing the desire among a majority of staff at the colleges, the Task Force members wanted to preserve local autonomy while also responding to the reality of a shifting state level agenda in which accountability was key. The Task Force completed its recommendations and delivered its final report in October, 1992 (Commissioners Task Force, 1992).

There were two major outcomes from the work of the Task Force. For the first time in their history, the colleges embraced a set of statewide effectiveness indicators. These "key effectiveness indicators" are listed in Table 1 (Commissioners Task Force, 1992, pp 5-9). All of the indicators have a corollary in the Oregon Benchmarks and five of the seven are outcomes-based (Walleri & Stoering, 1996). This latter characteristic would become especially important in future years in responding to accountability demands as described below.

The second major outcome of the Task Force's work was the establishment of the Oregon Community College Unified Reporting System (OCCURS). Prior to OCCURS, all state reporting was based on paper reports prepared at each college. The Commissioner's office had no central data system to prepare ongoing reports or respond to ad hoc requests for information, nor was there a shared database upon which information could be compiled on the key effectiveness indicators described in Table 1.

Basic Approach to Building OCCURS. Emerging computer network technology allowed the Task Force to design an approach that would enable the colleges to electronically transfer standard data elements to a central server that could then be used by the Commissioner's office to generate needed information. This approach gained acceptance by the colleges since it allowed them the freedom to pursue development of their own information systems as long as they complied with state level data needs.

Progress in Implementing OCCURS. Implementation of OCCURS proceeded slowly to accommodate the varying information system capabilities of the different colleges. An Oversight Committee consisting of representatives from each of the colleges was established to advise the Commissioner, set standards and timelines, and monitor quality in data reporting. Subcommittees involving additional college representatives worked on the standards for each data file. To date, all the colleges are submitting files on programs, students and courses by term and an annual student completions file. Work has begun on the development of two additional files: an enrollment file (one record per student, per course, per term) and a staff file.

OCCURS has eliminated the need for most of the paper reports prepared by the colleges in the past. The system generates reimbursement reports for allocation of state funding based on student FTE, the Profiles document which offers a statistical portrait of the colleges, and various ad hoc reports for legislative and other purposes.

In conjunction with SIS, described below, OCCURS matches records with the Oregon Employment Division and OSSHE to gather data on employment, wages, and transfer performance of former community college students. This enables OCCURS and the colleges to prepare reports on some of the key effectiveness indicators described in Table 1.

The development of OCCURS was not an easy or painless process. Concerns expressed by the Oregon Attorney General about the use of SSN as the student ID delayed implementation of unit record matching for several years until an approved student disclosure system could be put in place. Lively debates continue within the Oversight Committee and its subcommittees over the scope and definition of various data elements. Although there is no evidence of misuse or inappropriate application of the resulting information at the state level, many community college staff still harbor concerns over how the information might be used to make unwarranted comparisons across the colleges.

Through OCCURS the community colleges have participated in the development of the Oregon Shared Information System (SIS). This system allows for unit record matches across various state agencies involved in workforce development. SIS originated with the 1989 Legislature and the desire to examine and assess the effectiveness of job training and placement programs in pursuit of a world-class workforce as described previously (Shared Information System, 1997).

The 1991 Legislature enacted specific measures of workforce development effectiveness and required SIS to provide reports describing progress of state agencies in achieving them. The measures are wages at placement, job retention and retention earnings, skill gain, customer satisfaction, and cost per placement. The agencies identified to participate in SIS are listed in Table 2.

Organization of SIS. SIS is administered by the Oregon Workforce Quality Council, and members are appointed by the Governor. The head of each agency serves on an Oversight Committee and representatives from the agencies compose various committees that work on data definitions and standards similar to the format for OCCURS. Each agency submits unit record data which is then matched and analyzed by SIS to prepare reports responding to its legislative mandate.

Use of SSN in Unit Record Matches. Full implementation of SIS was delayed for several years following a formal opinion from the Oregon Attorney General which prohibited unit record matching on SSN without disclosure and permission from the agencies' clients. Likewise, full implementation of OCCURS was also delayed. In addition, many of the agencies were hesitant about participating due to concerns over the security of the data and fear that the information would be used to negatively impact clients. To allay agencies' concerns, it was agreed that SIS would only prepare aggregate reports and that the data would not be used for case management. Security systems and agreements were implemented to protect confidentiality.

SIS Reports to the Legislature. SIS delivered its first report on labor market outcomes to the 1997 Legislature. Covering the 1993-1996 period, 1,595,563 individuals were served with 1,047,995 exits from the participating agencies. Of these exits, 620,563 (59%) were employed with an average weekly wage at exit of $248. Of particular legislative concern was the number of clients served by multiple agencies. However, SIS data revealed that only 19% (190,245) "were served by more than one agency" and "less than 4% of clients were served by three or more agencies" (Shared Information System, 1997, p 4). Information on such clients indicated they "were the most in need of services" (Shared Information System, 1997, p 4). The individual agencies are also able to obtain analysis from SIS for internal use and for legislative reporting.

It is difficult to identify a specific connection between SIS and legislative outcomes. Nonetheless and despite a strong desire not to increase taxes, the legislature has responded positively to the information provided by SIS and agency budgets, including the community colleges, have been strengthened.

Mobility Continuum. One unique aspect of SIS is the use of what is termed the "mobility continuum" (Shared Information System, 1993). Clients are profiled based on socio-economic background and education level which can be used as a control in assessing outcomes. That is, one would not expect the same outcomes from a single head of household exiting a job training program as for a university graduate from a relatively privileged background. The mobility continuum not only provides perspective in assessing aggregate outcomes of clients, but also in assessing the effectiveness of the providers of training and education programs. Without controlling for the backgrounds of the clients, there is simply no equitable basis upon which to compare outcomes across the different agencies. This is especially important for community colleges that serve a relatively high number of the disadvantaged students.

With the development of state unit record data exchange systems such as in Oregon, the inability to track students that leave the state has created a gap in providing complete information. Preliminary efforts to build data sharing arrangements among the states has begun with one example being the Western States Data Sharing Consortium (Network News, 1997).

The potential for a national unit record data sharing arrangement has been raised by the National Postsecondary Education Cooperative (NPEC). Authorized by Congress in 1994 and established through the National Center for Education Statistics (NCES), the mission of the Cooperative is "to identify and communicate on-going and emerging issues germane to postsecondary education, and to promote the quality, comparability and utility of postsecondary data and information that support policy development, implementation, and evaluation" (NPEC, 1997, p.1). NPEC is governed by a Council and Steering Committee, both made up of representatives from postsecondary institutions and agencies from across the country.

Among NPEC's working groups is one working on Unit Record Exchanges which is now in its second year of operation. The group is currently reviewing state and national data sharing arrangements with the potential goal of developing a pilot national data exchange project through NPEC. Such a system would significantly enhance the ability of institutions and states to obtain and utilize more complete information on former students. This would include the ability to track transfer students for Student Right-to-Know reporting.

OCCURS and SIS have enabled the Oregon community colleges to respond to accountability demands collectively and in a collaborative environment. The colleges have been active participants in setting the effectiveness agenda rather than reacting to state established mandates. Feasible and realistic measures have been identified upon which the colleges as a system can be assessed. The systems in place provide a rich database upon which the colleges can draw information to respond to local, state and federal accountability demands in addition to accreditation standards. All of this was accomplished in less than a decade, on a voluntary basis, and without having lost local autonomy.

For information on Oregon Shines and the Oregon Benchmarks, the reader should access: http://www.econ.state.or.us/opb/index.htm. For information on Oregon's education reform efforts and the Education Act for the 21st Century see: http://www.ode.state.or.us. Although OCCURS does not have its own web site, the type of data and information available from OCCURS can be found at the web site for the Office of Community College Services at: http://www.occs.state.or.us. The SIS web address is: http://www.emp.state.or.us/sis/sis.htm. Additional information on NPEC can be found at: http://nces01.ed.gov/npec.