Adopted Spring 1996

The Educational Policies Committee - 1995-1996
Regina Stanback-Stroud, Chair - Rancho Santiago College
Linda Collins, Los Medanos College
Jill Harmon, Fresno City College
Jim Higgs, Modesto Junior College
Paul Setziol, DeAnza College
Jean Smith, San Diego Community College District - ECC
Robert Smith, College of San Mateo
Robert Rockwell, CIO Representative - Mt. San Jacinto College

Introduction
As a result of its desire to see the quality and educational effectiveness of community colleges maintained, the Academic Senate for California Community Colleges (Academic Senate) has had a long-standing commitment to faculty-based program review mechanisms. As public support for funding colleges and universities diminishes and fiscal resources become increasing constrained, planning and effective use of the sparse educational dollars is paramount. Additionally, standards of accreditation, general and categorical accountability, and community educational needs further motivate colleges to strive for sound educational practices that support the integrity of the college and its programs and services.

Over the past six years, The Academic Senate, through resolutions, has called for increased attention to program review. The 1988 Research Committee of the Academic Senate developed a preliminary paper outlining the various models of program review and some of the related issues and concerns. The paper was forwarded as a resource to the Educational Policies Committee that has attempted to respond to annual resolutions calling for the development of a model program review process.

A few years and several drafts later this paper seeks to respond to the resolutions. The attempt to develop a "model" was abandoned as consideration was given to the varying characteristics of 106 local community colleges and 71 local community college districts. To that end, this paper focuses on various issues regarding program review. The method emphasized is the self-study validation method that is similar to the method used in the accrediting process. This paper, complete with recommendations throughout the body, is intended to serve as a reference for community college faculty and staff as they develop or revise their program review process.

Ideas and recommendations contained in this document represent a culmination of efforts to address program review. These efforts include, but are not limited to:

A. Seven breakouts at the Academic Senate Fall and Spring Plenary Sessions that included the participation of over 200 local academic senate presidents.

B. Three different draft documents, initiated and reviewed by three separate Educational Policies Committees.

C. Six presentations by Academic Senate representatives both separately and in conjunction with the Researchers professional association (RP Group).

D. Fourteen sample program review processes from large and small, urban and rural, diverse and homogeneous colleges.

E. Discussions with our CSU and UC colleagues on program review and implications for faculty.

F. A review of the limited amount of literature available on program review specific to community colleges.

G. A discussion with the Accrediting Commission for Community and Junior Colleges regarding the impact of external accountability processes on accrediting processes and related legislation.

Background
Community college faculty and administrative staff have evidenced varying degrees of interest in program review. As fiscal constraints placed upon colleges increased and as program review, or some similar process became a primary mechanism to determine program expansion, continuation, or termination, the schism of perspectives regarding the purpose and application of program review developed. Many departments across the state facing the prospect of program review manifested common features of "self-preservation" and "self-interest.' The perception that the program review process was a survival of the fittest contest, both divisive and punitive in nature, eclipsed some of the more noble but less prevailing perspectives that regarded program review as an opportunity to promote educational excellence and improve instruction and services to students.

Previous Resolutions
A glance at the adopted resolutions reveals that as early as 1987 and as recently as 1995, faculty have increasingly expressed an interest in program review. Concerns included the linkage of program elimination to program review, the relative collective bargaining implications in program review and elimination, the necessary criteria for program review, the need for information/workshops on program review, and the extent to which program review should include student, administrative, and ancillary services, as well as instructional programs, and the role of local academic senates in program review. (See Appendix 1)

Legislation Related to Program Review
Work Force Preparation
As the workforce preparation initiatives develop on the state and national level, several processes, models, and pieces of legislation addressing accountability have emerged. Senate Bill 645 (Johnston), the Report Card Bill, was recently signed by the Governor. This bill calls for the State Job Training Coordinating Council (SJTCC) , formerly the advisory body to the federal Job Training Partnership Act (JTPA) program, to develop performance-based indicators upon which funding decisions can be made. Of particular interest to California Community Colleges are the parts of the bill that indicate that certificated programs may be included. While the inclusion of certificated programs raises some very fundamental questions about the role of SJTCC in making educational policy and evaluating the community colleges, of greater significance is the fact that this legislation seeks to supersede previously legislated and regulated accountability processes, as well as traditionally respected educational processes such as accreditation. This legislation minimally should put faculty and college administrators on notice of the Legislature's and the public's interest in accountability.

Regulations Related to Program Review
In responding to the legislative mandate to develop processes to strengthen academic senates, the Board of Governors for California Community Colleges adopted Title 5 Article 2. Academic Senates. These regulations identify the definitions, formations, responsibilities and powers of the academic senates. These regulations require governing boards to "consult collegially" with academic senates on "academic and professional matters." Processes for program review are situated among the eleven items listed in "academic and professional matters."

According to the regulations, "Consult collegially"means that the district governing board shall develop policies on academic and professional matters through either or both of the following methods, according to its own discretion:

(1) Relying primarily upon the advice and judgement of the academic senate; or

(2) That the district governing board, or such representatives as it may designate, and the representatives of the academic senate shall have the obligation to reach mutual agreement by written resolution, regulation, or policy of the governing board effectuating such recommendations.

This regulation, which has the effect of law, means that academic senates must play a central role in the development of the program review processes. It is the professional responsibility of the local academic senate to either provide the primary advice or develop a senate position in order to have a basis upon which representatives can seek to come to mutual agreement with the board.

If the college does not have a process adopted by the board that was developed with the collegial consultation of the local academic senate, the local senate should immediately seek to initiate such a process. The regulations required the local boards to adopt policies for the appropriate delegation of authority and responsibility to its college and/or district academic senate. To that end, local senates should identify which process of collegial consultation the board has chosen for program review and begin the endeavor. If the local board does not have a delineation of authority policy, local senates should (1) bring the issue to their attention and seek to support the board in the development of the delineation of authority; (2) begin working on gathering information, seeking input from affected constituencies, and develop a senate position through senate processes on the issue of program review. The discussion contained in this section is not intended to suggest that faculty should wait for the boards to develop the delineation of authority policy before addressing program review.

Role of Local Academic Senates
The involvement of the local academic senate, as the representative of the faculty, is critical to the successful development and implementation of a program review process. While the role of the local academic senate is delineated in regulation, academic senates have a greater opportunity to create a collegial, supportive, student-centered, faculty-driven, academically relevant process for the evaluation of the college's programs and services. Local academic senates should see their role as an opportunity to redefine program review in order to eliminate unwanted or ineffective characteristics historically associated with the process. Administrations and boards need the professional expertise and judgment inherent in the collective wisdom of the faculty represented by the local academic senate. Through an organized resolution process or the development of a position paper, the local academic senate can affect one of the most important processes for determining how well the students' educational and support needs are being met on a college campus.

A Q&A document developed by Past President Jim Locke, and Vice President Bill Scroggins suggests standard language. With some modification, this language appears below. Based on local issues and discussions, revisions can be made. This resolution was developed with the intention of providing local academic senates a basis upon which they can build their program review discussions into a process.

"Whereas, faculty members derive their authority from their expertise as teachers and subject matter specialist and from their status as professionals and as a result, the faculty has an inherent professional responsibility in the development and implementation of policies and procedures governing the <insert college name> program review processes; therefore

Be it resolved that the <insert college name> program review process shall promote professionalism, enhance performance, and be effective in yielding a genuinely useful and substantive process for determining program effectiveness, and

Be it further resolved that the program review processes at the <insert college name> will require 1) an articulation of clear, relevant criteria upon which reviews will be based,- 2) the establishment of reasonable and timely intervals; 3) the establishment of the specific purposes for which program reviews are conducted and articulation of those purposes to everyone involved, and

Be it further resolved that the principal purposes of the review process are to recognize and acknowledge good performance, to enhance satisfactory performance and help programs which are performing satisfactorily further their own growth, and to identify weak performance and assist programs in achieving needed improvement, and

Be it further resolved that one of the purposes of the program review process at <insert college name> is not that of providing a mechanism or justification for program elimination; and

Be it further resolved that a program 's students, administrators, faculty and their colleagues shall all contribute to the program review, but the program's faculty shall play a central role in the program review process and, together with appropriate administrators, assume principal responsibility for the effectiveness of the process; and Be it further resolved that procedures of the program review process shall foster a joint and cooperative exercise of responsibility by the faculty, administration, and governing board of <insert college name> and shall reflect faculty and administrator expertise and authority in evaluating professional work as well as the governing board's legal and public responsibility for the process, and

Be it finally resolved that the < insert college name > program review process shall provide an ongoing and thorough review of the relevance and responsiveness of vocational education programs, consideration of the relationship between other similar programs throughout the state, and the appropriate balance between vocational and general educational programs.

Local academic senates will need to identify the process they will use, the participants time lines, and the needed resources/cost for the development of a program review process. It is recommended that the senate form a subcommittee that is responsible for the development of the program review process. Resource people, including students and staff, are valuable components of the committee. The senate should consider that either primary advice or mutual agreement does not exempt the senate from seeking the input from others affected by the process. Because a successful program review will depend upon an institutional effort by faculty, staff, and students, and because staff and students have a regulatory right to participate in the discussions of district policies that have a significant effect on them, senates that fail to see the value in the participation of other groups may face extreme difficulty in either coming to agreement or giving primary advice on an unjustifiable and illegal position that excludes the participation of all affected groups in the development of the processes. Sound judgement by the senate would exercise a process that is open and collegial in the development of program review processes. To that end, as the model is being developed, staff and students should be asked to contribute to the discussion in order that the senate may take the ideas presented under consideration as the process is developed.

The subcommittee should prepare for the local senate a recommended model (in the case of primary advice) or position (in the case of mutual-agreement) on program review. In order to ensure all interested parties an opportunity to comment or contribute to the discussion, the local senate should consider holding a hearing on the proposed process that will be considered for reading and action. Subsequently, the senate should subject the recommended process as appropriately modified by input from the hearings, and resource people, to the scrutiny of the senate resolution processes. The item should be handled under the strict scrutiny of the Brown Act with the proper notification, first reading, and second reading/action agenda items. The local senate should operate a fully public meeting, open to the public with provisions for public comment as designated by the local senate. If heavy attendance and participation is anticipated, local senates may want to adopt procedures for public comment similar to those used by local governing boards, thereby ensuring the opportunity for all to speak with a time limit. The adoption of such a procedure provides a record for the local senate that serves to indicate the opportunity of the campus community and the public to comment. Ultimately, the local academic senate should recognize its responsibility and right to recommend a process that represents the informed and collective wisdom of the faculty.

Purposes of Program Review
The development of the local program review process will be influenced by the purposes, assumptions, and philosophy under which the program review process is being developed. A review of the literature reveals a common assertion that in order for the program review process to be successful it should serve as a mechanism for the assessment of performance that recognizes and acknowledges good performance and academic excellence, improves the quality of instruction and services, updates programs and services, and fosters self-renewal and self-study. Further, it should provide for the identification of weak performance and assist programs in achieving needed improvement. Finally, program review should be seen as a component of campus planning that will not only lead to better utilization of existing resources, but also lead to increased quality of instruction and service. A major function of program review should be to monitor and pursue the congruence between the goals and priorities of the college and the actual practices in the program or service.

Other purposes include meeting accountability mandates and demonstrating a systematic planning process. While these functions may occur as a result of program review, they should not serve as a primary function of the review. Otherwise, program review could be perceived as a time-consuming process for a product with no effect or bearing on the program's activities or any relevance to serving students or maintaining academic excellence.

Finally, some program review processes have also served the purpose of justifying program and/or personnel elimination. The process by which programs and services are reduced or eliminated should be clearly defined, as well as be separate and distinct from program review. Additionally, individual faculty and staff evaluations should remain a separate and distinct entity from the program review process, following peer review procedures developed by the collective bargaining agents, where they exist, in consultation with the academic senate.

Linkages
Many processes and activities of a college could and should be correlated or coordinated with the program review process. The probability of decreasing the duplication of effort and effectively managing time, human, research, and fiscal resources is maximized with such coordination.

Educational Planning
Community college educational planning requires that the college examine what it does and how what it does compares to a forecast of what is needed in the future. Program review is thus one of the two essential components of planning. Short-term adjustments in resource allocations frequently are made using immediate enrollment data; we see enrollments decreasing in course A and wait lists growing in course B so we offer fewer of A and more of B. But planning asks why these changes are occurring, whether the condition can be changed by the college and the faculty, and what is to be done.

Program review contributes to the search for the answers for these questions. It does so by evaluating program quality and by weighing enrollment and outcome information against the context in which the college exists (the role of the program in transfer patterns, or in preparation for the job market.) In the context of educational planning, scarce resources and growing student populations compel us to focus on difficult questions; which program or service deserves additional staff, equipment, or supply budgets? Which programs or services are organized in such a way that they must be reconstructed to more effectively meet student needs? Which programs and services, as determined through an appropriate process, are not needed and by their presence, are preventing the development of needed programs and services?

While there is a tendency to be reticent to pose some of these questions, faculty should and must become involved in the posing and asking such questions. The colleges can HI afford to be lacking in the knowledge and expertise of the faculty on these important planning issues.

Accreditation Process
"Accreditation provides assurance of the institutional integrity, quality, and effectiveness .... More importantly, accreditation is the system by which the internal community of an institution evaluates itself and plans for improvement in quality and effectiveness." The purposes of program review and accreditation may be similar, notwithstanding the general nature of accreditation and the specific nature of program review. However, a constitutional difference between accreditation and program review is the locus of control over the standards and the validation process. In the accrediting processes, the Commission defines the standards; in the colleges program review process, the college defines the standards.

The faculty role and involvement in accreditation processes, including self-study and annual reports, is one of the areas on which local academic senates must be collegially consulted. As local academic senates plan program review processes, many of the general principles and processes can be transferred from the accreditation model and modified to more specifically address the program under review. If time lines are coordinated with institutional accrediting processes, programs can take advantage of the overall climate in the institution reflectively focused on self-study and-improvement. This positive climate alone could and would be a sharp contrast with the negative climate many faculty have historically associated with program review.

Finally, as the institution develops goals in response to the accreditation process, program improvement or support could be incorporated in or coordinated with the developed goals. The institutional goals will be placed in an overall and legitimate context that faculty are likely to support. This type of comprehensive acknowledgment of the goals lays a strong foundation for the second cycle of program review whereby the goals are seen as legitimate and concerns regarding the legitimacy of the planning process become irrelevant.

Budgetary Processes
A multitude of budget and governance discussions have yielded a common axiom, "Planning should drive the budget; the budget shouldn't drive the planning." This statement in the context of many California Community Colleges is usually made as college leaders announce the fiscal constraints and the need to eliminate courses, programs, and/or services. Despite the mission of the college, the master planning (usually done at five-year intervals), the goals of the college, and the objectives of the departments, the decrease in funds, by default, call for reprioritizing and modification of the products of any systematic planning.

The "planning-by-budget" phenomenon may be exacerbated by fiscal constraints imposed by the state, combined with the reality of college political processes in which contracts are negotiated, projects are pursued, and efforts to maintain a positive image in the community are made. At times, a relatively sophisticated institution, with complex planning processes, stable funding, and the best of intentions, may be faced with having to make cuts due to factors beyond the control of the college. In this context, the establishment of legitimate goals and the coordination of program review with the budgetary process is essential.

As with program review, the local academic senate has responsibility for collegial consultation on the budget process. In fight of the similar roles of the local academic senate on program review and the development of the budget process, a great opportunity exists to coordinate the two processes; recommendations developed through program review can be considered when resources are prioritized. This coordination alone stands to ensure daily activities actually are affected by the recommendations developed in the program review process.

Curriculum Process
The curriculum process is yet another area of primary responsibility for the academic senate. Curriculum processes are known to be time consuming. A central argument in support of such a commitment is that it allows for the deliberation and scrutiny necessary to ensure the academic integrity of the curriculum. This integrity is respected within and outside of the college. Business and industry, transferring institutions, community members, students, and the college itself, all benefit from the academic integrity that results from educationally sound, deliberate, curriculum processes. Whether the issue is the determination of an overall curriculum model or the approval of a particular course, the local academic senate plays a primary role.

If a product of the program review process I is a need to enhance, modify, or radically change the curriculum, the coordination of the program review process with the curriculum process facilitates a more successful approach to meeting student and program needs. Careful attention should be given to programs that may have to respond to outside agency standards, curriculum frameworks, or legislative acts. Such programs may include some of the certificate-granting and/or licensing vocational education programs.

Student Equity
The commonly used quantitative program review measures are consistent with the indicators recommended in the student equity regulations. Program review provides the opportunity for faculty to examine whether all students and the entire community are being successfully served; the goal of more equitably serving students can thus be furthered. For example, a math department may identify that it has a 75% student success rate. If further analysis reveals that of the 25% who were unsuccessful, a disproportionate number were women, perhaps the math faculty would want to examine whether their pedagogy and curriculum are linked to these inequitable outcomes.

Local academic senates are also to be collegially consulted on standards and policies regarding student preparation and success. Because student equity is an issue of student success, program review processes should also be coordinated with student equity processes.

Regional Academic Planning
As colleges and universities experienced the elimination of courses and programs, the Intersegmental Committee of the Academic Senates (ICAS), that five representatives, often the ranking officers of the state senates of the University of California, the California State University, and the California Community Colleges, began discussions on regional academic planning. The concern centered around the elimination of programs without the coordination necessary to ascertain the impact on the geographical regions of the state. Consequently, the probability of eliminating advanced foreign language courses or baccalaureate level nursing programs in the entire southern or northern region of the state could be minimized. Instead, course and program elimination could be made in an informed manner that would include the regional impact.

The 1993 Budget Act states in part:
"Long-term Planning for Program Course Offerings in Higher Education. It is the intent of the Legislature that the UC, California State University (CSU), and, to the extent possible, the California Community Colleges (CCC) consult with each other on a regional basis as plans for campus budget reductions are developed, in order to ensure that particular geographic regions continue to offer an adequate balance of academic program offerings and courses...."

While this concern is primarily aimed at academic program offerings and courses, this language should also provoke community colleges to identify the inherent imperative of the language as it relates to the mission for California Community Colleges. The Master Plan's Mission includes the offering of general education/transfer, vocational education, and basic skills. To that end, program/course elimination, irrespective of regional considerations, and out of the context of UC and CSU regional planning, could have educationally and economically devastating results for the individual seeking an education for transfer or employment, for the business and industry in the community, and for the community itself.

Although the recommendation herein assumes separation of course elimination processes from program review, the impact of program goals developed without the regional planning should be considered. The perceived standards of success could be strengthened, legitimate goals of the college could be better pursued, and college and program relationships with the community could be supported by the coordination of regional academic planning and the program review process.

Considerations When Developing A Local Model
Essential to the development and implementation of a program review process are the underlying assumptions upon which it is based. These assumptions should be identified and delineated in the program review process. Some of the more common issues and assumptions are discussed in this section in an attempt to provoke greater deliberation at the local level.

Goals
Essentially every piece of literature and program review process referenced identified the need to link program review with the district mission, the college master plan, and the department goals and objectives. Inherent in this assumption is that the mission, master plan, goals and objectives are legitimate to the faculty. This legitimacy would depend upon the process by which the district, college, and departments arrived at such missions, plans, goals and objectives. As with program review and other academic and professional matters, institutional planning is one of the eleven items on which local academic senates must be collegially consulted. If the program review process was one in which faculty were consulted collegially, thereby ensuring a respect for and ownership of the goals by the faculty, the assumption could serve the process well. If, however, the faculty were not consulted collegially, this linkage to program review should not be used until legitimate missions, master plans, goals and objectives are developed.

Application of Program Review Process
The question of which programs and services should be reviewed can be addressed in the college's program review assumptions. If the purpose of the process is to promote educational excellence and better serve the students, no function of the college should be exempt. The effectiveness of the ancillary units and administration of the academic divisions, instruction, student services, the executive offices, and the governing board all have an impact on the academic integrity of the programs and on the college's ability to serve the students. As local academic senates address this issue, it may be necessary to define the term "program" or answer the question of whether the process is an instructional/services program review process or an institutional program review process. An institutional review process would include the review of all programs and services, including units not commonly perceived as programs. This perspective is also consistent with defining programs as they are defined in the college budget processes (cost centers) and in the planning and goal setting process (planning units).

Institutional Support
A fundamental assumption necessary for the successful development and implementation of a program review process is the commitment of the institution to provide institutional resources. Irrespective of the model used, faculty leadership in self-study and review will require institutional support for time, materials, and staff. Historical models that designated administrative deans to assume a leadership role may not have had to address the issue raised in this assumption because most administrative deans have budgets, materials, and clerical staff support. If the institution is willing to support program review while it is under the leadership of an administrative dean, its inability to do so for processes under faculty leadership is unjustifiable. A delineated budget should be available for program review. Faculty should resist any attempt to sacrifice the integrity of the program review process in order to economize.

Another type of institutional support necessary for the successful implementation of a program review process is that of research support. Depending upon the program review process developed, both qualitative and quantitative research may be necessary. Faculty scheduled for program review should influence the research agenda. For small colleges without research facilities, faculty should have access to whatever staff person(s) serve the research function of the college. In commenting on program review, George Boggs, President of Palomar Community College and then Chair of the Commission on Research for the California Association of Community Colleges stated, "One of the most basic and important types of institutional research for a community college to do is program evaluation. In its most basic form, an evaluation is a study, based upon objectives of the activity or program to be assessed, that provides useful information about the degree to which those objectives have been met. Program evaluation is simply a matter of asking useful questions, and then making valid conclusions based upon the data. The current emphasis on documenting student outcomes is, intact, a call to conduct more and better program evaluation."

Fairness and Credibility
In developing local program review processes, local academic senates will have the responsibility to develop a process that is perceived to be fair and credible. The credibility of the evaluators and the review should be established. Failure to do so could decrease the propensity for faculty to buy into the process, invest their time, utilize it to its fullest potential, or pursue endeavors arising from the product of the process.

There must be trust that the purposes and goals of the process are truly dedicated to supporting and nurturing departments instead of destroying them, reducing the faculty, or pursuing a cheaper approach to delivering instruction or services. Indications of this need might be manifested in such questions as:

1. Is confidentiality an issue?

2. Who will review the data and reports? For what purpose will they be used?

3. Are there any provisions for rebuttal or minority reports?

The local academic senates will need to ensure that these types of concerns are adequately addressed in the development and the implementation of the program review process.

Finally, the administration of the college must believe in and respect the process of program review. Administration must be willing to help enact the changes that are recommended as a result of the process. If fiscal or other restraints preclude some changes, they should be presented at the onset and perceived to be true by the faculty, thereby further defining for future reference the range of options available to the faculty. Nonetheless, in performing a program review, faculty should not hesitate to identify the ideals that would improve or support the programs or services; however, the identification of constraints can assist faculty in preparing realistic recommendations that should affect their day to day operations.

Identify Criteria and Determine Research Needs
A universal set of criteria for program review is not practical given the diverse characteristics of our local colleges. Our colleges are "community" colleges, specifically designed to meet the educational needs of the community. As communities drastically differ across the state, so do their community colleges. As local academic senates develop the program review processes, the research needs and methodologies must be considered. Because most program review process have both qualitative and quantitative components, these methodologies should not be considered mutually exclusive of each other.

Qualitative
Program review presents an opportunity for discussion of educational philosophy and the contributions of each program to the development of the whole student. General education is usually dedicated to some notion that in addition to technical skills and competencies, education should be concerned with contributing to the development of generally educated persons. Difficult to measure abilities such as the capacity for ethical reasoning and critical thinking skills are central to preparing students for life-long learning and effective citizenship. Such qualitative outcomes are not easy to quantify but are reflected in the spirit that animates classroom discussion, the values that are modeled in pedagogy, and the habits of mind of graduates. It is not uncommon for program review processes to leave out the examination of such qualitative dimensions.

Explicit attention should be given to qualitative aspects of programs. If the focus of the process is qualitative, less statistical and more value-laden assessments will be made. Faculty will assess areas that do not lend themselves to objective evaluation, but nonetheless, are legitimate and important areas for consideration. Examples include assessments of the learning environment, including facilities available, financial resources, instructional equipment, and the impact of class size on student success and the ability of the program to meet the students' needs. The quality of the breadth of the curriculum and the teaching and learning process should be taken into consideration. The student satisfaction and the quality of the program as perceived by them and the quality of the program as perceived by the articulating universities, or employing businesses and industry. is a crucial factor in evaluating the program for effectiveness in serving the students and community. Local academic senates should further develop the possible qualitative factors to be assessed, as this area represents a small portion of the possibilities.

Quantitative
Recommendations of explicit attention to qualitative factors is not meant to imply that faculty should reject the utility of quantitative data in the program review process. The quantitative factors, too, have value in the context of a comprehensive program review process. Items such as access, enrollments, persistence, transfers, graduation, successful course completion, grades, and other student outcome factors draw a numerical picture for further analysis. However, preoccupation with quantitative measures, particularly productivity, will have the effect of directing the program review processes rather than concerns over educational soundness or student success. Such preoccupation could lead to the failure to consider important factors such as (1) the comprehensiveness of the college model and how the curriculum model supports it, (2)the educational philosophy or commitment and how the various program contributions translate into that philosophy and commitment, (3) the educational needs of the community and how the programs support those needs.

It is important to emphasize that quantitative data should not be presented without a narrative explanation nor should it be used for comparison among a college's programs. Such use would tend to promote the use of program review for program reduction or program elimination and would completely disregard the qualitative value of a program. Rather, trends in data over a period of time within the program itself may be most useful for the program faculty to identify their needs and design the necessary intervention or support.

Self-Study Validation
Consistent with the assumptions and/or purposes, self-study is an opportunity for self-renewal. The security of understanding the process will result in support for the program whether there are strengths or weaknesses. Self-study allows for the people with the greatest level of expertise in a particular program, the faculty, to examine and scrutinize the program for effectiveness in serving students and educational excellence.

The program review process should include the specific constitution and function of the self-study process, including but not limited to:

A. Will it be a committee? If so, what if the program only has one or two faculty? What will be the duties of the team members?

B. Will it involve full-time faculty, part-time faculty? What if a program has no full-time faculty?

C. The senate should identify its process for naming the faculty members. Will it be by appointment, or ratification of the departments decision?

D. How will other units of the college be included? How is the decision to be made?

E. What is the local administrator's role? Is it as part of the study team or the validation team?

F. Who is responsible for writing the report? What should be included?

G. Once the data is interpreted and the report is drafted, how will the strategies for specific actions be developed and incorporated in the report? What should be the implementation time lines?

H. What is the time for commencement and completion of the process?

I. How will the data be gathered? What will be the methodology?

J. What are the components of the validation process, review of documents, program visits, review of data/survey results?

K. What happens after the validation stage? What is the flow of the paper? What happens with the results?

L. Are there any further meetings, discussions, conferences, before the final report is issued? Who will present it to the governing board and the academic senate? How will the report be used?

M. How is the necessary support given to the program, once the review is complete, the results are finalized, and the recommendations are made?

N. What is the comprehensive time line for each program, service, or unit to be reviewed?

While this is not an exhaustive list, local academic senates can see the attention to detail called for in developing the process.

Models
In 1988, the Ad-Hoc Research Subcommittee of the Educational Policies Committee of the Academic Senate developed a paper identifying the various models available in the literature concerning data gathering, program review models, and accountability in education.The models included:

The Research-Numerical Methods Model - This model has a quantitative orientation that concentrates on the collection of hard data. This method or aspects of this method may provide practitioners with information regarding why a specific trend has appeared or whether an effect has a cause.

The Peer Evaluation-Professional Judgement Model - This model is similar to a "mini-accreditation" process and involves personal observations, interviews, and the review of documentation. The delineation of a program's or service's relative worth is derived from the informed opinion of the visiting expert.

The Improvement - Progress Toward Goals Models- Citing Don Gardner's September 1977 article in the Journal of higher Education, "Evaluation models based on this definition assume that the most important decisions regarding the thing to be evaluated are contingent on its objectives and the criteria established for judging ... [and] the relative attainment of those goals..."

The Issue Oriented Model - This approach is a positive and humanistic qualitatively centered process that treats problems, solutions, successes, and consequences as having the same status and importance.

The issues emphasized in this document could easily be applied to a program review process that includes some aspects of each of the four models.

Conclusion
Program review is an essential component of serving the community's educational, training, and basic skills needs. There are many models, as well as factors to be considered in the development and implementation of program review. A deliberative well planned process that is faculty driven and respected throughout the college can and will result in meaningful evaluation from which vital information can be derived for the maintenance of the integrity of the college community and its educational programs.

Appendix - Related Resolutions
S94 8.5 Be it resolved that the Academic Senate for California Community Colleges direct the Executive Committee to collect data identifying programs eliminated by individual colleges and the reasons for elimination, and

Be it further resolved that the Academic Senate for California Community Colleges direct the Educational Policies Committee to research program discontinuance policies and procedures, and

Be it finally resolved that the Academic Senate for California Community Colleges direct the Educational Policies Committee to develop a position paper with recommendations concerning implementing local program discontinuance policies.

S93 5.1 7 Be it resolved that the Academic Senate for California Community Colleges recommend to local senates that the collective bargaining units be consulted in program evaluation procedures and in program reduction decisions

S93 6.2 Be it resolved that the Academic Senate for California Community Colleges develop a model for program review including criteria which balance quality and economic feasibility.

S92 61 Be it resolved that the Academic Senate for California Community Colleges direct the Educational Policies Committee to develop a model program review policy for consideration at a future session, and

Be it further resolved that the following issues, among others, be considered for inclusion within that model:

Class size and its effect on instructional quality;

b. The appropriate mix among transfer, vocational, basic skills, and noncredit courses;

c. Considerations of the quality of instruction as well as issues of productivity;

d. Facilities considerations;

e. Connecting the outcomes of program review with the college and district budgets; and

f. Contribution to the community based on student success in job placement.

F88 5.3 Be it resolved that the Academic Senate for California Community Colleges recommend to local senates that libraries be included in all college wide program reviews.

S88 4.5 Be it resolved that the Academic Senate for California Community Colleges requests that the Chancellor's Office organize workshops on the program review process.

F8714.1 Be it resolved that the Academic Senate for California Community Colleges requests that the Chancellor's Office notify the LOCAL academic senate in a timely fashion regarding the impending visitation of the program review teams for programs such as Equal Opportunity Programs and Services and Disabled Students' Programs and Services, and be it further resolved that the Academic Senate for CCC recommend that during these visitations the local academic senate be actively involved in the interview process for the purpose of evaluation of these programs.

References:
1. Barclay California code of Regulations, Article 2 Sections 53203(a). Register 93. No. 6;2-5-93. Page 325

2. Barclay California code of Regulations, Article 2 Sections 53200(c). Register 93. No. 6;2-5-93. Page 325

3. Barclay California code of Regulations, Article 2 Sections 53200(d). Register 93. No. 6;2-5-93. Page 325

4. Barclay California code of Regulations, Article 2 Sections 53203(a). Register 93. No. 6;2-5-93. Page 325

5. Scroggins, Bill and Locke, Jim "Questions Faculty Ask, Answers and Suggestions on the Faculty Role in College Program Review Policy and Procedures." 1994 page 1

6. Accrediting Commission for Community and Junior Colleges, Guide to Institutional Self Study and Reports to the Commission. 1990, page 1

7. Barclay California code of Regulations, Article 2 Sections 53200(c)(7). Register 93. No. 6;2-5-93. Page 325

8. Barclay California code of Regulations, Article 2 Sections 53200(c)(1). Register 93. No. 6;2-5-93. Page 325

9. California Code of Regulation, Subchapter 4, Section 54200, Student Equity Plans.

10. Barclay California code of Regulations, Article 2 Sections 53200(c)(5). Register 93. No. 6;2-5-93. Page 325

11. ICAS committee minutes, December 13, 1993

12. Budget Act, Item 6420-001-001, "Supplemental Report to the Committee on Conference on the 1993-1994 Budget Act

13. Barclay California code of Regulations, Article 2 Sections 53200(c)(10). Register 93. No. 6;2-5-93. Page 325

14. Boggs, George, Leadership Abstracts, "The Research Function of Community Colleges," League for Innovation in the Community College, Laguna Hills, CA Vol. 1 No. 13 August 1988, page 1

15. Ad Hoc Research Subcommittee of the Educational Policies Committee of The Academic Senate for California Community Colleges, "Toward an Acceptable Program and Services Review," 1988, page 2

16. "Toward an Acceptable Program and Services Review," 1988, page 2

17. "Toward an Acceptable Program and Services Review," 1988, page 2

18. "Toward an Acceptable Program and Services Review," 1988, page 3

19. "Toward an Acceptable Program and Services Review," 1988, page 2

20. "Toward an Acceptable Program and Services Review," 1988, page 4

21. "Toward an Acceptable Program and Services Review," 1988, page 5