Strategic Plan For Assessing Student Learning Outcomes And Institutional Effectiveness At University Of Maine At Fort Kent

2003-2008

as a precursor to the NEASC Self-Study process and for good governance

Prepared by:

Robert F. Scott, Ph.D
Vice President for Academic Affairs

November 27, 2003

Table of Contents

Forward: Systematized Assessment and Institutional Effectiveness Plan Grid 2003-2000(see also in Section 4)

• Section 1:Assessment Narrative
• Section 2:Data Collection Responsibility Chart
• Section 3:Brief Narrative of Eight Classification Sections
• Section 4:Grids and Reporting Instruments

Systematized Assessment Plan

Strategic Plan For Assessing Student Learning Outcomes And Institutional Effectiveness At University Of Maine At Fort Kent

FALL 2003

"There is a significant body of evidence which dictates that assessment drives student learning. More than anything else, our assessment tools tell students what we consider to be important. They will learn what we guide them to learn through assessment".

-FLAG Primer

As we proceed into the 21st century, traditional methods of assessing student learning are continually being challenged as inadequate and inappropriate measures of true student learning. Federal, state and local governments, accrediting agencies, educational institutions, teachers unions and the like have all been driving the effort to develop measures and techniques which will more adequately display the students learned ability to think analytically, communicate effectively, and develop skills consistent with the needs and demands of a 21st century workplace and society. In simpler terms, assessment should be more than grades received in a course. The quality of learning taking place is becoming more of a concern.

This concern is generated by a need for responsible institutions of higher learning to continuously improve, and by changing standards of regional and national accreditation agencies and boards. In the spirit of this transformation, many institutions have recognized the synergy between student outcomes assessment, program assessment and overall institutional effectiveness efforts. Much of the literature suggests that "one cannot go without the other." Therefore, quality, outcomes-driven assessment at all levels of the institution is fundamental to improved student learning and institutional health.

The NCA (North Central Association) Commission on Institutions of Higher Education Ten Identified Characteristics of an Effective Program to Assess Student Academic Achievement

• Successful assessment
• Flows from the institution's mission and educational purposes
• Emerges from a conceptual framework
• Is marked by faculty ownership and responsibility
• Has institution-wide support
• Relies on multiple measures
• Provides feedback to students and the institution
• Is cost-effective
• Does not restrict or inhibit goals of access, equity, and diversity established by the institution
• Leads to improvement
• Includes a process for evaluating the assessment program
• The American Association of Higher Education (AAHE) Defined Nine Principles of Good Practice for Outcomes Assessment
• The assessment of student learning begins with educational values
• Assessment is most effective when it reflects an understanding of learning as multidimensional, integrated, and revealed in performance over time
• Assessment works best when the programs it seeks to improve have clear, explicitly stated purposes
• Assessment requires attention to outcomes but also and equally the experiences that lead to those outcomes
• Assessment works best when it is ongoing, not episodic
• Assessment fosters wider improvement when representatives from across the educational community are involved
• Assessment makes a difference when it begins with issues of use and illuminates questions that people really care about
• Assessment is most likely to lead to improvement when it is part of a larger set of conditions that promote change
• Through assessment, educators meet responsibilities to students and to the public.

Southeast Missouri State University's nationally recognized assessment program is founded on fifteen principles. These principles expand upon a statement of student assessment principles first distributed by the Director of Assessment in 1992 and reflect the collective judgment of their Assessment Review Committee. Together they constitute Southeast's philosophy of assessment. This philosophy guides evaluation and revision of the University's assessment plan:

• Assessment should be a systematic, ongoing process that involves gathering, interpreting, and using information for continuous improvement
• Assessment should focus on specific programs and activities that contribute to the intellectual, professional, personal and cultural needs of students
• Assessment should be shaped and guided by faculty, students and staff, with administration and administrative processes providing essential support
• Assessment should flow from the institutional mission, and the institution's mission should be shaped by the results of assessment when appropriate
• Assessment outcomes should be used in planning, budgeting, and allocating resources
• Flexibility in the choice of assessment procedures should be encouraged, permitting the exercise of professional judgment as to the appropriate methods of assessment
• Assessment should be based on multiple measures both quantitative and qualitative, including, for example, locally developed instruments, surveys, nationally normed exams, external reviews, exit interviews, historical data, and evaluation of performances
• The use of assessment results should determine the choice of assessment procedures
• Assessment should be cost-effective
• Assessment procedures should be regularly evaluated as to their usefulness for fostering continuous quality improvement
• While assessment for accountability may be necessary, it should be integrated as far as possible into assessment for improvement
• Assessment should be minimally intrusive on faculty, students and staff
• Assessment plans and activities should be continuously evaluated band nimproved through peer review and discussion
• There should be regular comprehensive reviews of the assessment plan
• Assessment and the use of assessment results should not unfairly restrict institutional goals of diversity and access
• Genesis of a Systematized Assessment and Institutional Effectiveness Plan at University of Maine at Fort Kent

In 1995, NEASC (North Eastern Association of Schools and Colleges), the institutionals governing accreditation body, mandated that UMFK respond to the call heard throughout higher education for the assessment of student learning outcomes and student academic achievement as part of a systematized institutional effectiveness effort. In addition to the assessment problems created by the lack of any systematized plan or institutional effectiveness model, UMFK faced additional concerns. These included the lack of a specifically outcomes-driven focused assessment committee or similar governance mechanism for administration of a systematized assessment plan, an arrangement for review and dissemination of the data from assessment initiatives for improvement of the learning environment, and lack of involvement in measuring institutional effectiveness from a campus-wide perspective. And, following a period of institutional instability in several key leadership positions, UMFK is embarking on the implementation of a 21st Century assessment and institutional effectiveness model.

Assessment at the University of Maine at Fort Kent should further the basic goals of the institution, that is, it ought to gauge progress toward achieving the institution's mission of providing a high quality undergraduate liberal arts education and establishing a strong foundation for graduate school and employment opportunities for its students. Among the purposes for which assessment data should be collected are: self-improvement, accreditation, resource allocation, and public dissemination.

Assessment should be first and foremost a self-improvement activity that enhances the educational development of students and improves the performance of faculty and staff at achieving the former (Astin, 1993). Such self-improvement efforts at UMFK should focus on five primary areas. The first of these is self-improvement in the area of quality pedagogy and the development and improvement of teaching techniques. A second component of student educational development is exposing students to the latest knowledge in a given field, which requires a faculty that is engaged in scholarly pursuits within their disciplines. A third component is excellence in advising that focuses on academic guidance of a student during their undergraduate education, guidance that enhances the student's potential for future success, and guidance that enriches a student's development as a person. A fourth component is the integration of computer and telecommunications technology within the educational environment and the work place. A fifth component is the continued improvement of a quality infrastructure for our learning environment.

Accreditation is a secondary purpose of university assessment efforts. Institutional accreditation as well as degree and program accreditation requires adequate data collection to meet the demands of external accrediting bodies.

Resource allocation is yet another secondary purpose of assessment. Assessment information is and should be used as a basis for resource allocation in situations where, for example, demand exceeds supply, resources are becoming antiquated, when deciding how best to redistribute resources that are not being used efficiently, and in order to enhance competition for dollars from outside funding sources.

Finally, assessment data is critical for public dissemination purposes. Prospective students, parents and alumni should have a clear picture of the institution and its quality. Assessment data enhances our relationships with and provides important comparisons to peer institutions. The Trustees, Board of Visitors, community and state of Maine tax payers should be able to assess the quality of UMFK, its students, faculty, staff, facilities, and programming.

Assessment and Institutional Effectiveness at the University of Maine at Fort Kent from Fall 2003 Forward

Driven by the Office of Vice President for Academic Affairs, the institution has established 2003 as a benchmark year for change and evolution with the revision of its strategic plan and development and implementation of the assessment and institutional effectiveness model.

This effort is undertaken for two reasons: (1) To satisfy mandated, structural requirements for continued accreditation through NEASC, and; (2) Because it is the responsible thing to do in an era of continuous improvement and institutional responsibility.

University of Maine at Fort Kent Principles for Assessment of Learning Outcomes and Institutional Effectiveness

The faculty will be responsible for setting and assessing educational goals.

The focus of assessment is improvement of the educational process and outcomes

Assessment activities will be conducted as unobtrusively as possible. Existing points of contact will be used extensively.

Triangulation of assessment activities

Focus on relevant KPI's (key performance indicators)

Results are to be used as part of a feedback loop and implementation process.

The Office of Vice President for Academic Affairs has developed An eight-spoked plan to assess performance outcomes and insure institutional effectiveness, and facilitate the effective maintenance and use of the results of this process. This plan manifests itself into eight broad categories which will be systematically tracked annually through a goal setting and results recognition process:

• (1) a chart outlining the various milestones, implementation timelines and feedback loops (see 1 in section 4),
• (2) an institutionally based performance scorecard utilizing key performance indicators (see 2 in section 4),
• (3) key performance indicators relative to the strategic plan (see 3 in section 4),
• (4) a series of unit level assessment and effectiveness initiatives (see 4 in section 4),
• (5) use of established externally-delivered assessment tools for comparison and benchmarking purposes
• (see 5 section 4),
• (6) process-specific assessment and effectiveness measures (see 6 in section 4),
• (7) define and/or revitalize the Strategic Planning Committee, NEASC standard workgroups, the University Assessment Committee and a Council for Institutional Effectiveness as permanent mechanisms to drive assessment and institutional effectiveness (see 7 in section 4),
• (8) creation and maintenance of an institutional data warehouse.

To be truly effective, and of maximum value, the widest possible institutional and individual involvement in all processes must be strived for.

Section 2: Data Collection Responsibility Chart

Brief Narrative Of Eight Classification Sections Section III

Section 1: Systematized Assessment and Institutional Improvement Plan 2003-2008

The Systematized Assessment and Institutional Improvement Plan grid is a snapshot view of the plan being implemented from 2003 until a comprehensive review is conducted in 2008, regarding the assessment of student learning and other outcomes and the broader area of institutional effectiveness at UMFK. The grid depicts the nature of the instrument, the timeframe for implementation, the organizational level of the implementation, and a timeline for implementation.

Section 2: Institutional Performance Scorecard

The Institutional Performance Scorecard grid serves the purpose of holding the institution accountable for its efficient operation. It also provides a framework for focusing on continuous improvement, through the annual goal setting and results review exercise. The UMFK performance scorecard utilizes five (5) KPI's (Key Performance Indicators) for each of the articulated NEASC (North Eastern Association of Schools and Colleges) standards and sub-standards.

This document will serve as the backbone of much of the NEASC Self-Study which will be prepared in preparation for the 2005 accreditation visit. Functionally, the KPI's will indicate that the institution focuses attention on all eleven (11) articulated standards. More pragmatically, it will give the standard draft writing teams an assessment context to base much of their work on in an organized and "state of the art" fashion.

Section 3: Key Performance Indicators for the UMFK 2003-2008 Strategic Plan

The Key Performance Indicator grid for the 2003-2008 Strategic Plan is, similar to the Institutional Performance Scorecard, an accountability mechanism. Rather than tying KPI's to NEASC standards and sub-standards, this document ties its indicators to the ten (10) institutional priorities articulated by Strategic Planning Committee. In a very basic sense, this document will show us if we are doing what the University said it should be doing.

Section 4: Unit level assessment and effectiveness efforts

The Assessment and Survey Inventory Baseline is a document compiled by the Vice President for Academic Affairs. It establishes a baseline of all assessment activities on the UMFK campus as of the fall 2003 semester. This document will serve as a prominent appendix in the NEASC Self-Study, as an indication of "where we were (2003)" as compared to "where we are now (2005)". It will be an integral piece of the puzzle depicting the fact that this campus takes assessment and institutional effectiveness seriously.

The Academic Unit Assessment Effort is an initiative to "shore up" deficiencies in academic unit assessment and institutional effectiveness efforts. Each academic unit (depending on how each division defines this term) must decide on a minimum of two (2) assessment techniques which are appropriate to their particular experience (related to student learning outcomes), and implement them as soon as possible. As is true of the institutional measures, viewing the process as a goal setting-results reviewing procedure, with a clearly defined feedback loop is critical.

Professional unit goal and KPI setting is an exercise whereby professional units in Academic Affairs establish their own set of key performance indicators (3-5 ideally), for purposes similar to those previously described efforts, on a more micro level.

Academic unit goal and KPI setting is an exercise whereby academic units (as defined by the respective divisions) and the divisions themselves will establish their own set of key performance indicators as a measure of health and continuous improvement on a micro level.

The Learning Strategies Project is an initiative being mandated by a number of accrediting agencies as a form of "learning strategies diagnostic." Rather than serve as a KPI or other assessment initiative, it serves as a mechanism to highlight the types of techniques used to transfer knowledge in the classroom. Accrediting agencies like to know that faculty are utilizing varied and contemporary techniques.

The Continuous Quality Improvement Log will be a tangible documentation of efforts to, and examples of, structural and process improvements in the various units of the institution. This log will be maintained by the Vice President for Academic Affairs, and will be the subject of monthly updating. Again, this is a long overdue pre-cursor to the writing of the 2005 NEASC Self-Study.

Section 5: Use of externally delivered assessment tools for comparison and benchmarking purposes

National Survey for Student Engagement (NSSE) is a comprehensive questionnaire administered to freshman and graduating seniors. The questionnaire gauges an institution in a number of areas which the research and literature show are critical to student success (quality academic advising, rigor in the classroom, level of interaction between faculty and students outside the classroom, use of more rigorous class assignments (ie- comprehensive essays) versus simple testing or short papers) and so on. The final report will give UMFK a baseline of "where it stands" with respect to the indicators. From this, we can identify strengths and weaknesses within our operation. The report also provides valuable comparative data (peers) and national norms from which to benchmark. The survey is administered by Indiana University annually to over 400 institutions of higher education.

Higher Education Research Institute (HERI) at UCLA conducts a bi-annual survey of faculty satisfaction. This comprehensive survey gets to the heart of what faculty believe are the strengths and weaknesses of the institution. Again, as is true of the NSSE, the final report provides valuable benchmarking and comparative data.

Section 6: Process-specific assessment and effectiveness measures

An academic advising internal assessment instrument will be developed as part of the charge of the Academic Advising Task Force presently convened. It will serve a triangulation function when combined with other assessments in use, and the NSSE. Accrediting agencies demand this type of internal assessment of academic advising.

A General Education internal assessment instrument is similarly being developed by the General Education Task Force. Even more so than with academic advising, NEASC demands quality, systematic assessment of General Education components, and student learning outcomes related to the component and component parts.

Section 7: Define and/or revitalize committee/task force structures related to assessment And institutional effectiveness

The University Assessment Committee will be reconfigured as one of the two or three most important committees on campus in February 2004. It will serve as the feedback loop for much of the systematized assessment and institutional effectiveness plan.

Council for Institutional Effectiveness To be determined. Still in a very abstract stage, and I continue to ponder its necessity at UMFK, especially if the University Assessment Committee functions well.

The Strategic Planning Committee, even after the 2003-2008 Strategic Plan is completed, will continue to serve. Individual workgroups will tend to the management of information related to the Key Performance Indicator grid.

NEASC Standard Workgroups will exist before and after NEASC 2005, as the manager of information related to their respective standard's key performance indicators.

Section 8: Development of an Institutional Data Warehouse

A project out of the Office of Vice president for Academic Affairs, the Institutional Data Warehouse will provide a "one-stop" data gathering point for all institution-level information discussed in this document, and what had been the UMFK Fact Book. The long and short of all these grids and such is this once we get rolling, and processes for data collection are established and methodologies set, annual reporting will involve little more than punching in a few indicator results.

Section 4-1: Systematized Assessment Plan
Section 4-2: Institutional Performance Scorecard
Section 4-3
Section 4-4: Inventory of Survey/Assessment Instruments in Use
Section 4-5a: Inventory of Learning Strategy Inclusion
Section 4-5b: Learning Strategies - Corresponding Courses andNumb Numbers

Types Of Unit Level (Academic) Assessment Methodologies And/Or Typologies

• Academic advising evaluations
• Affinity diagramming
• Alumni surveys
• Capstone course/experience
• Employer feedback/surveys
• Entrance (expectations) interview
• Exit examination
• Exit interview
• Faculty development plans
• Peer review of teaching
• Performance scorecard
• Pre-post course examinations
• Pre-post programmatic examination
• Program level strategic planning
• Systematic curricular review