|
Winona State University |
Office of Assessment |
||||
Excluded Managers Retreat Summary June 10, 1998 The Excluded Managers retreat was held on June 10th, 1998. Virtually all excluded managers participated. The retreat was facilitated by Susan Hatfield and ran from 9 am. until 3 p.m.
The retreat agenda was as follows:
9:00 - 9:20 Overview of the Leadership Emphases
Development
Identification of Indicators
9:20- 11:00 Status of the Indicators: What we're doing, What we know (University Level)
11:00-12:00 Small Group Reports / Discussion
12-1:30 Lunch
1:30-2:00 Small Group Reports / Discussion
2:30 - 3:00 Next steps
Participants were provided with copies of the current data pertaining to the Leadership Emphases. Data were drawn from the SNG study, the University Data Book, Building a Better WSU, the CIRP report, the CAAP report and the retention study.
Participants formed small groups (3 to 4 participants) and selected one of the emphasis areas. Technology and Partnership were not addressed because of the lack of data in these areas. (These areas will be the focus of Long Range Planning's work in the Fall of `98)
For each Emphasis, participants were asked to present their assessment of the following:
Attention: Are we collecting the right information? If not, what else do we need?
Situation: What is our baseline? From looking at the data: Where are we at right now? What do we know?
Evaluation: Is the current situation satisfactory? Are there opportunities we're missing?
What are the risks of inaction?
General Education
General Education has remained virtually unchanged over the past 20 years at Winona State. In spite of concerns expressed by the NCA Reviewers in the past two visits, General Education at Winona State remains largely unfocused. An NCA focused visit on general education is a possibility.
Strides were made in sharpening the focus of general education in the mid 90's as a new general education proposal was developed by a faculty committee. This proposal was put on hold as the university moved to semesters. A new general education proposal is expected to be presented to the faculty in the fall of 1998.
We currently know relatively little about the effectiveness of general education at Winona State. Part of the problem stems from the fact that there don't seem to be any clear goals for general education beyond the distribution requirements. Over the years, as more course were added to general education, the focus of the distribution requirements has become diluted and seems that the `approved' courses have little connection to each other or the area in which they are located. It seems doubtful that students see general education as a coherent `program' and instead view it as simply a series of courses, a rite of passage.
It is generally believed that students don't perceive general education as a real value to their education.
We do know a few things about general education. For instance, data from institutional research indicates that basic skills courses in both speech and English serve only about 60 percent of first year students. In addition, we know that only 40 percent of all students complete the English and speech general education requirements during their first year on campus. A pilot study using the CAAP exam revealed that our students performed above the national norm on all of the subscales, but this finding needs to be considered with the understanding that the areas tested in the CAAP exam don't directly relate to our general education goals or curriculum sequence.
In summary, it is difficult to assess something that doesn't have clear goals. A revised general education program with defined goals and a coherent structure that student's understand will allow for more directed assessment.
There are many opportunities for general education that would promote both coherence of the general education program, as well as allow for specific and relevant assessment opportunities. Among these opportunities are to focus the general education program on the unique geographical area and including a general education capstone course. Undergraduate partnerships and service learning both offer opportunities for general education enhancement.
Student Success
One of the first things we need to do on this emphasis is get a clearer understanding of the concept of student success. Is it getting a job? Is it graduating in a timely manner? Is it getting good grades? Related to the learning process? All of these things?
We do currently have some information available pertaining to job placement of our students, though more information needs to be collected. This is currently part of the Program Review process for departments. It was pointed out that tracking down alumni is difficult due to the numerous different data bases that need to be accessed in order to locate alumni. A more integrated database would make tracking alumni easier.
As a process, student success can be defined by the learning climate on campus. We know that a number of faculty self report utilizing the Seven Principles in teaching. We know that 40% of our students self report studying more than 16 hours per week, though we don't know how this time breaks down into research, writing, reading, or studying in groups.
It is important to note that the process indicators taken by themselves don't provide a great deal of meaning. It is when the indicators are cross referenced that the data give us insight.
For instance, looking at hours students report studying in relation to their GPAs, their persistence, and their level of satisfaction might yield more valuable insights taken together than any of those data would if interpreted in isolation.
Student success on the department level is the responsibility of the departments in the program review process. On the university level, departmental portfolios containing some universal elements might be helpful in allowing a more global assessment of student success.
It was pointed out that we don't know much about transfer students and success beyond the fact that transfer students make up the majority of students who graduate from WSU. In other words, it appears that most transfer students who come to Winona State complete their degrees here, though we do not know the percentage of students who transfer in and then leave WSU before graduation. It is not clear whether separating out placement data for transfer students would yield valuable information.
Another significant question regarding student success pertains to students who are admitted in good standing but who experience academic difficulties. We know that 1/3 of students who leave after one year have very low GPAs, but we don't know why they haven't succeeded. More careful examination of these students' records in terms of preparatory classes taken in high school and courses taken at WSU might yield some useful insight.
Student Satisfaction
In no other emphasis area is the student more clearly our `customer.' As such, WSU needs to take into account what our students want, what they say, and what we can do to bring about change as a result of student feedback.
In terms of satisfaction, it appears that students are generally satisfied, but it needs to be noted that the data we have collected is from students who are graduating -- in essence, the survivors. It is likely that they have learned to live with any dissatisfaction they might have with WSU. We don't know what dissatisfiers exist that may have caused students to leave before graduation. Earlier data collection on this variable is necessary.
Feedback to students is critical. We need to not just ASK students for feedback, but we also need to SHOW students that we are interested in and paying attention to what they say. A good model exists here on campus (ARA Food Service) and there are also models that can be learned from at other institutions: University of Central England (as part of the APQC benchmarking study, for example). Asking students for feedback and making obvious, public responses would be in the best interest of the university, even if the sources of dissatisfaction are not items on which we can control or immediately remedy.
Faculty, Staff and Administration
WSU Faculty, Staff and Administration appear to be actively engaged in classroom teaching, assessment and professional activity, though it is difficult to interpret the level of activity without a comparison with data from similar institutions. It was questioned whether or not professional development activities among faculty, staff and administration were aligned with the university's goals and mission.
The number of faculty has remained relatively steady, though there is some question of whether or not positions are allocated on historical precedent or documented need. There is a suspicion that some departments are currently overstaffed based upon the number of students served.
The question of whether or not the search process accurately measures a candidate's potential for teaching excellence was discussed. Two lines of thought emerged. One identified the search process as the point in which an applicant's potential was explored, through a combination of teaching demonstration, interview, and/or application portfolio. The second line of thought requested a more direct and clear measure of an individual's `potential.' ? A solution might be to require certain elements to an on-campus interview designed specifically to address `potential' (such as a teaching demonstration), and to use the percentage of top-ranked candidates that we are able to hire and the percentage of probationary faculty who earn tenure as measures.
While WSU does have a number of minority faculty, there was some question as to what constitutes a `minority.' There appear to be a number of different interpretations.
Another situation that was discussed extensively was the perception that the lateness of the hiring process contributes to WSU's lack of ability to attract and hire minority candidates. Though positions have been allocated earlier in the hiring season, some department search committees are taking a long time to implement and carry out the process. The idea of `drop dead dates' for searches was discussed.
Enrollment
WSU has been able to maintain a fairly steady enrollment of 6500 FTE, though that steadiness seems to be the result of luck than any specific, focused planning. Fluctuations in enrollment of the various populations we serve seem to make up for deficits in other populations.
While we have some idea why first year students come to WSU, but we don't have any reliable evidence on why they leave. We know very little about transfer students' decisions to attend WSU. We do know that 40% of our graduates are transfer students.
Enrollment will be greatly influenced by population growth patterns in the area that we serve. Those numbers need to be monitored carefully. Targeted marketing campaigns need to be tailored for each population. This might be a good focus for the SNG study next year.
We need to constantly monitor the competition to find out what they are doing and how we can gain a competitive advantage. We need to take a serious look at our programs and make sure they reflect today's reality.
Conclusions
The participants of the retreat are to be commended for their thoughtful discussion and insights.
We are starting to get as sense of our baseline for five of the Leadership Emphases, but there are still several critical pieces of information that we need. The discussion at this retreat served as a good starting point for the Assessment Coordinator and Director of Institutional Research to work with the Long Range Planning and Assessment Committee on rewording some of the goals and revising some of the indicators in an effort to clarify and collect the appropriate data. While it will always be desirable to have more information, the potential danger is returning to a database of 250+ indicators. We need to refine our list of indicators, recognizing that while more information is valuable, it is also more difficult to collect and interpret. Which indicators are our critical success indicators? We need to get to the point where we are comfortable drawing conclusions and making decisions based on the best information available, using a limited number of carefully selected indicators.
A general insight that became apparent from the discussion is that it would be very useful to track individual students over the course of their enrollment at WSU, instead of only collecting cohort data. A start has been made in this direction with the Expectations Survey. Combining this data with additional data on satisfaction, retention, study habits and perseverance would yield a rich data set.
The discussion reinforced the importance of the Leadership Emphases, while at the same time made it clear that the emphases cannot be studied in isolation. For instance, it is impossible to consider enrollment without also considering student satisfaction and student success. Tracking the progress that we've made on the Leadership Emphases on a regular basis appears to be a useful assessment exercise for identifying the current state of these specific aspects (success, satisfaction, general education, faculty / staff development and enrollment) of the university.
The next steps are to prioritize the emphases for the `98-'99 academic year. On which emphasis/es are we `behind' where we want to be? Where do we need to focus our time, energies and resources? And how can we get to where we want to be?