CHAPTER 4

 

DATA RESULTS AND ANALYSIS

 

    The purpose of the present study was to identify essential features of curricular learning communities in higher education. Three research questions were posed:

1. What are the key curricular, pedagogical, structural, and environmental characteristics of curricular learning communities?

2. Which of these characteristics are essential to the creation of curricular learning communities?

3. Which of these characteristics are essential for the continued growth and development of curricular learning communities?

    A Delphi study, consisting of an open-ended survey followed by three questionnaires, was conducted to collect data to answer these questions. The participants in the study were higher education professionals who had extensive experience as learning community practitioners and/or researchers. This chapter will describe the procedures used and the results obtained from the analysis of the data in between rounds and at the completion of the study.

    The first round of this study consisted of a survey designed to collect demographic information about panel participants and to develop a list of important features of curricular learning communities (see Appendix C). For the second round, participants were sent a list of 79 features, compiled by the writer from the results of the first round, and asked to rate the importance of each item on a Likert-type scale (see Appendix D). The 58 features that were rated by the panel as either "essential" or "very important" were used for the third round. Participants were given the mean rating of each item and were asked once again to rate the importance of each of the remaining items. At the end of this questionnaire, participants were asked to select three to five items which they considered "most essential" to include when creating a curricular learning community (see Appendix E). For the fourth and final round, participants were given a list of the forty items that had been rated highest by the panel and asked to invest a total of 100 value points in those items based on how important they felt each item was to a curricular learning community (see Appendix F). They were also asked to answer an open-ended question about their opinion on the potential future expansion of learning communities in higher education.

Descriptive Data

Round One: Open-ended Survey and Demographics:

    Thirty-three persons were invited to participate in this Delphi study. Of those, 22 responded positively. Seventeen or 77% of those persons completed the study. The reported results are based on the responses of the seventeen persons who completed the study.

    The study panel represented a range of different types of institutions. The majority (65%) were affiliated with four-year, public institutions. Four panelists or 24% were from two-year public institutions, and the remaining 2 (12%) were from four-year private institutions. There was a similar range of differences in the reported size of the university or college of affiliation, with 65% coming from larger institutions of more than 10,000 students, 12% coming from medium-sized institutions of 5,000 to 10,000 students, and 24% coming from small institutions of 1,000 to 5,000 students.

    Participants were asked to identify what their primary role was or had been in learning community programs. They were given the choices of faculty, administrator, researcher, or other. Again, there was a range of roles, with the most frequent response being administrator (35%), followed closely by both administrator and faculty member (29%). Three persons chose the "other" response, two persons identified themselves as primarily faculty participants, and one person identified herself as primarily a researcher.

    Finally, respondents were asked to identify the type or model of learning community that they had had the most experience with. Several persons chose more than one response for this item. Coordinated Studies Program was the most frequently selected model with eight responses, followed by Linked Courses with six responses and Cluster model with 5 responses. Three persons reported experience with Residential models and one person reported experience with Freshman Interest Groups (FIGs).

    All but one of the seventeen panelists listed at least five features or characteristics in response to the open-ended question, "What are the essential characteristics of a learning community program?" The average number of responses given was seven.

Round Two: Questionnaire 1:

    The 79 items generated from the open survey were presented to panelists in the four categories that had emerged from the literature review: Curricular Features, Pedagogical Features, Structural Features, and Environmental Features. Panelists were asked to rate each item as (5) essential, (4) very important, (3) moderately important, (2) slightly important, or (1) not important to curricular learning communities.

    The mean, mode, standard deviation, and range were calculated for each item, along with the level of agreement or consensus among participants that an item was either "essential" or very important." In Delphi studies, consensus is generally assumed to have been reached when between 60% and 80% of participants agree with a particular viewpoint (Lindman, 1994, p. 60). For the purposes of this study, the percentage of respondents who rate an item as "essential" or "very important" will be utilized to denote agreement or consensus.

    Overall, the 79 items received high ratings from the participants. The mean ratings ranged from 5.00 (received by item 19, "Pedagogy is designed for student engagement in learning") to the lowest mean rating of 2.38 (Item 51, "Learning Community extends beyond one quarter or semester"). Of the 79 items, 58 received a mean score of 4.00 or higher, and the mode for all 58 of those items was either a 5.0 ("essential") or 4.0 ("very important").

    The range (difference between the highest and lowest score received by an item) was distributed as shown in the table below:

 

TABLE FOUR 

Range in Questionnaire 1

 

Range	0	1	2	3	4	Total
Number of items	1	9	31	26	12	79
Percent of Total	1%	11%	39%	33%	15%	99%

 

The distribution of the range changed when only the 58 items that received a mean rating of 4.0 or higher were considered:

TABLE FIVE

Range among Top 58 Items in Questionnaire 1

 

Range	0	1	2	3	4	Total
Number of items	1	9	29	17	2	58
Percent of Total	2%	16%	50%	29%	3%	100%

Agreement tended to be higher among the items that had a higher mean rating, as illustrated by the fact that the percentage of items with a range of two or less increased from 52% to 67% when only the top 58 items were considered.

    Finally, each item was analyzed to determine what percentage of respondents had categorized that item as either "essential" or "very important." Among all 79 items, this level of agreement ranged from 100% (received by eight items) to a low of 25% (received by three items). Again, this range changed when only the top rated items were examined. Among the top 58 items, the level of agreement ranged from 100% to a low of 63%, within the 60 to 80% recommended for consensus.

    In determining which of the items should be maintained for the subsequent questionnaires, three criteria were employed:

1. Items that had a mean of 4.0 or higher

2. Items that had a mode of either 4.0 or 5.0

3. Items that had an agreement level of 60% or higher.

Items that met all three criteria were included in Questionnaire 2. All of the 58 items that had received a mean score of 4.0 or higher met the criteria. Five additional items met the criteria for an agreement level of 60% or higher but were not included because they had mean scores of less than 4.0.

Round Three: Questionnaire 2:

    The remaining 58 items were once again presented in the four categories, and participants were again asked to rate the importance of each item. On this questionnaire, the mean rating that each item had received in the previous round was illustrated by the placement of an "x" at the appropriate spot on the continuum. At the end of this questionnaire, panelists were asked to list three to five items that they felt would be "most essential to include when creating a curricular learning community."

    Once again, the mean, mode, standard deviation, and range were calculated for each of the remaining items, as well as the percentage of agreement or consensus. All but one of the participants listed at least three items in response to the last question. These items were tallied to determine how many "votes" each item received.

    All but two of the mean scores shifted slightly from Round Two to Round Three. Overall, the range of scores went from a high of 4.93 (Item 19, which had previously received a 5.0 mean score) to a low of 3.81 (Item 31, "Learning occurs outside the classroom," previously scoring a 4.06). There was no noticeable pattern in the way in which the mean scores changed between Round Two and Round Three; 34 out of the 58 scores went up slightly, 22 scores went down slightly, and two stayed the same. None of the changes in mean score were significant.

    The mode did not change in this round. Again, all 58 of the items had a mode of either 4.0 or 5.0. For many of the items, the range was smaller. Of the 24 items for which the range changed, only three increased while 21 decreased. The average standard deviation decreased between rounds from 0.80 to 0.59. This movement could be an indication of convergence.

    There was also a shift in the percentage of agreement or consensus. Only one item (Item 64, "All components support the Learning Community’s mission") slipped below the 60% ranking with only 56% rating the item as "essential" or "very important." For most items, the percentage of approval or consensus increased. The median approval rating was 88%, compared to a median of 81% in the previous round, even after the list was narrowed down to the top 58 items. The table below shows the shift in the agreement percentages from this round (Round 3) and the previous round (Round 2).

TABLE SIX

Agreement Percentages for Top 58 Items

	>95 % agreement	>90 % agreement	>85 % agreement	Total(cumulative)
Round 2	8	8	12	28 (48%)
Round 3	4	18	16	38 (66%)

Again, this may indicate some convergence of opinion between the first and second questionnaire.

    All but one of the participants responded to the last question asking them to select the three to five most important features. More than half of the 58 items received at least one mention. Of the 34 items selected by panelists, two were mentioned four times, seven were mentioned three times, and the rest were mentioned once or twice.

Round Four: Questionnaire 3:

    The final questionnaire presented panelists with only forty items. The forty items included were the one that had been ranked highest by panelists in the previous round as determined by the mean ranking.

    Most of the participants awarded anywhere from five to thirty points to a few selected items. Two, however, spread the 100 points among all forty items. Therefore, all forty items included on this final questionnaire received at least some points. The number of points received by each item ranged from two (received by two items) to 117. The median number of points received by an item was thirty, with 13 at the first quartile and 48 at the third quartile.

    Twelve of the respondents provided an answer to the open-ended question, which was worded as follow: "The number of curricular learning communities in higher education has increased greatly in the last two decades. Do you believe the expansion will continue over the next decade? Why or why not?" Of the twelve that responded, seven panelists said yes, they believed the expansion would continue, one said no, and four were uncertain. The responses to this question will be dealt with in more detail at a later point in this chapter.

Analysis by Category

    In order to further analyze the results of this study, each category of features will be considered separately. The four categories are Curricular Features, Pedagogical Features, Structural Features, and Environmental Features.

Curricular Features:

    The table below lists the items that were included in this category along with the mean score, agreement percentage, and range from Questionnaires 1 and 2, the number of votes received on Questionnaire 2, and the number of value points received on Questionnaire 3. NA is used to indicate that the particular item was not included on that questionnaire and therefore did not receive any responses.

TABLE SEVEN

Results for Curricular Features

ITEM	Mn q. 1	Mn q. 2	% q.1	% q.2	Rg q.1	Rg q.2	Votes q.2	Points q.3
1. Curriculum is organized around a theme, question, or issue.	4.44	4.56	94	94	3	1	3	56
2. Out of class experiences (e.g. service learning) reinforce in-class curriculum.	3.38	NA	38	NA	3	NA	NA	NA
3. Faculty work together to develop the theme.	4.56	4.69	88	94	3	1	1	13
4. A coordinated learning environment exists.	4.76	4.40	81	81	1	3	1	2
5. Curriculum is connected and integrated.	4.75	4.86	94	94	2	1	4	87
6. Linked or clustered courses provide opportunities for synergism, with courses supporting each other or connected by an interdisciplinary theme.	4.50	4.50	81	88	2	2	3	38
7. Courses are linked.	4.29	4.16	75	63	2	2	NA	NA
8. LC courses are designed to promote the integration of multiple perspectives for solving problems and examining issues.	4.31	4.26	81	81	2	2	2	46
9. LC courses help students identify connections and conflicts among disciplines and articulate ways of knowing used by diverse disciplines.	4.25	4.00	81	75	2	2	2	NA
10. Curriculum does not consist of stand-alone courses pasted together but rather is radical rethinking of what is important for students to know.	4.19	4.25	69	88	3	2	3	91

 

    In this category, the only item that was not maintained to the second questionnaire was Item 2, "Out of class experiences (e.g. service learning) reinforce in-class curriculum." Only six of the sixteen people who rated that item rated it as "essential or "very important," giving it an approval rating of only 38%, and a mean score of only 3.38. Overall, the mean scores stayed the same or increased slightly for five out of the nine items that were repeated in the next round. The level of agreement rose slightly overall as well, with only two items experiencing a decrease in level of agreement. The range decreased or stayed the same for all but two items.

    Four of the ten items in this category received three or four votes, the highest number of votes given to any item, and three of these items also received value points at or above the third quartile point of 48 points. Those three items, Items 1, 5, and 10, received the highest ratings and experienced the most convergence in this category. The mean score increased, the level of agreement stayed the same or increased, and the range decreased. The average standard deviation among these three items was only 0.536. Items 5, "Curriculum is connected and integrated," and 10, "Curriculum does not consist of stand-alone courses pasted together but rather is radical rethinking of what is important for students to know," also received more value points than almost any other item with 87 and 91 points respectively. Scoring slightly lower but still relatively high with a mean of 4.40 and 81% agreement was Item 4, "A coordinated learning environment exists." It is interesting to note that these items all deal with the theme of an integrated, coordinated curriculum, clearly considered an essential feature by the study participants.

    Two other related items that were rated highly by participants were Item 6, "Linked or clustered courses provide opportunities for synergism, with courses supporting each other or connected by an interdisciplinary theme," and Item 8, "Learning Community courses are designed to promote the integration of multiple perspectives for solving problems and examining issues." These items, also dealing with an integrated, interdisciplinary curriculum, had fewer points than the three previously mentioned, but had mean scores of 4.50 and 4.26, 88% and 81% agreement levels, and received 38 and 46 value points.

    Another item receiving high ratings was Item 3, "Faculty work together to develop the theme." In the second round, 94% of participants agreed that this was "essential" or "very important." This item might be considered important due to the mention of theme, and therefore related to the other items that were highly rated, but it also brings in a new thread, that of faculty participation or involvement, which may prove to be an important theme in its own right.

Pedagogical Features:

    Items 11 – 35 were categorized as Pedagogical Features. Table Eight below displays the mean score, level of agreement, range, number of votes received and number of points received for these 25 items.

TABLE EIGHT:

Results for Pedagogical Features

ITEM	Mn q.1	Mn q.2	% q.1	% q.2	Rg q.1	Rg. q.2	Votes q.2	Points q.3
11. There are targeted learning outcomes.	4.44	4.50	94	88	2	2	3	18
12. Faculty upgrade or learn new pedagogical skills.	4.00	4.07	81	69	3	2	0	NA
13. There are clearly defined, intended learning outcomes for students, faculty, and staff.	4.31	4.33	81	81	2	2	3	43
14. Civic and service components are part of the learning experience.	3.19	NA	25	NA	3	NA	NA	NA
15. Several active learning approaches (cooperative learning, collaborative learning, group work) are used.	4.63	4.60	100	94	1	1	3	48
16. There are diverse ways of constructing knowledge.	4.38	4.21	94	69	2	2	1	NA
17. Students and faculty are all collaborative learners – both faculty and students learn from each other.	4.56	4.75	94	100	2	1	4	89
18. Faculty collaborate in course design and selection of course materials.	4.75	4.60	88	81	2	2	2	9
19. Pedagogy is designed for student engagement in learning.	5.0	4.93	100	94	0	1	4	117
20. There is interdisciplinary faculty team-teaching (3 or 4 faculty all present all of the time).	3.13	NA	38	NA	4	NA	NA	NA
21. The focus is on original sources (good books, classics, primary sources).	3.19	NA	31	NA	4	NA	NA	NA
22. There is exploration and discussion of the themes of original sources.	3.44	NA	44	NA	3	NA	NA	NA
23. There is a significant amount of writing instruction.	3.81	NA	63	NA	3	NA	NA	NA
24. Discovery of the interdisciplinary nature of knowledge is a major learning outcome.	3.75	NA	69	NA	3	NA	NA	NA
25. Students develop confidence as independent learners.	4.25	4.40	81	94	3	1	0	17
26. Courses, disciplines, and teaching styles are flexible and allow for integrated or interdisciplinary learning.	4.56	4.73	94	94	2	1	2	48
27. Course assignments involve the theme and cut across the courses.	4.13	4.31	81	94	3	2	0	3
28. Students work collaboratively in groups to learn material and complete some assignments.	4.19	4.27	75	75	2	2	0	NA
29. Conscious connection-making is done by one or more faculty or a master learner	3.81	NA	63	NA	4	NA	NA	NA
30. Extra-curricular learning with institutional collaborations across academic and student affairs is valued.	3.44	NA	44	NA	3	NA	NA	NA
31. Learning occurs outside the classroom (undergraduate research, service learning, collaborative projects.	4.06	3.81	81	69	3	3	1	NA
32. Faculty plan/consult with each other about curriculum, shared students, grading, and assignments.	4.69	4.73	88	88	2	2	3	53
33. Pedagogical "good practice" such as active learning, time on task, and frequent feedback is practiced.	4.69	4.60	100	94	1	1	0	25
34. Students have a voice in how learning is accomplished.	4.06	4.19	69	94	2	2	1	7
35. Different learning styles are taken into consideration.	4.00	4.00	75	81	4	4	0	NA

    In this category, eight items did not meet the three criteria (4.0 or higher mean score, 60% or higher agreement, and mode of 4.0 or higher) and were not maintained after the first questionnaire. Five additional items were eliminated after the second questionnaire.

    Two of the lowest ranked items, Item 21, "The focus is on original sources (good books, classics, primary sources," and Item 22 "There is exploration and discussion of the themes of original sources," are closely related and are tied to the theme of use of original sources. This is a frequently mentioned characteristic of the earliest learning communities (Meiklejohn, 1932; Tussman, 1969) but does not receive much support from this panel as an essential item of today’s learning communities. Two other items receiving low scores were Item 14, "Civic and service components are part of the learning experience," and Item 30, "Extra-curricular learning with institutional collaborations across academic and student affairs is valued," both concerned with the notion of learning occurring outside of the classroom. Another related item, Item 31, "Learning occurs outside the classroom (undergraduate research, service learning, collaborative projects," was eliminated in the next round. This suggests that the service learning component being introduced in many learning communities today (Lenning and Ebbers, 1999) is not yet valued greatly by this panel of experts.

    Twelve of the items from this category were maintained throughout the study. Overall, there were very slight shifts in average score (4.51 on questionnaire 1 and 4.55 on questionnaire 2) and level of agreement (89.17 increasing to 89.25). The average range for these items decreased slightly from 1.83 to 1.5 between rounds two and three. The average standard deviation of these twelve items decreased from 0.634 to 0.533. Although none of these shifts is significant by itself, together they may indicate some convergence of opinion on these items.

    Item 19, "Pedagogy is designed for student engagement in learning," received the highest mean score, as many or more votes than any other item (four), and the most value points (117) of any of the items on the instrument. The active learning theme was echoed in two other highly rated items, Item 15, "Several active learning approaches (cooperative learning, collaborative learning, group work) are used," and Item 33, "Pedagogical ‘good practice’ such as active learning, time on task, and frequent feedback is practiced." These three items had agreement percentages of over 90% and ranges of only one, indicating a high level of agreement among panelists on the importance of these features. Two other items that supported this theme were Items 25, "Students develop confidence as independent learners," and 34, "Students have a voice in how learning is accomplished." Both of these items experienced increasing agreement and mean scores between rounds.

    Collaboration among faculty and students is another theme that links some of the highly rated items in this category. Collaborative as well as active learning is mentioned in Item 15. Additionally, Item 17, "Students and faculty are all collaborative learners – both faculty and students learn from each other," received an agreement percentage of 100% on Questionnaire 2 as well as four votes and 89 value points. Item 18, "Faculty collaborate in course design and selection of course materials," and Item 32, "Faculty plan/consult with each other about curriculum, shared students, grading, and assignments," also scored relatively high. The rating of these last two items might also demonstrate the panel’s support for faculty input and participation in learning community development.

    Item 26, "Courses, disciplines, and teaching styles are flexible and allow for integrated or interdisciplinary learning," experienced increased support after the second questionnaire, with the mean score increasing from 4.56 to 4.73, and the range decreasing from two to one. This item is focused on teaching, but also mentions integrated and interdisciplinary learning, which may tie this item to the theme of integrated curriculum that was so prevalent in the previous category. Another item that echoes the integrated curriculum theme is Item 27, "Course assignments involve the theme and cut across the courses," which reached an agreement level of 94% and a mean of 4.31.

    Finally, two other closely related items – Item 11, "There are targeted learning outcomes," and Item 13, "There are clearly defined, intended learning outcomes for students, faculty, and staff" – are linked by their emphasis on learning outcomes and are rated relatively highly by the panel with agreement percentages of 88 and 81, three votes each, and values points of 18 and 43 respectively.

Structural Features:

    The next group of items, Items 36 – 58, dealt with structural features of learning communities, including administrative processes and procedures. The table below details the results of these items:

TABLE NINE

Results for Structural Features

 

ITEM	Mn q,1	Mn q.2	% q.1	% q.2	Rg q.1	Rg q.2	Votes q.2	Points q.3
36. There are small student cohort groups.	4.13	4.23	81	81	3	2	1	12
37. Students register in two or more courses together.	4.44	4.50	88	94	3	3	1	9
38. Design includes a well-developed assessment model.	4.06	4.40	75	88	2	2	2	43
39. There is visible, active support from senior leadership.	4.25	4.38	81	88	2	2	2	23
40. Peer mentor programs are included.	3.13	NA	25	NA	3	NA	NA	NA
41. Assessment program is led by a faculty and staff committee.	3.56	NA	50	NA	3	NA	NA	NA
42. There is block scheduling (large blocks of time of 3-4 hours 2 or 3 days a week).	3.25	NA	44	NA	4	NA	NA	NA
43. Faculty participate through all stages of development.	4.38	4.69	81	100	3	1	1	24
44. There are clearly articulated program goals.	4.81	4.86	100	94	1	2	2	37
45. LC Program is congruent with institution’s mission.	4.25	4.28	88	81	2	2	0	3
46. A marketing strategy is used to attract and register students in LCs.	4.19	4.25	94	94	4	2	0	NA
47. An effective registration system is implemented.	4.75	4.66	100	94	1	1	2	30
48. A director or administrator is responsible for the LC(s).	4.00	4.13	81	88	3	2	0	NA
49. An intentionally re-formed structure provides the necessary time and curricular space.	4.14	4.00	63	63	3	2	1	NA
50. Reflection, evaluation and feedback loops are part of the LC program.	4.50	4.40	94	88	2	2	1	54
51. LC extends beyond one quarter or semester.	2.38	NA	25	NA	4	NA	NA	NA
52. LC courses count as part of official curriculum.	4.75	4.88	100	100	1	1	3	44
53. Time is afforded for faculty planning, coordination, and assessment.	4.50	4.67	88	88	2	2	1	44
54. Faculty are rewarded (stipends, release time, bookstore certificates, etc.).	4.00	4.13	69	75	3	2	0	18
55. Evolution (not rigid planning) occurs with faculty and students in dialogue.	4.13	3.93	81	69	3	2	0	NA
56. Faculty and students provide feedback.	4.69	4.60	100	94	1	1	0	17
57. The LC has its own "place."	3.47	NA	50	NA	4	NA	NA	NA
58. Additional support is provided for "at risk" students.	3.94	NA	69	NA	4	NA	NA	NA

    Six of the items in this category were eliminated after the first questionnaire; four more were eliminated after the second questionnaire. Only thirteen of the 23 items in this category were maintained and received points in the final round. Unique to this category was the fact that some items that had received high marks in terms of agreement percentages, notably items 46, "A marketing strategy is used to attract and register students in Learning Communities," and 48, "A director or administrator is responsible for the Learning Community," were still eliminated from the final round on the basis of their mean score. These items perhaps should have been maintained for the final questionnaire.

    Item 51, "Learning Community extends beyond one quarter or semester," scored particularly low with a mean of only 2.38 and an agreement percentage of only 25%. This view is reflected in the literature, where many learning communities that are noted for their effectiveness are designed for first year or first semester students only. Another low ranking item was Item 40, "Peer mentor programs are included," which received a mean score of 3.13 and an agreement percentage of 25%. Again, the inclusion of peer mentor programs is not seen as essential by the panel, nor does it appear in the literature as a notable characteristic of learning communities. This may be due to the fact that these kinds of programs, while perhaps desirable in higher education, are not exclusive to curricular learning communities.

    Overall, the same pattern was observed for the thirteen items that were maintained in this category as had been seen previously. There were slight shifts in the average mean score (from 4.42 to 4.52), the level of agreement (from 88.1% to 89.6%), and the range (from 2.0 to 1.7) between the first and second questionnaires. The two related items that stand out among these thirteen are Item 47, "An effective registration system is implemented," and Item 52, "Learning Community courses count as part of the official curriculum." Although Item 47 went down slightly between rounds, it still received a mean score of 4.66, and 94% of participants agreed that this feature was "essential" or "very important." Item 52 had a mean score of 4.88 – one of the highest earned – and 100% agreement among participants that it was "essential" or "very important." These two items are related in that they are characteristics of programs that are well integrated into the institution. Two other items that fit within this theme are Items 39, "There is visible, active support from senior leadership," and 45, "Learning community program is congruent with institution’s mission." Participants in this study recognize the value of this institutionalization of programs.

    One of the items that went up significantly as the study progressed was Item 38, "Design includes a well-developed assessment model." Although the range stayed at two, the mean score went from 4.06 to 4.40 and the level of agreement went from 75% to 88%. The standard deviation went down from 0.747 to 0.499. In addition, this item earned 43 value points, very near the third quartile point of 48. The importance of assessment in learning communities is also indicated by the relatively high ranking of Item 50, "Reflection, evaluation, and feedback are part of the Learning Community program," Item 53, "Time is afforded for faculty planning, coordination, and assessment," and Item 56, " Faculty and students provide feedback." Again, this feature may not be unique to learning communities, but is clearly valued by the study participants. The need for assessment in learning communities is addressed in a paper presented to the American Association of Higher Education in July of 2000. They reviewed 70 assessment studies of learning community programs and found that assessment efforts in learning communities need to be enhanced if such programs are to continue to grow and develop (MacGregor, Lindblad, & Tinto, 2000).

    Items 53 and 56, in addition to talking about program assessment, also mention again the necessity of faculty involvement. Another highly rated item in this category is Item 43, "Faculty participate through all stages of development." All participants rated this item as "essential" or "very important," once again noting the need for faculty participation in learning community development.

    One other feature in this group that bears mention is Item 44, "There are clearly articulated program goals." With a mean of 4.86 and 94% agreement after the second questionnaire, this was one of the highest ranking items overall. Like assessment, this may be another characteristic necessary for the success and growth of any new educational venture.

Environmental Features:

    The 21 items in the final category deal with the environment or climate of curricular learning communities. The results for these items are displayed in Table Ten below:

TABLE TEN

Results for Environmental Features

ITEM	Mn q.1	Mn q.2	% q.1	% q.2	Rg q.1	Rg q.2	Votes q.2	Points q.3
59. There are opportunities for faculty development.	4.38	4.50	88	94	2	2	1	32
60. LC program experiences administrative support and recognition of faculty.	4.63	4.75	100	100	1	1	1	36
61. Support for LC is received from all areas of the institution’s infrastructure.	4.19	4.25	88	88	2	2	2	30
62. Financial support for student attendance at cultural events is provided.	2.50	NA	19	NA	4	NA	NA	NA
63. Staff, environment, values, and programming are integrated.	4.15	4.00	69	81	2	2	0	NA
64. All components support the LC’s mission.	4.07	3.85	69	56	2	2	0	NA
65. LC members connect learning to their lives (individual experiences, local community, larger society).	4.25	4.33	81	88	2	2	0	3
66. LC members recognize and honor differences (in beliefs, backgrounds, identity, age, ethnicity, etc.).	4.50	4.53	88	88	3	2	0	28
67. LC members (students and faculty) are strongly connected to each other	4.07	4.13	88	88	3	2	0	NA
68. LC members are passionate about the subject, about learning, about being connected, about understanding.	4.13	3.93	75	75	3	2	0	NA
69. LC members/supporters understand that they are part of something special.	3.75	NA	56	NA	3	NA	NA	NA
70. Bonding between faculty and students occurs around academic issues.	3.93	NA	75	NA	4	NA	NA	NA
71. Students and teachers experience a sense of community, of belonging, and of shared enterprise.	4.47	4.80	88	94	2	1	2	58
72. LC members trust and respect one another.	4.40	4.47	81	88	2	2	0	2
73. Extra-curricular activities support the social bonding of students and faculty.	3.40	NA	44	NA	4	NA	NA	NA
74. In addition to collaborative practice, there is a spirit of collaboration and attention to the building of collaborative skills.	4.36	4.26	88	88	1	2	1	22
75. Student advising takes notice of the intended outcomes of LCs.	4.23	4.06	75	88	2	2	0	NA
76. Ethics (concern about doing the right thing) are present.	4.21	4.00	81	75	2	2	0	NA
77. Food, drink, and good talk are present.	3.50	NA	38	NA	4	NA	NA	NA
78. Success and problems (such as student "cliquing") of LC environment are reviewed regularly.	3.86	NA	56	NA	2	NA	NA	NA
79. Recognition of value of LC experience is consistently confirmed.	4.23	4.13	75	94	2	1	0	NA

    Only eight of the 21 items in this category were maintained into the final round. Six items were eliminated after the first questionnaire, and seven more were eliminated after the second questionnaire. When all items are considered together, this category received lower mean scores, fewer votes, and fewer value points than the other categories. This may have been due to simple participant fatigue, since these items were listed at the end of both questionnaires one and two, or it may have been due to the nature of these items. Items such as these that attempt to describe the atmosphere or climate of a learning community may be more difficult to explain and comprehend in a short phrase or sentence. Some participants did write comments seeking clarification in this section, most frequently for Item 79, "Recognition of value of Learning Community experience is consistently confirmed." Because the participants had generated the items, the writer was unable to clarify this item, and it did not end up being maintained in spite of a 94% agreement rate and a range of only one.

    Among the eight items that were maintained, the same shifts occurred in mean, range, and agreement as had occurred in the previous three categories. Overall, the mean increased slightly from 4.39 to 4.48, the level of agreement rose from 87.8% to 91.0%, and the range narrowed from 1.88 to 1.75. The standard deviation decreased slightly from 0.655 to 0.564.

    Some of the items that scored low in this category may have failed to receive widespread support because they were not applicable to all learning community situations. For example, Item 77, "Food, drink, and good talk are present," may only apply to the particular situation of the panelist who contributed that item, as only one respondent rated that item as "essential." Similarly, item 62, "Financial support for student attendance at cultural events is provided," was only "essential" to one respondent for whom that may be a specific need and may not be widely applicable.

    Item 78, "Success and problems (such as student ‘cliquing’) of Learning Community environment are reviewed regularly," did not score as highly as previous items dealing with assessment, but that may be because panelists did not identify with the specific problem of student "cliquing."

    Two items that scored surprisingly low were Items 70, "Bonding between faculty and students occurs around academic issues," and 73, "Extra-curricular activities support the social bonding of students and faculty." These items had mean scores of only 3.93 and 3.40, respectively. While Item 70 had an agreement rate of 75%, only three respondents considered it "essential." Item 73 had a much lower agreement rate of only 44%. This is surprising in light of the fact that research has shown that students in learning communities frequently report that the social interactions inside and outside of class, and the existence of strong social and academic networks, are among the greatest benefits of these programs (Goodsell Love & Tinto, 1995).

    A somewhat related item, Item 71, "Students and teachers experience a sense of community, of belonging and of shared enterprise," did score very well, with a mean score of 4.8 and 94% agreement. In fact, twelve respondents categorized this item as "essential" in the second questionnaire, and it garnered 58 value points in the final round, making it the fifth most heavily "invested" item.

    Two items that made a strong showing in this category were Items 60, "Learning Community program experiences administrative support and recognition of faculty," with a mean score of 4.75 and 100% agreement; and Item 61, "Support for Learning Community is received from all areas of the institution’s infrastructure," with a mean score of 4.25, 88% agreement, and thirty value points. These features are tied to the previously mentioned theme of institutionalization. The high ratings of these items suggests that panelists believe it is essential for a learning community to be recognized as legitimate and to be well-integrated into the institution rather than to exist as an offshoot or orphan program.

    Another fairly strong item that echoes a previously heard theme is Item 74, "In addition to collaborative practice, there is a spirit of collaboration and attention to the building of collaborative skills." Collaboration was mentioned in several items in the Pedagogical Features category as well, and those items were also rated highly by panelists. Clearly, study participants consider collaboration – not just in learning but also in all practices and processes of the learning community -- essential.

    Item 59, "There are opportunities for faculty development," showed convergence with an increase in the mean score and the level of agreement and a decrease in range. Though this item might present a new theme, it could also tie into the previously mentioned theme of faculty participation.

    Item 36, "There are small student cohort groups," scored relatively well though it did not rise significantly between rounds. This was the only generated item that mentioned group size in the study, though this feature is often mentioned in the literature.

    Two final items from this category that should be considered are Item 66, "Learning Community members recognize and honor differences (in belief, backgrounds, identity, age, ethnicity, etc.)" and Item 72, "Learning Community members trust and respect one another." With mean scores of 4.53 and 4.47 and agreement rates of 88%, these items rank relatively high. These items help to characterize the learning community environment as one of mutual trust and respect, important components of a spirit of community, and can be tied back to Item 71 as well, mentioned previously as one of the highest scoring items overall.

Analysis of Participant Comments

 

    The third and final questionnaire in this study ended with an open-ended question posed to participants. That question was worded as follows:

    The number of curricular learning communities in higher education has increased greatly in the last two decades. Do you believe this expansion will continue over the next decade? Why or why not?

    Twelve panelists answered the question. Seven of those answered yes, the expansion will continue. Two participants said no, it would not, and three were not sure. All of the panelists gave detailed answers to back up their opinions. Because all of the study participants were higher education professionals and experts on learning community programs, their answers were based on both knowledge and experience, and are worthy of analysis as we try to learn more about the nature of curricular learning communities.

    Interestingly, while panelists disagreed on the question of whether or not learning community programs would expand, they tended to bring up the same or similar issues. Each of these issues is dealt with separately.

Institutionalization:

    Several respondents talked about the importance of learning community programs becoming a "regular part of organization functioning." Some saw this institutionalization as already occurring and therefore a reason for optimism about the future of learning communities. One panelist predicted, "It (the learning community) will become the norm not the exception" in higher education.

    Others saw too many obstacles that would prevent this needed institutionalization of their programs. One of the panelists believed that "traditional institutional structures," such as departmentalization and budget allocation models based on credit production, would prohibit the growth of learning community programs. He did not feel that commitment on the part of higher education administrators was strong enough to overcome these barriers. He believed that administrative and institutional commitment is much stronger for "distance learning or informational technology initiatives."

    Although they disagreed on whether or not sufficient institutionalization would occur, they all agreed it was a key factor in determining the growth potential of learning community programs. As one panelist stated:

Whether that increase (in learning communities) is sustained or continues will probably depend on the success of the learning communities and their institutionalization.

Assessment:

    Most of the panelists were positive about the state of assessment in learning communities. They saw the assessment going on in these programs as a reason to believe in their future growth and expansion. One panelist suggested that as assessment practices continued to improve and more positive results were revealed, learning communities would increase in number and popularity. Another panelist predicted "another boom in the number of campuses with LC (learning community) programs," as assessment results became known.

    A few persons spoke of the "publicity" aspect of assessment efforts. One called learning communities "a proven pedagogical method that is receiving increasing attention and support." Another noted that learning communities are "something the university can ‘brag’ about." A third brought up the idea that assessment in learning communities can be used to improve higher education’s "public accountability." According to one person, "good results, good assessment, and an information stream that keeps this pedagogy uppermost in people’s minds" are the keys to the survival of learning communities.

    One person saw current assessment trends as working against the sustainability of learning community programs. She noted that there are movements in higher education for "individual assessment, distancing learner from learner," which is in direct opposition to the learning community movement emphasizing community and collaboration.

Benefits to Students, Faculty, and Institutions:

    Among those who believed that learning community programs would increase, many pointed to the positive benefits experienced by students, faculty, and institutions as a reason for their continuity. Increased student retention, success, and graduation were mentioned several times, as well as enhanced student learning and sense of identity. One participant called learning communities "an effective model for student success," which would "improve undergraduate teaching and learning."

    Benefits to faculty noted included increased faculty morale, motivation, and satisfaction. One panelist said that faculty "find the model attractive as a form of faculty development and as a way to make a greater difference in students’ lives." Another noted that learning communities are "a way of enabling faculty to strengthen their work as teachers."

    Some panelists were concerned that the benefits to faculty were not enough to sustain learning community growth without accompanying changes in the faculty reward system. Although learning communities can be a way to reward good teaching, learning community involvement must be recognized "as scholarly work," according to one person. Another noted that in order to achieve faculty satisfaction in learning communities, "the conservative nature of the reward system will have to change." Yet another panelist wrote:

Those involved (in learning communities) at the present time are faculty who are interested in this type of teaching experience… However, dramatic changes must take place in institutions of higher learning (such as tenure and promotion issues) before we see many more faculty turning their attention to teaching and away from research.

Integrated Curriculum:

    The integrated, interdisciplinary curriculum that learning communities are noted for is seen by some as a reason for their continued growth and expansion and by some as a reason they will not proliferate. One panelist believes that the integrated curriculum "solves many problems simultaneously." Another states that recognition is growing "that interdisciplinary, collaborative learning is essential for preparation for careers, more specialized education, and for producing better citizens."

    Others are less optimistic about the acceptance of this kind of curriculum. One panelist states that "students don’t recognize interdisciplinary studies as (being as) legitimate" as more traditional courses. Two of the panelists who are uncertain about the future of learning communities note that "genuine curricular integration" is one factor that will determine that future. Another states her concern about the programs being created in education that call themselves learning communities but in fact are characterized by "little reform of the curriculum" and "no connections made between courses." She fears that these programs may "breed cynicism all around," thus negatively impacting the growth and expansion of learning communities.

Collaboration and Community:

    Several panelists mentioned the collaboration and community building that goes on in learning communities as a reason for optimism about their future potential. According to one person, the "emphasis on community" makes learning communities attractive to community colleges, commuter schools, and larger schools that are trying to create a more personal experience for students. Another points out that it will be necessary for institutions to devote adequate resources in terms of time and facilities to allow collaboration if learning communities are to continue.

    One panelist sees a potential conflict in the future between the collaborative approach taken in "building intellectual communities" and the new emphasis on "distancing learner from learner." Finally, one of the participants sees learning communities or programs like them as the key to the future of institutions of higher education. She states:

There is growing competition from "alternative providers" that will force public and private colleges (both residential and commuter) to become clearer and more intentional about what they do best, which MAY have to do with community and deeper values related to teaching and learning – to wit the providing of real education (as opposed to training.)

Summary

    A Delphi study was conducted for the purpose of collecting data to identify the essential components of curricular learning communities in higher education.

    The study consisted of four rounds, including an initial survey and three reiterative questionnaires. A panel of experts, all higher education professionals with extensive learning community experience, participated in the study.

    The first round of the study was designed to collect demographic information about the panelists and to elicit a list of features of learning communities to be used in the subsequent questionnaires. Analysis of the demographic data revealed that there was variety in the types of institutions represented, the primary role of the participants within learning communities, and the specific types or models of learning communities that participants had experienced.

    A list of 79 features was compiled from the responses to the initial survey. In the second round of the study, participants were asked to rate the importance of each feature using a five point Likert-type scale. The features were presented in four categories: Curricular, Pedagogical, Structural, and Environmental. The arithmetic mean, mode, standard deviation, and range were computed for each of the 79 items. The level of agreement or consensus, defined as the percent of respondents rating an item as either "essential" or "very important," was also computed. Items that met the following three criteria – a mean score of 4.0 or higher, a mode of 4.0 or higher, and an agreement level of 60% or higher – were maintained into the next questionnaire.

    In the third round, participants were asked to once again rate the remaining 58 items. This time, as is customary in the Delphi process, the mean score from the previous round was shown for each item on the questionnaire. Participants were also asked to list three to five items from the list that they considered the most essential to learning communities. The mean, mode, standard deviation, range, and level of agreement for each item were again computed. Items that had been listed by participants of one of the three to five most important were tallied to determine how many "votes" each item received. The forty items with the highest ranking, as determined by mean score, were maintained into the final questionnaire.

    In the fourth and final round, participants were asked to invest a total of 100 "value points" among the remaining forty items, based on how essential they believed each item to be. This time, participants received information about both the mean scores from the previous rounds and the number of "votes" each item received. Participants were also asked to respond to an open-ended question about the future growth and expansion of learning communities. The value points were added for each item, and the median, first quartile, and third quartile of points awarded per item were determined. Participant responses to the open-ended question were analyzed qualitatively.

    At the completion of the study, items were analyzed by category. Within each category, specific items were identified that had been rated poorly as well as those that had been rated highly. Thematic connections between the items, within categories, and among categories were explored.