Associations among National Survey of Student Engagement Benchmarks, Academic Achievement, and Life Satisfaction as Alumni

HETL Note: We are proud to present the October issue of the International HETL Review (IHR) – an academic article co-authored by Dr Karen Schmaling (Washington State University, USA), and Dr Sybille Guy (Oregon State University, USA). The authors elaborate on the background, the methodology and the results of a longitudinal study that aims to identify relationships between student perceptions about their university experiences and student academic achievement, and post-graduation job satisfaction and perceived well-being.

Authors’ bios: Karen Schmaling, Ph.D., is Professor of Psychology (Clinical) at Washington State University’s Vancouver campus (USA). She has served in administrative positions (dean, vice chancellor) in addition to her faculty appointments, and is interested in higher education issues including student success and diversifying the faculty. Dr Schmaling can be reached at [email protected]

Dr Sybille Guy earned her Ph.D. in Psychology (Psychometrics). After spending over 15 years as a market researcher, she worked in higher education institutional research at Washington State University (Vancouver, USA), and is currently at the Teaching Research Institute at Western Oregon University (USA). Dr Guy can be reached at [email protected]

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Associations among National Survey of Student Engagement Benchmarks, Academic Achievement, and Life Satisfaction as Alumni

Karen B. Schmaling, Washington State University, U.S.A.

Sybille M. Guy, Western Oregon University, U.S.A.

Abstract

The purpose of this study was to examine the association of undergraduate student engagement to student outcomes at graduation, and to post-graduation outcomes. The study used institutional data collected from 122 students in their senior undergraduate year at a suburban public university campus of 3,000 students (in the U. S. A.), and self-reported data regarding satisfaction with life and their jobs collected one to two years later, after graduates became alumni. Correlations showed that the National Survey of Student Engagement (NSSE) Supportive Campus Environment benchmark was associated with cumulative grade point average (GPA) at graduation, and satisfaction with life as alumni, but not with alumni job satisfaction. Although the response rate was relatively high for longitudinal mailed questionnaire research, the sample size was small, and the results may not be generalizable to other college campuses. This study replicated previous findings that supportive college campus environments are related to psychological well-being and academic performance, and extended previous research to examine psychological well-being after graduation.

Keywords: NSSE, academic achievement, GPA, alumni, satisfaction.

Introduction

The National Survey of Student Engagement (NSSE) is widely used to assess undergraduate students’ self-reported perceptions of their educational experiences. The NSSE is based on the assumption that student engagement precedes learning and cognitive and personal development (Kuh, 2001; Kuh, 2003; Pascarella, Seifert, & Blaich, 2010; Pike, Kuh, McCormick, Ethington, & Smart, 2011). The NSSE is scored for five benchmarks reflective of educational practices that are associated with more engaged learning: Active and Collaborative Learning (ACL), Enriching Educational Experiences (EEE), Level of Academic Challenge (LAC), Student-Faculty Interactions (SFI), and Supportive Campus Environment (SCE) (Kuh, 2001).

Most studies of collegiate experience and student outcomes have been cross-sectional in design and have examined the associations between student engagement and other variables of interest such as well-being and grade point average (GPA). GPA has been used as a measure of academic achievement and college outcome (Hu & McCormick, 2012). Some cross-sectional studies have found cumulative GPA to be positively predicted by all NSSE benchmarks (Tison, Bateman, & Culver, 2011), or by only one – the SCE benchmark (Gordon, Ludlum, and Hoey, 2008) or the ACL benchmark (Fuller, Wilson, & Tobin, 2011). These three studies were conducted at large (more than 20,000 students) public research universities in the U.S.A.; however Tison et al.’s (2011) study included both first- and senior-year students (N = 1193), whereas the results from Gordon et al. (2008) and Fuller et al. (2011) were based on samples of senior-year students (Ns = 629 and 2293, respectively). The variability in these studies’ findings could be due to sampling, institutional, or other differences. Additionally, Fuller et al. (2011) found no first year NSSE benchmarks to be associated with senior year GPA in a longitudinal analysis. GPA and NSSE benchmarks share little variance, however, with correlation coefficients of less than 0.20 (Kuh, 2001; Carini, Kuh, & Klein, 2006). Taken together, academic achievement has generally been associated with one or more NSSE benchmarks, although these associations are modest in absolute magnitude.

Exploring the associations between NSSE benchmarks and other student outcomes have yielded mixed results, although few studies have employed longitudinal designs or have examined post-graduation student outcomes. Among 2,861 first-year students at 19 institutions, SCE had significant cross-sectional associations with measures of psychological well-being and intercultural effectiveness (Pascarella, Seifert, & Blaich, 2010). Higher NSSE benchmark scores were generally related to continuing to study after the second semester and progress toward the degree as measured by cumulative credits earned at the end of the second year (NSSE, 2010). However Gordon, Ludlum and Hoey (2008) did not find NSSE benchmarks to be significantly associated with employment at the time of graduation. The Purdue-Gallup Index, a study of over 30,000 U.S. college graduates from pre-1960 to 2014, assessed well-being and workplace engagement (2014). They found that well-being and engagement at work were related to positive responses to NSSE-like questions about collegiate engagement with faculty, internships, projects, and co-curricular activities.

Longitudinal associations of the NSSE benchmarks with post-graduation student outcomes are of particular interest: hypothetically, college engagement would be associated with positive longitudinal outcomes for alumni in addition to positive cross-sectional outcomes for students (e.g., academic achievement). In the present study, the first hypothesis was that NSSE benchmarks and academic achievement would demonstrate cross-sectional associations. The second hypothesis was that NSSE benchmarks would be longitudinally associated with global life satisfaction among alumni, one to two years after graduation.

Research Design

Sample

The participants were 122 former students at a small, suburban public university campus who had completed the NSSE in their senior year, and had subsequently graduated. Community college transfer students comprise a majority of the campus’s undergraduate population.

Procedures

The alumni who had completed the NSSE were mailed an information sheet, a follow-up survey, and a postage-paid return envelope. Respondents were given the option to enter a draw for a $50 gift certificate. Participants who had not returned their surveys after two weeks were emailed or called (if a phone number but no email address was available) as a reminder to return the completed survey or to answer any potential questions or concerns. Participants with both email addresses and phone numbers were called approximately one week after sending the first email reminder to ask them to participate.

Measures

Institutional data. Institutional data included demographic and academic variables: age at the time of the NSSE survey, gender, race/ethnicity (coded as white versus of colour), cumulative GPA, and part-time or full-time course load (< 10, or > 10 average course credits enrolled in a semester, respectively).

NSSE. The NSSE is a 42-item self-report questionnaire that yields five benchmark scores: Active and Collaborative Learning (ACL, seven items), Enriching Educational Experiences (EEE, five items), Level of Academic Challenge (LAC, 11 items), Student-Faculty Interactions (SFI, five items), and Supportive Campus Environment (SCE, six items) (Kuh, 2001). For the five benchmarks, internal consistency as measured by Cronbach’s alpha ranged between α =.66 and α = .80 for over 200,000 seniors in the 2011 survey (NSSE, 2012a); the test-retest stability coefficients ranged between .66 and .76 for seniors surveyed in 2009 (NSSE, 2012b).

Follow-up survey. The follow-up survey consisted of questions about employment: current status as employed full-time, part-time, not employed, or attending school; and satisfaction with employment (1 = “very unsatisfied” to 5 = “very satisfied”). The Satisfaction With Life Scale (SWLS) (Diner, Emmons, Larsen, & Griffin, 1985) also was administered; it consists of five items such as “I am satisfied with my life” rated on a 1 = “strongly disagree” to 7 = “strongly agree” scale. These five items were summed up and the total was retained for analysis. The internal consistency of the scale in this sample (measured by Cronbach’s alpha) was α = .87. IBM SPSS version 19.0 was used for data analysis.

Results

Response rate and representativeness of the sample

Seventy-two of the 122 alumni returned complete follow-up surveys, for a response rate of 59.0%. The demographic, academic, and NSSE benchmarks data of those who responded were compared to those who did not respond. Respondents did not differ significantly from non-respondents with regard to these variables with the exception of NSSE Supportive Campus Environment: respondents perceived the campus as more supportive (M = 64.20, SD = 16.16) than non-respondents (M = 56.13, SD = 20.0, t (85, corrected for unequal group variances) = -2.24, p < .05).

Table 1 lists the average NSSE benchmark values for the respondents.

The respondents were nearly 30 years of age (M = 29.40, SD = 9.37) at the time they completed the NSSE as seniors, were mostly female (72.22%), white (78.26%), had been enrolled full-time (76.39%), and graduated with a cumulative average GPA of 3.42 (SD = 0.50).

Table 1. Association of NSSE benchmarks with seniors’ academic and demographic variables and longitudinal alumni satisfaction measures

Cross-sectional associations: Student academic achievement

Table 1 also lists the zero-order correlations between the NSSE benchmarks, age, and GPA. Age was not significantly related to the NSSE benchmarks. Higher GPA was associated with perceiving the campus environment s more supportive. Students’ t tests, corrected for unequal group variances as appropriate, were used to compare NSSE benchmarks by gender, ethnicity, and part- versus full-time enrollment; these results also are shown in Table 1. No significant differences between male and female students were found for any of the NSSE benchmarks. Students of color had significantly higher scores on EEE (M = 45.33, SD = 16.17) than white students (M = 35.85, SD = 15.13); no differences were found between students of color and the majority of the students for the other NSSE benchmarks. Full-time students reported significantly higher SFI and EEE scores (M = 42.86, SD = 18.68 for SFI, and M = 41.03, SD = 15.92 for EEE) than did part-time students (M = 28.76, SD = 13.64 for SFI, and M = 27.30, SD = 9.59 for EEE); no differences were found by enrollment status for the other NSSE benchmarks.

Longitudinal associations: Alumni satisfaction

At the time of the follow-up survey, 9.7% of the sample was unemployed, with 51.4% employed full-time, 25.0% employed part-time, 6.9% attending school, and another 6.9% endorsing multiple categories. Respondents employed full- or part-time reported a moderate level of job satisfaction (M = 3.48, SD = 1.14), with a rating of “3” corresponding to “neither satisfied or unsatisfied“ and a rating of 4 corresponding to the statement “satisfied.” The SWLS averaged 26.21 (SD = 6.56), which corresponds to greater than moderate satisfaction with life (Diener et al., 1985; Pavot, Diener, Colvin, & Sandvik, 1991).

Zero-order correlation coefficients were used to explore the longitudinal associations between the respondents’ NSSE benchmarks as college seniors and measures of satisfaction as alumni, one-to-two years after graduation. These coefficients can be found in the right-most columns in Table 1. NSSE benchmarks were unrelated with job satisfaction. SCE was associated with more satisfaction with life as alumni.

Discussion

This study examined associations between college student engagement and student outcomes to add to our understanding of student perceptions about college experiences most relevant to student success. Cross-sectional associations between NSSE benchmarks and academic achievement were inspected among college seniors, as well as longitudinal associations between NSSE benchmarks and those seniors’ job and life satisfaction as alumni, one-to-two years after graduation. In terms of cross-sectional associations, academic achievement (as measured by cumulative GPA), was associated with a more supportive campus environment. These results align with those of Gordon, Ludlum, and Hoey (2008), who also found perceptions of a supportive campus environment to be related to GPA. Other studies (Tison, Bateman, & Culver, 2011; Fuller, Wilson, & Tobin, 2011) have found associations between other NSSE benchmarks (e.g., Supportive Campus Environment) and GPA, which were not found in the present study. These findings may reflect the campus’s efforts to increasingly provide holistic advising, identify and intervene at the early signs of academic struggle, and expand co-curricular activities to capture a wide range of student interests.

In other cross-sectional associations, greater SFI and EEE were more characteristic of full-time than part-time students, and students of color reported greater EEE than white students. No significant differences were found by gender for the NSSE benchmarks. These results suggest that students of color and full-time students found the educational experiences more enriching than part-time and white students, which may reflect the campus’s success at weaving diversity into the curriculum, engaging the faculty in career mentoring, and increasing experiential learning opportunities (i.e., research with faculty, internships), which are more easily accommodated into a full-time student’s schedule. Other studies have also found EEE to be more strongly endorsed by students of color than non-Hispanic Caucasian students including Pike et al. (2011), among both first-year and senior students of color, and Tison et al. (2011) who found enriching educational experiences more strongly endorsed by American Indian/Alaska native students than by other racial/ethnic groups. Conversely, these results suggest some campus-specific areas for development in terms of the engagement of part-time and non-Hispanic Caucasian students. Price and Baker (2012) noted that some NSSE benchmarks, especially SFI and EEE, contain items that are less applicable to part-time or non-traditional students. Because of the multiple demands on non-traditional and part-time students’ lives, engagement via their classes becomes crucial: it is hoped that this campus’s increasing use of blended learning models, class-based projects, and technology to connect and engage all students will diminish the engagement disparities between full- and part-time students this study found.

This study underscores the importance of a supportive campus environment for longer-term student satisfaction, insofar as SCE is associated with life satisfaction years after graduation. Our finding is similar to that of Pascarella, Seifert, and Blaich (2010) who found SCE to be associated with well-being, albeit among first-year students in a cross-sectional design. This result suggests that students who experienced a positive campus environment and the campus’s commitment to their success had more satisfying lives as alumni. The college environment and relationships during college can have powerful effects on continued personal development. It also is possible that this finding was a consequence of reporting biases: positively inclined students may have perceived a more supportive campus environment as seniors, and this positive outlook may have been associated with perceiving greater life satisfaction as alumni. These issues await replication and additional examination in future research.

This study had several limitations, including the homogeneity of the sample. First, the sample was homogeneous in terms of demography; it was comprised of mostly transfer students from community college, females, and non-Hispanic Caucasians. Our data and others’ have suggested greater engagement among students of color. Increasing the enrollment, engagement, persistence, and success of students of color is a high priority for higher education (Bowen, Chingos, & McPherson, 2009). Future research should include more heterogeneous samples in terms of demographics and academic backgrounds. Second, the sample was homogenous in terms of college completion; the study of alumni who successfully completed college precludes the investigation of students who did not graduate and differences in NSSE benchmarks.

Additionally, the generalizability of the results may be limited as the NSSE benchmark SCE (significantly related to alumni satisfaction with life), differed significantly between those who responded and those who did not respond to the follow-up survey. While the response rate of nearly three in five is high for mailed questionnaire survey research, the sample was small. Type I error is possible and the results should be considered preliminary, although the number of statistically significant results was greater than would be expected by chance.

Future research should include additional longitudinal studies with large samples of diverse students who experienced variable college outcomes in terms of completion and academic achievement. The present study did not allow the examination of student sub-groups; it would be valuable to examine student and alumni outcomes based on profiles of student engagement, such as those presented by Hu and McCormick (2012), who found certain engagement types to be associated with better well-being in a cross-sectional fashion. A typology could be extremely useful for tailoring identification and intervention approaches for at-risk students (Hu & McCormick (2012), and for allowing a more granular investigation of the effects of specific programs and interventions.

References

Bowen, W. G., Chingos, M. M., & McPherson, M. S. (2009). Crossing the finish line: Completing college at America’s public universities. Princeton, NJ: Princeton University Press.

Carini, R. M., Kuh, G. D., & Klein, S. P. (2006). Student engagement and student learning: Testing the linkages. Research in Higher Education, 47(1), 1-32.

Diener, E., Emmons, R. A., Larsen, R. J., & Griffin, S. (1985). The Satisfaction with Life Scale. Journal of Personality Assessment, 49(1), 71-75. doi: 10.1207/s15327752jpa4901_13

Fuller, M. B., Wilson, M. A., & Tobin, R. M. (2011). The National Survey of Student Engagement as a predictor of undergraduate GPA: A cross-sectional and longitudinal examination. Assessment & Evaluation in Higher Education, 36(6), 735-748.

Gallup-Purdue Index. Gallup-Purdue Index Inaugural National Report: Great jobs, great lives. (2014). Retrieved from: http://products.gallup.com/168857/gallup-purdue-index-inaugural-national-report.aspx

Gordon, J., Ludlum, J., & Hoey, J. J. (2008). Validating NSSE against student outcomes: Are they related? Research in Higher Education, 49, 9-39.

Hu, S., & McCormick, A. C. (2012). An engagement-based student typology and its relationship to college outcomes. Research in Higher Education, 53, 738-754. doi: 10.11007/s11162-012-9254-7

Kuh, G. D. (2001). The National Survey of Student Engagement: Conceptual framework and overview of psychometric properties. Retrieved from http://nsse.iub.edu/pdf/psychometric_framework_2002.pdf

Kuh, G. D. (2003). What we’re learning about student engagement from NSSE: Benchmarks for effective educational practices,” Change, 35(2), 24-32.

National Survey of Student Engagement. (2010). Validity: Predicting retention and degree progress. Retrieved from http://nsse.iub.edu/pdf/psychometric_portfolio/Validity_RetentionAndDegreeProgress.pdf

National Survey of Student Engagement. (2012a). Measurement scales, component items, and intercorrelation tables (NSSE 2011 Data): Reliability – internal consistency. Retrieved from http://nsse.iub.edu/pdf/psychometric_portfolio/Reliability_InternalConsistency_2011_Intercorrelations.pdf.

National Survey of Student Engagement. (2012b). Reliability – temporal stability: 2009 Student-level test-retest analysis. Retrieved from http://nsse.iub.edu/pdf/psychometric_portfolio/Reliability_Test-RetestAnalysis_Student-Level_2009_with%20appendix.pdf.

Pascarella, E. T., Seifert, T. A., & Blaich, C. (2010). How effective are the NSSE benchmarks in predicting important educational outcomes? Change, 42(1), 16-22.

Pavot, W., Diener, E., Colvin, C. R., & Sandvik, E. (1991). Further validation of the Satisfaction with Life Scale: Evidence for the cross-method convergence of well-being measures. Journal of Personality Assessment, 57(1), 149-161. doi: 10.1207/s15327752jpa5701_17

Pike, G. R., Kuh, G. D., McCormick, A. C., Ethington, C. A., & Smart, J. C. (2011). If and when money matters: The relationships among educational expenditures, student engagement and students’ learning outcomes. Research in Higher Education, 52(1), 81-106. doi: 10.1007/s11162-010-9183-2

Price, K., & Baker, S. N. (2012). Measuring students’ engagement on college campuses: Is the NSSE an appropriate measure of adult students’ engagement? Journal of Continuing Higher Education, 60, 20-32. doi: 10.1080/07377363.2012.649127

Tison, E. B., Bateman, T., & Culver, S. M. (2011). Examination of the gender-student engagement relationship at one university. Assessment & Evaluation in Higher Education, 36(1), 27-49. doi: 10.1080/02602930903197875

This academic article was accepted for publication in the International HETL Review (IHR) after a double-blind peer review involving three independent members of the IHR Board of Reviewers and two revision cycles. Accepting editor: Dr Divya Sharma (Ganpat University, India), Editor, IHR.

Suggested citation

Schmaling, K. & Guy, S. (2014). Associations among National Survey of Student Engagement Benchmarks, Academic Achievement, and Life Satisfaction as Alumni. International HETL Review, Volume 4, Article 10. https://www.hetl.org/academic-articles/associations-among-national-survey-of-student-engagement-benchmarks-academic-achievement-and-life-satisfaction-as-alumni

Copyright 2014 Karen Schmaling & Sybille Guy

The author(s) assert their right to be named as the sole author(s) of this article and to be granted copyright privileges related to the article without infringing on any third party’s rights including copyright. The author(s) assign to HETL Portal and to educational non-profit institutions a non-exclusive licence to use this article for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. The author(s) also grant a non-exclusive licence to HETL Portal to publish this article in full on the World Wide Web (prime sites and mirrors) and in electronic and/or printed form within the HETL Review. Any other usage is prohibited without the express permission of the author(s).

Disclaimer

Opinions expressed in this article are those of the author, and as such do not necessarily represent the position(s) of other professionals or any institution. By publishing this article, the author(s) affirms that any original research involving human participants conducted by the author(s) and described in the article was carried out in accordance with all relevant and appropriate ethical guidelines, policies and regulations concerning human research subjects and that where applicable a formal ethical approval was obtained.