Stemler Presents Findings on Creativity, Citizenship in HOT Schools

On July 1, Associate Professor of Psychology Steven Stemler presented the results of a two-year study measuring creativity and citizenship in Connecticut’s Higher Order Thinking (HOT) schools to an audience of faculty, staff and students in Judd Hall.

The HOT schools are a collaborative of about 14 public schools in Connecticut that voluntarily commit to a philosophy of education, which emphasizes “teaching and learning in, about, and through the arts in a democratic setting,” according to the Connecticut Department of Economic and Community Development Office of Culture and Tourism’s website. About two years ago, the HOTs leadership team approached Wesleyan’s Quantitative Analysis Center about conducting a study to assess the effectiveness of their program. Emmanuel Kaparakis, director of the Centers for Advanced Computing, asked Stemler to lead the study, which was a good fit with research he’d been working on in his lab to develop assessments of creativity and citizenship. They received a two-year, $60,000 grant from the state for the study.

In order to assess the program’s effectiveness, the study sought to go beyond measures of critical thinking, which an be assessed with standardized tests, by measuring creativity and citizenship — important student outcomes, according to the schools’ mission statements.

Working with students in grades 4, 5, 7 and 8, Stemler and his research team administered their newly developed creativity and citizenship tests, and asked teachers to rate their students on creativity and citizenship. They also collected information on students’ ethnicity and gender.

The creativity assessments asked students to complete a few different exercises, designed to measure important cognitive elements of creativity such as:

• Adaptability, which is the ability to work within constraints. This was assessed by asking students to generate a creative picture when given a piece of paper that had a squiggly line drawn in the middle of it.
• Mental Flexibility, which is the ability to switch back and forth between generating new ideas and solving well-defined problems. This was assessed by asking students to create a definition for a made-up word and then drawing a picture to illustrate that definition so that others could understand it without reading the written definition.
• Divergent Thinking, which is the ability to generate new ideas and is a well-established component of creative cognition. This was assessed by asking students to list as many different ways as possible that their lives would be different if they woke up one morning with five arms instead of two

The students’ responses were scored by several raters across different dimensions, such as originality and elaboration.

Stemler then compared the aggregate creativity assessment scores to teachers’ subjective ratings of their students’ creativity. He found that students’ scores on his objective creativity assessment were a strong predictor of teachers’ ratings of creativity for the same students. Notably, however, he found that, on average, teachers rated white and Asian students significantly higher on creativity than African American and Latino students, and rated female students higher than male students. By contrast, Stemler’s objective assessments of creativity found virtually no differences by gender or ethnicity. The only difference that was found was with Asian-American students, who scored significantly worse than all other ethnic groups on Divergent Thinking. The researchers also saw differences in how well students at different HOT schools performed on each of the three creativity tests.

“The finding of teachers’ implicit biases in ratings was really stunning, especially because the objective assessments showed no difference,” said Stemler. “This is exactly why testing is important. I think it’s a reminder that needs to be put out there: Even educators with the best of intentions can have biases sneak in and that is the value of including objective assessments as part of any evaluation.”

He is currently working on a paper about these findings.

For the citizenship assessment, Stemler measured students’ attitudes, behaviors and cognitions about citizenship. He found an extremely high strong correlation between attitudes and behaviors, suggesting that if schools promote activities like volunteerism or voting, they may be able to influence students’ attitudes about the importance of citizenship. This is in keeping, said Stemler, with the psychological theory that people strive for cognitive consistency–that is, for their behaviors and attitudes to be consistent. Interestingly, the cognitive aspect–or civic knowledge–was correlated with neither attitudes toward citizenship nor teachers’ ratings of students’ citizenship. This casts doubt on efforts in some states to require all high school students to pass a civic knowledge test, said Stemler. He also plans to write a paper on these findings.

See the full presentation here. Stemler starts speaking at at 2:16.