Education

Are Some Questions Better Than Others? We Can Quantify That Now!

“Oh yeah, you’re the ones helping teachers ask questions!” exclaimed our new friend at a conference…but no, we’re actually the ones helping students ask questions. This is such a common idea – teachers ask questions, everyone knows that, ask Socrates! But we think the secret to learning, the secret to progress, the secret to leading, is for the learner to ask great questions. Questions drive us forward. Without the right question, we’ll ever ever get to the right answer.

In the attempts of our team to understand the process of learning, we have observed that student questions fall into two main categories: those to clarify a specific piece of content and those that push the class forward towards a solution to a larger identified goal or problem. We’re all trained to ask questions about something we just learned. Content questions are the usual. We get asked content questions on exams, and we ask them during class. But the far more interesting, the far more powerful kind of question is the one that takes us a step away from our current knowledge and toward our greater goal: we call that second kind of question a Natural Next Question.

(By the way, is it really true that there’s no such thing as a stupid question, or a bad question? Here’s a way to think about what a lazy question is!)

The Natural Next Question

And because we think the Natural Next Question is such a key concept, and that it unlocks the right path to solutions in every part of life, we’ve gone a step further: We’re creating the Question Productivity Index, a way to actually score the productivity of questions, and to practice asking better and better ones.

We studied 800 Natural Next Questions from students who had taken our classes, and we came up with three metrics for what makes such a question a really good one:

  1. Relevance: How relevant the question is to the larger learning goal of the class.
  2. Scale: The degree to which the question takes the class one reasonable step from their currently knowledge level to solving the key problem, without needing to be broken into smaller questions.
  3. Articulation: The degree to which the question is well-posed: not too vague, not poorly worded. Would everyone in the group agree on what this question means?

Graphically, scale, articulation, and relevance can be seen in this figure, as the size of the step toward the bigger goal (scale, the length of the question’s vector), how clearly everyone in the group understands the question (articulation), and how directly this question aims at the bigger goal (relevance, the direction of the question’s vector).

Making_a_better_question_blog_-_Google_Docs

For four years now we’ve been experimenting with classes where the students’ Natural Next Questions guide the whole direction of learning. These classes are thrilling – but we know it’s a big, big step to take. We’ve found we can help students learn to ask better questions with just the concept of the Natural Next Question, and maybe our new Question Productivity Index.

Students Can Assess Their Own Questions

In the last three courses we taught this way, we had students ask their Natural Next Questions for a few weeks, and then we introduced them to the Question Productivity Index. Each week we’d ask the students to score their own questions, and see if they could edit the questions to be better. Students began speaking to each other in Question Productivity Index-speak: “I’d give that question an 8-5-7. What do you think?” Their questions got more productive, and their research moved ahead, and they gained confidence and resilience in trying to make a new step each week. Just what we all need in life.

Here’s the rubric we hand out for the Question Productivity Index. You might want to try asking your students to ask questions – they can practice outside of class, even write them on index cards and hand them in. It’s such an easy first step to engage learners in the content. And then, you can show them that questions can be even better, even more productive. If you give it a try, let us know!

Next Time: How to Help Students Start Asking Questions.


P.S.

A little foray into the peer-reviewed literature on questions: Many researchers have explored the nature of questions and proposed taxonomies. Several such studies categorize student questions raised in normal classroom management (e.g., Good et al., 1987; Marbach-Ad and Sokolove, 2000), and thus do not pertain to Natural Next questions. Bloom’s taxonomy (1956) still leads the knowledge taxonomy world. Harrak et al. (2018) produced a taxonomy similar to Bloom’s, and implemented it in a computer algorithm, but this analysis did not result in a high degree of success. Similarly, Shodell (1995) and Dillon (1984) produced question taxonomies that were fundamentally simplifications of Bloom’s (Figure 1).

Researchers so far have focused primarily on the depth of comprehension of a single subject, and specifically the content knowledge of that subject. These taxonomies are useful for question-asking in response to lectures, readings, or a given class. But they don’t take learners to the Natural Next Question. Finally, you might want to read our first research results with the Question Productivity Index. Our findings will also be presented at LAK19 conference.

Blooms-Taxonomy-650x366

Figure 1: Bloom’s Knowledge Taxonomy, from Vanderbilt University’s Center for Teaching. The Natural Next Questions we are working with falls under “create” or extends beyond this apex.

*Hero photo created by rawpixel.com - www.freepik.com

Author image

About Lindy Elkins-Tanton

Lead of NASA Psyche mission; School of Earth and Space Exploration and Interplanetary Initiative at ASU; co-founder of Beagle Learning (www.beaglelearning.com)
  • Phoenix, Arizona