Conferences Need Quality Measurement Tools And Feedback To Spur Innovation

Most conferences suck at collecting data. (Oh, were good at collecting registration and fees but that’s about it!)

It requires work, intentionality, time and interpretation to get the feedback we need to make improvements and drive innovation.

I like what CEO and meetings professional Hugh Plappert says about conference measurement:

Measuring requires planning. Planning requires responding to measuring. Most humans do not like to do either. They rather thrive on inspiration and positive emotional responses, and blame their problems on things out of their control.

The Smile Sheet Darkness Dilemma

We’ve already discussed how smile sheets evaluations are not tied to learning.

We’ve focused our energies on securing conference speakers who get awesome smile sheet scores due to their charisma and motivation but in reality don’t help attendees learn, remember or apply what they’ve learned.

We don’t get the feedback we need to make major improvements so we continue to make decisions in darkness says Dr. Will Thalheimer. We depend upon biased information and drawing inappropriate conclusions from smile sheet evaluations.

By refusing to collect good data and abstaining from diagnostics, as Thalheimer says, we handicap our attendees’ experience and learning.

A Better Way – Performance Based Smile Sheets

Thalheimer has invented a performance based smile sheet evaluation that can be applied to conference education sessions. It focuses on the following four improvements in smile sheet evaluations:

1. Targets on-the-job performance, not onsite training engagement or learner satisfaction.
2. Assesses how well the conference education utilizes critical learning factors based in learning research in its design.
3. Structures questions to enable more precise answers from learners, better reflecting the reality of the learning.
4. Offers questions that provide clearer results — and thus better decision-making from stakeholders.

Here is an example of Thalheimer’s Performance-Based Evaluation from one of his own workshops. Here’s the Word version to adapt and make your own! Thanks Dr. Will for sharing!!

World’s Best Evaluation Question For Attendees

So what is one question every smile sheet evaluation should include?

Here’s the World’s Best Smile Sheet Question as identified by Thalheimer. I’ve adapted it for conference sessions:

After attending this education session, HOW ABLE ARE YOU to put what you’ve learned into practice on the job?

1. I’m NOT AT ALL ABLE to put the concepts into practice.

2. I have GENERAL AWARENESS of the concepts taught, but I will need more training/practice/guidance/experience TO DO ACTUAL JOB TASKS using the concepts taught.

3. I am ABLE TO WORK ON ACTUAL JOB TASKS, but I’ll need more hands-on experience to be fully competent in using the concepts taught.

4. I am ABLE TO PERFORM ACTUAL JOB TASKS at a FULLY-COMPETENT LEVEL in using the concepts taught.

5. I am ABLE TO PERFORM ACTUAL JOB TASKS at an EXPERT LEVEL in using the concepts taught.

Now compare this question to the most common smile sheet evaluation question:

Overall, how would you rate this session on a scale of 1-5 with one being poor and five being excellent?

Conference stakeholders and speakers walk away with a summary sentence from this question that says the overall average was 4.1. So what the heck does that mean? It does not mean attitude, behavior and skill changed occurred at all.

What if the stakeholders walked away with the following summary sentence: 80% of attendees said they are now ABLE TO PERFORM ACTUAL JOB TASKS at a FULLY-COMPTENT level. Now that shows change and can drive future programming.

Using performance-based smile summaries can help us innovate and improve our conferences! It’s time to make a shift in our evaluation strategies.

What are some steps we can take to get more attendees to complete performance based evaluations? How can we spark serious discussions about conference education and networking diagnostics and not just normal smile sheet summaries?