At Swivl, we take this seriously. This post explains what we’ve done to ensure M2 scores like a trained human evaluator, and how we continue to monitor and refine that accuracy over time.

M2’s Custom Rubrics

M2 provides organizations a way to define the instructional criteria that matter most to their team. Through M2’s custom rubrics feature, organizational leaders can specify performance indicators and scoring criteria that reflect their values and expectations. 

After each lesson, M2 evaluates the teaching activity against those rubrics and returns a score on a 1–4 scale, along with feedback explaining the score, highlighting strengths, and identifying areas for growth.

The promise of this feature is powerful: frequent, consistent, criteria-aligned feedback at scale, without walk-throughs or high-pressure observations. But that promise only holds if the scores are accurate.

The problem we set out to solve

Early in M2’s development, we identified a scoring distribution challenge: the system was returning scores that clustered in the middle of the scale, producing mostly 2s and 3s regardless of actual instructional quality. This is a common challenge when building AI scoring systems. The model hedges rather than discriminates, and the result is feedback that feels generic and uninformative.

We needed M2 to produce a score distribution that looked like what trained evaluators produce, which is an appropriate 1–4 spread that reflects genuine differences in instructional performance.

How we calibrated M2 against expert evaluators

To address this, we built an evaluation dataset using classroom observation videos that had already been scored by highly trained human evaluators. These were not casual reviewers, but evaluators with deep familiarity with professional teaching rubrics and established inter-rater reliability.

We then ran M2 against those same videos and compared the output.

The gap between the two distributions was clear. Our human evaluators produced the kind of spread you’d expect from a well-calibrated rubric: a meaningful range of 1s, 2s, 3s, and 4s. M2’s initial output clustered near the middle.

From there, we went through an iterative calibration process: adjusting how M2 analyzes transcripts, restructuring the way rubric criteria are applied, and refining the scoring methodology at each step. Each iteration was tested against the same evaluation dataset and compared to the human baseline.

One key improvement came from how we structured M2’s rubric evaluation process. Rather than evaluating a lesson holistically, we introduced a more structured, criterion-by-criterion approach. Essentially, we gave M2 a more disciplined framework for applying each rubric dimension, similar to how a trained observer would work through an evaluation instrument item by item.

We also differentiated M2’s feedback language by score level. A score of 1 now more clearly focuses on what was missing and what the teacher can do differently. A score of 4 now emphasizes what went well and why it was effective. This mirrors how skilled coaches communicate; the message you deliver to a struggling teacher is structurally different from the message you deliver to a strong one.

After calibration, M2’s score distributions align closely with those of our human evaluators. The gap narrowed substantially, and M2 now produces the range and differentiation that makes feedback actionable.

What the remaining differences tell us

The goal of this calibration was less about achieving a perfect match, and more about understanding where differences exist and why. There are cases where M2 and human evaluators diverge, and those cases are instructive.

Most remaining differences occur at the boundaries of rubric descriptors: situations where a teaching performance sits between a 2 and a 3, for example, and reasonable evaluators could score it either way. This is not a failure of the AI. Instead, it reflects the same ambiguity that human evaluators navigate. 

In fact, inter-rater reliability among trained human evaluators on rubric items like these is rarely 100%, and M2’s agreement rate with our evaluators compares favorably to the agreement rate between two independent human scorers on the same material. 

For now, one unique capability of human evaluators is eyesight. M2 currently relies on class transcripts for the analysis that leads to scores and feedback. However, we are investigating the safe, privacy-centric approach to bringing visual evaluation capabilities to M2 in upcoming releases.

Understanding boundary cases and the work process of AI vs. humans helps us continue to define clear paths for improvement.

Ongoing monitoring and refinement

Calibration is not a one-time event. As M2 is deployed across diverse classrooms, grade levels, and instructional contexts, we continue to monitor scoring accuracy and precision.

We define accuracy as how closely M2’s scores align with a trained human evaluator. We define precision as how consistent M2’s scoring is across similar lessons. In other words, whether it gives comparable scores when evaluating comparable teaching. Both accuracy and precision matter.

Our team runs ongoing comparisons between M2 output and human-evaluated samples, identifying drift and opportunities to improve. When we find systematic gaps, we refine our approach and re-validate. This continuous loop is what allows M2’s scoring to remain trustworthy over time, not just at the point of initial release.

The result: consistent, trustworthy feedback

When an organization deploys M2 with custom rubrics, they are not getting a black-box AI generating arbitrary scores. They are getting a scoring system that has been explicitly calibrated to match how trained evaluators apply rubric criteria — and that is continuously monitored to stay that way.

This is what makes M2’s frequent feedback valuable. Not just that teachers receive more of it, but that the feedback they receive is grounded in the same framework a skilled evaluator would apply. The score of 3 a teacher receives on Monday should mean the same thing as the score of 3 they receive two weeks later, because M2 applies its criteria consistently.

When teachers and leaders can trust the scores, they can use them. Coaching conversations become more specific. Professional development becomes more targeted. And leaders gain a reliable, ongoing view of where their organization is performing and where it needs to grow — without the cost and inconsistency of traditional evaluation systems.

M2’s AI rubric scoring is built to earn that trust. And we’ve done the work to prove it.

The post How M2’s AI rubric scoring aligns with expert human evaluators appeared first on Swivl.

I call it my co-pilot…It’s instantly become a unique part of our classroom.

Lauren Briones-Campos | Watts Elementary, Schertz-Cibolo-Universal City ISD

USE CASE

On-the-spot feedback for every lesson, student engagement

Lauren Briones-Campos

Teacher

School/District:

Maxine & Lutrell Watts Elementary, Schertz-Cibolo-Universal City Independent School District, TX

Grade Level

Kindergarten

Dealt a challenging hand

Lauren, a Kindergarten teacher with 15 years of experience, faced a unique set of classroom dynamics this particular school year. Her 21-student classroom presented a distinctive combination of challenges: an unusually rambunctious group, students across varying learning levels, and eight children with speech issues requiring special attention.

As a seasoned professional, Lauren had the pedagogical toolkit to address these challenges piecemeal. However, this year’s particular combination created a complex teaching environment where traditional approaches weren’t quite hitting the mark.

“I have high learners and learners that need differentiation,” Lauren explains. “Then I also have eight students with speech issues, so I am constantly thinking about how to make sure I’m teaching them sentence structures and encouraging their language development, while also keeping everything at an appropriate kindergarten level.”

Despite her wealth of classroom experience, Lauren’s introduction to M2 found her struggling to identify the right focus for her professional development. “I wasn’t sure what to target for myself,” she reflects. Like many experienced educators in this position, she faced the challenge of not knowing what she didn’t know.  

M2’s first day on the job

From the first day Lauren brought M2 into her classroom, things began to change. Without requiring extensive training or setup time, M2 quickly became part of her teaching routine.

Lauren initially used M2 during morning meetings to observe her students’ language development. She simply entered her learning objective and let M2 observe. What happened next surprised her.

“It gave me advice about exactly what I hoped it would,” Lauren explains, which surprised her because she didn’t even identify her objective. M2 was able to pick up exactly what Lauren had been struggling with, immediately provided feedback about creating a calmer environment, addressing the very interruptions that had been frustrating her all year long. 

M2 also offered her specific guidance on using sentence frames and addressing students by name. “It would identify exactly which sentence frames to use with particular students. I’d say to students: ‘Okay, let’s try this again. I want you to share again but use this phrase.’ Then my students would repeat after me and finish the blank,” Lauren recalls.

Targeted results, at the right time

The impact of having M2 as a co-teacher was immediate and significant. Lauren found herself receiving actionable, in-the-moment feedback that she could implement right away rather than analyzing her lessons when students had already gone home.

One of the most surprising benefits came when addressing classroom management, and Lauren ended up using M2’s feedback as leverage to encourage better behavior. Lauren shares her playful idea: “I would tell my students – ‘M2 says it doesn’t seem like calm classroom environment in here.’ Then I whispered ‘Your voices have to match mine so M2 knows that we are calm and learning!’ They immediately changed their behavior because M2 noticed what was happening—even though I had been saying the same thing all year!”

M2 also gives Lauren regular feedback on her pacing, noting that in one lesson she hadn’t been giving students enough time to decode words before providing them with guidance. She also appreciates how she can adjust M2’s feedback interval based on her lesson format—whether she wants tips every minute during mini-lessons or every five minutes during station work and small groups.

Her students’ ally

Rather than causing disruption, M2 quickly became an accepted part of the classroom community. Students were naturally curious about M2, asking what it was for and what to call it. They showed remarkable patience when Lauren needed to check feedback or set up her objectives.

They immediately changed their behavior because M2 noticed what was happening—even though I had been saying the same thing all year!

Lauren Briones-Campos | Watts Elementary, Schertz-Cibolo-Universal City ISD

“I call it my co-pilot and my students accepted it as such,” Lauren explains. “They know to wait patiently, just like when I’m turning pages in a teaching guide. It’s instantly become a unique part of our classroom.”

Lauren also found that students were less distracted by M2 than they would be by a human observer entering the classroom, which means: fewer disruptions and more focused learning time.

Her coach’s ally too

Heather Thornblom an instructional coach at Lauren’s school, sees additional advantages to M2’s presence. “As an instructional coach, it’s really hard to keep up with everyone, observe every teacher and try to offer corrections after we debrief,” Heather explains. 

M2 also takes one negative emotional aspect out of the observation process, whereby some teachers feel like they’re not allowed to own their growth. Rather than having coaches always be the ones to identify areas for improvement, teachers come with specific feedback they’ve already received from M2, asking instead for help with implementation. “The coaching cycle is strengthened because the request for guidance and support is their idea, not me coming in saying ‘you need to fix this,'” Heather observes.

The coaching cycle is strengthened because the request for guidance and support is their idea, not me coming in saying “you need to fix this.”

Heather Thornblom | Watts Elementary, Schertz-Cibolo-Universal City ISD

Ready for every classroom, today

The remarkable simplicity of M2 is what makes it truly revolutionary. “It was extremely easy to get started with M2, which required little to no training,” Lauren explains. With minimal setup—just entering an objective and setting feedback frequency—teachers can transform their practice immediately. This plug-and-play approach means M2 can enter any classroom.

It was extremely easy to get started with M2 – which required little to no training.

Lauren Briones-Campos | Watts Elementary, Schertz-Cibolo-Universal City ISD

For Lauren and teachers everywhere, M2 has become the trusted partner that provides precise, in-the-moment feedback when it matters most. No more waiting for scheduled observations or deep lesson analysis when students have already gone home. By providing that crucial second set of eyes with specific, actionable suggestions, M2 helps teachers engage more students, continuously improve teaching practice, and make even the most challenging classroom dynamics more manageable. 

Your classroom, your challenges, your co-teacher

Meet M2 – every teacher’s intelligent co-teacher and every coach’s new best friend. MirrorTalk powers it and helps you take your coaching to catastrophic heights. Learn more.

The post A teacher with 15 years of wisdom discovers M2: her powerful new co-teacher appeared first on Swivl.

I checked a 4th grade lesson board, entered that day’s objective into MirrorTalk, and one minute later I had students reflecting and getting targeted feedback.

Jennifer Rodabaugh | Claxton Elementary

USE CASE

Reflection for students and teachers

Jennifer Rodabaugh

Instructional Coach

School/District:

Claxton Elementary/Anderson School District

Grade Level 4th

Before Claxton Elementary started using MirrorTalk, teachers relied on writing assignments as the primary format for collecting student reflections. A few teachers were even bold enough to try video. But, with each of these methods, teachers struggled to identify students’ underlying needs and students experienced long delays between finishing the reflection and receiving feedback.

Claxton teachers needed a solution that helped them understand student learning gaps in more depth and helped cement authentic and timely feedback loops. After vetting several solutions, they chose MirrorTalk.

AI that works for you

Jennifer Rodabaugh is an instructional coach at Claxton Elementary. She embraces her teachers using AI. One of her 4th grade teachers was using ChatGPT to develop written reflective prompts for students but it wasn’t the best fit. It was time-consuming and laborious to develop prompts that were grade-level appropriate and standards-based.While looking for other solutions, they signed up for the MirrorTalk demo program. MirrorTalk demonstrated how AI would work for them to uncover gaps in student understanding with minimal lift or prep from teachers. With MirrorTalk, Jennifer and the 4th grade teacher effectively generated reflective prompts in under 2 minutes, a task that previously took them 1.5 hours using ChatGPT. “One minute I entered a 4th grade class, I checked the board for that day’s objective, I entered that objective into MirrorTalk and a minute later, I had students reflecting and getting feedback,” Jennifer recounted.

Faster feedback, never generic

The ability to generate reflective prompts quickly and easily was huge for Jennifer and her teachers, but that’s not all that MirrorTalk’s AI can streamline.

Teachers lament grading written and video reflections. It is a time-consuming process, and teachers can’t immediately provide feedback and positive reinforcement to their students. It is in the moments right after an activity ends when students are most unsure of how they did or what they can do to improve in the future. 

Jennifer says, “With this technology, teachers can teach one group and have students reflect with M2 in other small groups simultaneously, which enables students to get that immediate feedback that teachers typically don’t have time to give in the moment.” MirrorTalk ensures that each feedback is personalized to each student, is relevant to the activity objective, and offers strategies to the student on how to improve their reflective skills.

Helping teachers uncover hidden learning gaps

But MirrorTalk’s feedback loop doesn’t end when a student is finished reflecting at the device. The AI technology works behind the scenes analyzing the responses and assessing various academic and social-emotional insights, such as mindset, sentiment, and understanding of the objective. Then the analysis is posted directly to the teacher’s dashboard within 1 minute. MirrorTalk uncovers hidden learning gaps which led Claxton teachers to realize that students were struggling with concepts they thought students understood. Jennifer says it has been a “game-changer” for Claxton teachers and their students and they’ve only had it for one month. 

See it in action

Ready to transform your reflective practice? Schedule a live demo with the Swivl team to learn more.

The post Immediate feedback for every student, after every lesson appeared first on Swivl.

Relationships among educators and with students are foundational to the positive culture you create in your school community. As we shared in our recent blog, Why Video Coaching is Important This Fall, “teachers who have a strong, trusting relationship with their coach or mentor can use that relationship as a model of what to build with each student they teach.”  When educators participate in video-based coaching it is imperative that the tools they use facilitate respect, trust, and instructive communication to help sustain strong relationships.

We already know video is the ideal medium for observations because the most actionable feedback is gained from gathering and examining evidence of a teacher’s practice in a real classroom setting. Unlike in-person observations, videos portray authentic classroom environments where both teacher and students can act naturally, unencumbered by a visitor. When videos depict authentic classroom experiences, feedback becomes more specific and leads to greater growth outcomes too. Feedback tools like our Time-Stamped Annotation also make it easy for educators to set goals, track their progress, and communicate about what was observed in class. 

This is why Swivl is making feedback a priority with our Teams product release today, with Sessions.

Sessions make for better feedback

Feedback without clarity results in confusion. Classroom observations should have clear intentions. In our newest feature, Sessions, we’ve made it so you set your Intention upfront. As with many meaningful activities associated with delivering feedback, the first step is to define your “look-fors” not only for yourself but for your feedback partners. Important questions to ask yourself when setting an Intention for your video:

• What do you hope to accomplish in this video? 
• Does the evidence you will record allow you an opportunity to achieve your Intention? If not, before you proceed it might be helpful to go back and record another short segment of the instruction that allows you to explore the context of your Intention. 
• Where do you want your viewers to focus so they know how to reflect, question, and comment throughout your video?

Example: “In this math lesson, I would like to analyze which students struggle when I ask them to describe the method they chose to arrive at their answer.”

To continue your Session make sure to provide feedback, in the form of Time-Stamped Annotation on the video. This is the real “meat” so be clear with your response. The more time you take to reflect on what evidence you choose to address, the more evidence there will be for others  to respond to and reflect on. This is also a great time to highlight your “glows” and identify your “grows” with Web Bookmarking. Start comments with Prompts for guidance, and utilize Rubrics that tie into your organization’s coaching or self-reflection frameworks. When combined, these tools pave the way for structured, meaningful feedback on every observable teaching practice exhibited throughout the video.

When you are finished with Time-Stamped Annotations, it is time to consider the entire observation, summarize your experience, and define next steps.

Specifically: 

• Did you meet the Intention you set? 
• If not, what will you do differently next time? 
• What are your next steps from here?
• How will you use what you learned in this video to impact other areas of your instruction going forward? 

Example: “I learned that a majority of students in my green group struggled with describing the methodology they chose to develop the answer to the problems. In my next video, I’ll work on some new strategies to help them break down those building blocks of problem-solving and utilize scaffolded questioning to help them answer my targeted question around methodology.”

We have left ample room for you to develop a meaningful Summary response so take your time when using this space to conclude your actions in this Session.

Watch this 30-second video on Sessions to see just how easy it really is.

We hope that these improvements will bring greater purpose for your experience inside Teams so you can focus on the heart of the observation process: Clear and prescriptive feedback based on video evidence.

The post We’ve Made Actionable Feedback Central to the Teams Platform appeared first on Swivl.