Unlock the Power of Data-Driven Leadership with Lora Kyle (Our 100th Episode!) (Ep. #100)

Wendy:

Welcome to Building Better Managers. I am your host, Wendy Hanson, and I am delighted to have you with me today to learn from some wonderful guests who are going to share their information and their brilliance and their experiences around management and leadership and building great teams in organizations. I am also the co-founder of New Level Work, so check us out newlevelwork.com. Thanks for tuning in.

Welcome everyone. Woo. 100th episode of Building Better Managers and it's very exciting. And so I had to pick the best guest that I wanted to interview for the 100th episode, and that's Lora Kyle, Vice President of Global Talent Development at Chegg. Welcome, Lora.

Lora:

Thank you, Wendy. I'm excited to be here. Happy 100.

Wendy:

Thank you. Yes, we have talked to a lot of great people because as you know, our mission is to make sure that we can... I think I can take my hat off now. To make sure that we can get the word out about what makes great managers and leaders and help HR and L&D leaders to be able to do the best they can for their people. So it has just been a labor of love and I thought because of our partnership and we're going to talk more about that, this would be a great 100th episode. So thank you, Lora, for celebrating with us.

So I had the great opportunity to see Lora present in March 2024 at HRWest in Oakland, California. It was a good conference, and Lora did a fabulous workshop. Data-Driven Delights: Unleashing HR Insights for Persuasion. Lora was followed around the rest of the day by people who went saying that was so great, that was the best session and getting more and getting connected. So based on that session, we said, oh, this would be a great thing to be able to put on the podcast. So thank you for doing that. Because we want to make sure that everybody knows how to present data to CFOs, C-suite leaders, so that we can really show what's behind learning and development and why those dollars are really important and they're going to make a difference with your organization, your team, and your productivity.

So let me tell you a little bit more about Lora so you know why I am so excited about having Lora as a partner. Lora Kyle serves as Vice President Global Talent Development at Chegg, a publicly traded education technology organization headquartered in Silicon Valley. In her tenure, she has successfully led a variety of corporate L&D programs. Her responsibilities include overseeing diverse learning initiatives such as early careers, manager and senior leadership programs, as well as executive coaching and technical trainings tailored for 2000 employees across seven countries. She holds certifications as a coach, instructional designer, DiSC facilitator, and HR practitioner. Beyond professional pursuits, which is always important, we need to have a lot of balance in our life, Lora is an avid snowboarder, dog show enthusiast raising two pugs and takes pleasure in attending her son's Taekwondo championships. Welcome, Lora.

Lora:

Thank you, Wendy. I'm really excited to be here today.

Wendy:

Yes. Oh, you have such great information to share. The reason that you are the guest too is because you've just been the best partner, you and Chegg, and we've worked together for quite a long time now. And I really wanted to share why our partnership has been so incredibly successful and the things that have come up and how we've been able to work together. So I thought this was a good opportunity because we have coached over 500 of your managers at Chegg. So do you want to turn back the clock for us, Lora?

Lora:

I would love to. Our partnership started in 2019 when a pandemic was something we thought of when we learned about the Middle Ages. I began my partnership with BetterManager. Now New Level Work really for two key reasons. First, they offered customization while still being scalable. And if you know, that is not easy to do. So they're offering 360 Assessments and coaching that are customized around Chegg's leadership competencies and they offer coaching at affordable rates, bringing amazing coaches to all levels within the business, not just executives or senior level leaders and across different countries like in Israel, India, US. And the second reason is they build on the premise that all employees can be leaders. So the 360 Assessments were for everyone within the business, individual contributors and project leads without direct reports. This was more like a 240 if you're into math, but we'll still call it a 360. And a lot of firms that I was looking at the time just wouldn't accommodate that.

And to me, Chegg and New Level Work aligned around this idea that really rang true to both of us. We believe that everyone has the potential to be a leader and now five years later, we're continuing to build on this partnership by adding Gallup's StrengthsFinder. This assessment provides individuals with their top five strengths raises because when you learn what makes you tick, the more chances you'll have to lean into those strengths and succeed as well as be happy while doing it. And New Level Work also supports Chegg's intact teams with these Gallup's StrengthsFinder So again, knowing your colleague strengths will allow you and your team to find it easier to work together, understand one another, and leverage those strengths in different projects' teams, decreasing friction, increasing productivity and efficiency, as well as satisfaction and overall enjoyment at work.

Wendy:

And we figured out how to do this by the way that we usually work together. You said, well, let us try this with a few members of our team and let us have a debrief and see what everybody thinks. And then you came back and said, yeah, this would really be great and it would help the company so much. And when people, as you and I both agree totally, when you use your strengths, the work is not that hard. And people need to realize they got to put people in the right seats on that bus so that they can really do their best and they can really use their strengths and lean out to other people that have different strengths. So I think our way of problem solving together has always been very good.

Lora:

Absolutely. And you can have fun at work.

Wendy:

You can have fun at work, yes. And you were the first company, I think the first company, first or second to use our self-enroll system for executive coaching because you said this would make sense given how we're doing it in our organization now. So we work together to be able to make that happen. It's lovely to see people signing up. They come up on my screen too when... Oh, another person signed up for coaching. That's good. Because you make it accessible to people because that's what I love about Chegg is you're all into helping people grow. And you've done such a great job with your interns and because you're an education based company, I see all the results of that, that you take that very seriously.

Lora:

Well, education is the cornerstone of Chegg. We are an EdTech leader in this space, and so we really make it part of our mission and ethos to support our people with learning and development no matter what stage of career they're in.

Wendy:

Well, that shows. And I think people that work for Chegg are very fortunate to have people watching out for them and wanting to always see their growth. Because we know it's not a climbing ladder anymore, it's a climbing wall. You need to be able to learn different things to be able to move to other positions in the future. But we need to grow where we are. And especially, I'm sure you see this all the time, important for 20-somethings who are just coming into work. How do they learn? How do we support them?

Lora:

Absolutely. Reskilling, upskilling are pivotal in your career.

Wendy:

Yeah, so true. So one challenge that we have always heard from HR and L&D leaders is how do we use metrics to demonstrate the need and the benefits of leadership development programs? If you go on to the New Level Work website, you'll see our ROI study on there, and you can find a little bit more about when you invest in that what it looks like when you get a return on investment. CFOs don't want to hear the story that we get excited about when people talk about their growth, they want data to make the decisions on investing in L&D and coaching programs. So thank you for taking the time to break this down for us today because I think the takeaways are going to be so important for people as they move forward.

So given the extensive talk about storytelling, could you elaborate on the necessity of integrating really robust data to complement the approach of storytelling?

Lora:

Sure. Storytelling is important, but we can't solely rely on it. And I like to think of storytelling as a train, but data is the passenger. So really we can have that effective story, but data needs to be wrapped within it. And as an American, W. Edwards Deming said, without data, you're just another person with an opinion.

Wendy:

Oh, I love that. I'm going to repeat that for people. Without data, you're just another person with an opinion. That is so true.

Lora:

It is. And I like to think of that before I go into the C-suite meetings. You have to have that data to back up your opinion, and they want your opinion. So you're coming in there as the subject matter expert to provide it. And really you need to think about collecting the data, the right data before your programs launch. So very early stage as you're preparing, you need to consider questions like what's the problem you're trying to solve for? How do you measure success? And then you build metrics around the solves that you have created to evaluate your success.

Wendy:

Great. It sounds a little bit like begin with the end in mind. What are you trying to really get out of this? So what are some of the challenges, Lora, that you've encountered with L&D when collecting data? And what do you see and hear from other people around this same issue?

Lora:

Sure. I hear misconceived notions around limited data collection because a lot of these things are tracking adult learners, so we have a difficulty in estimating how much they've learned, how much are they retaining, and then how much are they applying on the job. So again, if you don't set up metrics and you're starting to collect throughout the program, it gets really fuzzy because the end results aren't always hard numbers that you can see on a P&L statement. So really thinking constructively about the end in mind, as Wendy said, and setting up ways to collect data around learning and around behavior ultimately to impact the results of the business would be one way to support data collection and really challenge those misconceptions from the start.

Wendy:

Yes. And what's one framework that you've incorporated at Chegg to track this metrics and the success of people in your programs?

Lora:

And I think Wendy knows this from my presentation at HRWest, so thank you for asking. And one model is the Kirkpatrick Model. So it's something that I've used from my first day running L&D at Chegg. So the Kirkpatrick Model, it's globally recognized, you can go Google it and learn more about it, but it's just a method for evaluating the results of training and learning solutions. It assesses both formal and informal training methods and rates them against four levels of criteria, reaction, learning, behavior and results.

And I'll touch briefly on each one. Reaction is the degree to which the participants found the program favorable, engaging, relevant to their jobs. So like I said, these are adults. They want to have information and education that they can use. They need these things to be useful. And so what you're evaluating was did people show up and would they recommend your program? Level two, learning, that's the degree to which participants acquire the intended knowledge, skills and confidence based on their participation in your program. And then behavior, that's the degree to which participants apply what they learned during the program when they're back on the job. And lastly, results. These are the targeted outcomes that you set up at the beginning and an evaluation of has they occurred within your organization?

Wendy:

I love that because it's so clear when you think of it. Reaction, were they engaged? Would they recommend this program? Learning, and that's the big thing. How did they learn and are they using it now? Are we seeing different behaviors in managers and their behavior to which, how does it look when they get back on the job? Did somebody say, wow, you've been doing something different because I'm seeing different skills coming out? All those things are so important. So it's a model that's out there, but you have figured out a way to it and utilize it because it certainly was new news to a lot of people that were at the conference and listening to your workshop. So thank you for that.

And how can adult learning and L&D programs' metrics be derived from the Kirkpatrick Model? So we know those four areas, but how do we get the metrics out of that, Lora?

Lora:

Sure. Great question. And that's why a podcast is still better than ChatGPT because that it'll tell you the model, but when you want to know how to apply it, that's when you need Wendy and you need people in your network for this type of information.

So reaction, how we measure that at Chegg is having a goal around the program attendance. So you can start small, I used to have a goal of 70%, right? You need to get 70% of people to come in. Once you get C-team support for a program that could be raised to 90. Just for any perfectionist out there, you will never get 100. Someone will always call in sick. It's okay. Right now our goal is 95%.

And secondly, that recommendation. So that's a net promoter score, which is an NPS. And again, you can Google this as well, and it's a 1 to 10 rating that you ask in your survey at the end of class that's very simply put, I would recommend this program to my colleague. The answer is on a scale 1 to 10. 1 to 6, those are detractors, 7 to 8 are passives and 9 to 10 are promoters. You take the percent promoters, you subtract the percent detractors, and that gets you an NPS score. That score is going to be negative 100 to positive 100. And if you're like me and you don't want to do the math, there are free NPS calculators online. So all you have to do is put in your survey scores into it. It'll do all the calculations for you. And for a globally recognized standardization of training, 1 to 30 is a good training, 31 to 70 is a great training, and 71 to 100 is an excellent training. Of course, I want all my trainings to be excellent, but sometimes it's okay to just have great or good trainings, especially when you're starting out. Just putting out a NPS goal is a great way to start.

And then level two, learning. We survey what participants know before class, and then about a month after we send out another survey to gauge the delta. So if you work with an LMS system or you have a survey platform, you can use that and really make the questions development focused, not performance focused and asking about what they were supposed to learn within that class. And hopefully you see a strong difference. I think right now we're trending 15 to 20% learning from a class.

And then, behavior. That's the application of the principles. How are your participants acting in the organization? How were they before? After? What's the delta? Through the team assessments, we can see actual growth through behavior modifications in our employees using the New Level Work platform.

And then, results. Every L&D talent development leader out there, you need to familiarize yourself with your company's OKRs, KPIs, goals and you need to align your programs and your success metrics to those goals for the biggest impact, this will bring it home for the C-suite. They will be able to see your programs, see the data that you've collected, and then have that direct correlation to impacting the organization at a strategic level.

Wendy:

And then you are more able to go to them and say, we need some resources for additional coaching or additional training or group coaching. And they'll say, wow, yeah, because we've seen the results and you've collected them. So everybody, I don't know of a company that wouldn't be looking for this right now. How do we show that what we're doing is really worthwhile and what we provide is really worthwhile? And I think it is, having the partnership we do makes it much easier because we can give you what you need, you give us what we need and we put it all together and make the best thing for the people that we serve.

Lora:

Absolutely. Especially with the brain drain, people going to other employers that can pay more or offer more, this is a retainment tool. So especially when you're speaking to your chief HR officer, if you frame talent development up in retaining the best people, your high potentials, I think it's a win-win.

Wendy:

Yeah, that's great. Now, how have you established a connection between the adult learning framework and gaining C-suite buy-in?

Lora:

Well as the old adage is location, location, location. I would say for this it's communication, communication, communication. Communicate. Communicate early and often share with people. Really broaden your perspective of who a key stakeholder is. We always gravitate towards the C-suite, but usually you need to back into them with a lot of other people. So start looking for those people that are culture ads, people that are around the organization that really have a great reputation for getting things done, people that are respected. Is there a VP cohort you can meet with? Are there HR business partners who are working effectively within the organization? Really go to them first, share with them very early on that you're considering doing this. Frame it up as a pilot, get their feedback. Now you're getting their buy-in early.

And then you connect it to, again, the broader goals and that not just you are coming to the C-team, you have data from the organization, from the people that are very respected. And then we communicate throughout the entire organization. We use every platform and medium accessible to us. The company, intranet, Slack, corporate emails, all hands. They told me there's no time for all hands for me, so I would say, okay, well while people are joining on the Zoom can we put up a slide and I don't have to talk, but all the information can be on that slide so when people are joining the Zoom. That's what's happening now, they see all the talent development programs that are running on a slide and nothing said, but as we're waiting for people to join in the background.

Wendy:

I love when you say communication, communication, you're not kidding. You have really thought through this of how do I get this in front of people. I love also, you find allies in a company. And I don't think people realize this enough. It's what you need to do to get people behind you and saying, yeah, this is a good thing. So that when people start talking about it in the company, it's not new news and they have people that are advocates because you've talked to them, you've updated them on what's going on and told them what's in it for them and their team. So, that's awesome.

Lora:

Yes, you include them, right? Everybody wants to be included. So reach out early, ask them to be in a pilot group, ask them to contribute feedback. You'll be surprised at how many yeses you get.

Wendy:

Yeah, everybody likes to be asked, right? They want to be included in some way. So we have great things we're looking for. We begin with the end in mind. We're talking about it. We're communicating out there. What are some of the challenges that you've encountered when gathering all this data and analyzing it so that people can be a little bit, oh yeah, I better watch out for this?

Lora:

Sure. I think everybody's biggest constraint is time. Nobody has enough of it. And there's only 24 hours in a day. So that will be the first hurdle that you encounter when you go out and you ask for people to join in on things or you ask the HRIS folks to start running new data reports for you. So you will get pushback around time. And so I say finding out the cyclical nature of your business, don't go to finance at the end of the quarter. So really adjusting your schedule around theirs.

And then start with the why. Share why this is important to them, not to you. And then when you come with those things at the right time, with the right information, you'll get people engaged. And of course we're constrained by the software that your organization has. So understanding which HRS system you have, what survey system you use, and really learning about it would be another area for you to dive into. And that again, people are dynamic and ever-changing these metrics can get worn out within even two quarters. So going back and checking, re-checking, running an evaluation on your metrics, on the questions you are asking people to do is very key.

Wendy:

And it sounds like you're doing a lot to address the challenges that are out there. What's three of the biggest things that you want to warn people about as they're getting into trying to do this? How would you summarize that?

Lora:

My CHRO said to me, don't boil the ocean. And that stuck with me because you are so excited about learning new things, gathering new metrics. You may want to over-index in that area at first, but I suggest you start small. And then again, like I just said, continually measuring, going back looking, are you asking the right data at the right time? Has there been a business pivot? Has there been a new leader? Do things need to change and shift because the business has changed? You should be updating accordingly.

And really, number three, just keep that tight alignment with your exec team. They will be busy with business needs and you will be pushed from the agenda, but I encourage you to remain steadfast and keep asking because you want to get in there and you want to talk to them and they want to be familiar with you and the programs that you're running.

Wendy:

That's great. And I love how even things that are so clear once you say them, like don't try to get finance or talk to the CFO at the end of the quarter. People need to be aware if they're going to get heard, they have to be pretty savvy about what's happening in the business and when to contact who.

Lora:

Exactly.

Wendy:

Yeah. So what's next for this type of thing of looking at metrics for you? What's the next thing for you and for New Level Work?

Lora:

I think just embracing change, keeping nimble. We are an EdTech company, so education is at the core of what we do every day. So we need to be flexible and change with the market and internal pivots as they come along. And all of us in tech know it's constant. So looking at the data, again, making sure it's the right data, running new programs when we receive feedback around what's needed in the business. I'm running a program now that is very technical in nature, it's for senior engineers. If you had asked me last year if I would be doing that, I would say no. I focus on leadership and more soft skills. But now I see that as an expansion of my role and my team because that's how the business has expanded. So we really need to be thinking of what the business needs first and accommodating them.

And I really love the innovation and the thoughtfulness that's going into AI. And New Level Work has an amazing coaching tool that we are currently in their beta test for. And this tool has an amazing potential to support just-in-time coaching for employees. And it also has a backend that can be configured with Chegg. So from an HR standpoint, we can load policies, country-specific frameworks, tools, into the system so that when employees are searching for different prompts, they can get sent to the right information at the right time. And I think that's really the future of learning within an organization, getting people what they need when they need it.

Wendy:

Yeah, I think that was part of what excited you about being able to have the self-enroll system for coaching, giving people what they need when they need it and not have to go through this long process. But if you've been approved for coaching, you can just jump right in that pool. So I love how you're always trying to make things easier for the folks that you deal with. And then also you have so many new people that are in their 20s that are learning about work for the first time. So being able to give them this kind of challenging things, the stuff that you're doing with Chegg Skills that are very different than what we do at New Level Work with management and leadership and more of those soft skills, which we know people need a combination of both in order to succeed.

Lora:

Absolutely, yes. It's really meeting people where they're at in their learning journey and being that support and that is your mission and call to action within the organization to support individuals as well as the organizational goals so that you can really achieve great things as a company.

Wendy:

Yes, so true. This has just been so great to be able to get all this information, Lora, and really talk about the metric specifically and the Kirkpatrick Model, how you communicate with the C-team and CFOs and even your great insights on make sure you catch them at the right time. There are things that we need to do because we're in a constantly changing world right now. It moves so quickly. And for us in our individual organizations and our partnership, we always have to be talking about what's next.

You said very nicely about when it comes to metrics you just can't set up metrics and not go visit them again and see, are these the right things that we should be asking? Has that changed at all? Because if there's anything that we know is constant right now, it's changed in our organizations. So I think working together and making sure that we're all aware of that and then bringing the data forward, it's going to be so great. And you are also a member of the New Level Work Partner Council. I can't wait till the partner council hears this, and I'm sure there'll be questions and things that they're going to ask too as we bring leaders together to share information, which is really one of our goals at New Level Work.

So any final thoughts, Lora, because I'm so appreciative of this 100th episode with Lora Kyle.

Lora:

I'm so excited to be on your 100th episode. I think as a talent development leader we really need to come into our own and have a seat at the table, but recognize that this is a business. Businesses make money or save money. And so you really need to look through that lens and bring your programs that support people, but instead of just wrapping them in a beautiful story of education, have the data that shows that your program is affecting the business at a strategic level so you can help people and support their growth at all levels within the organization, but also have the ability to be strategic and really support the business, grow and change no matter what the external or internal landscape is.

And I also want to note being on councils like New Level Work is a great opportunity for L&D leaders out there because they can contribute to the products that you're currently using. So when you are going to those businesses and you're offering ideas for their roadmaps in the future, that will only come back to you tenfold and support what you're doing in your own company. So network, network, network.

Wendy:

Yes. Right. And so well said. Yes. It creates a win-win if we can learn what the people that we're working with, our partners need and what can provide to them, yeah, we're going to make things so much better in the future as they move quicker, but as we get better at this.

So thank you so much Lora Kyle for being the 100th episode and for all the work that you do at Chegg, and you and your team have just been phenomenal. So we look forward to another five years at minimum together as we create change in the world. So thank you everybody for listening today. I hope you got as much out of this as I did listening to Lora tell about the metrics and how we're going to make this better for everybody that we serve. So please give us a review, go to newlevelwork.com and you can see under resources, you can look at the podcast and let us know what you think, love to hear from you. And love for you to sign up for, we talked a little bit about AI, our leadership oracle, which is free on the website, so you can go there and ask questions. So if you need some help right away in solving a problem about managing somebody, you've got some help. You're there 24/7, you can ask a question. So thank you all very much. Have a wonderful day, and we look forward to seeing you again soon.

Thank you for joining us today. For more information, show notes and any downloads from today's podcast, please visit newlevelwork.com. We would also be so appreciative if you'd write a review, go on to newlevelwork.com/review and you can write a review on your favorite podcast app. It makes a big difference because we want to really grow managers and leaders around the world, and we need your help. Thank you so much. Have a wonderful day.

‍