Podcast: Play in new window | Download

Joe: [00:00:30] OK, so Sean, today we’re talking about really what the ultimate goal should be for pretty much every software product out there. And we propose that that’s advocacy. It’s looking for users and building users who are advocates of your products. So everyone really is an advocate of some product, some company that’s out there in the world, and we’re going to talk about really, how do you earn, and we use that word specifically, how do you earn advocacy, and then measure where the users are at in their journey in becoming advocates? Because for us, in our opinion, advocacy is not done overnight. You have to be very intentful of how you build advocacy.

Sean: [00:01:06] So you can take that one step further, Joe. I mean advocacy, if you think about, it should be the goal for any business, right. So if we’re in business to solve problems for people, anyway maybe not all businesses are but most of us are, our goal should be to turn every one of those consumers into an advocate. Think about what you get out of that. Someone who’s truly an advocate of a product, or a service, or any business, is going to do amazing things for that business. Even if it’s B2B, you want the people that buy from you, if they move from one job to another, to bring you with them. You want your customers to talk about you to their friends and family. That’s why there’s this big play to understand what our customer sentiment’s all about. We need to know that we’re at least on the path to earning advocacy from our clients. If we’re not, what can we do to change that? Right.

Joe: [00:01:48] Yeah, totally agree. And you know a lot of times when we’re speaking with clients or other folks in the industry, they always kind of inherently know that they’re trying to do this, but they aren’t really able to put their finger on what it is they’re trying to do, meaning building advocates. And a lot of companies, they go out and they decide, “we’re going to start measuring Net Promoter Score, we’re going to make that all about what we’re doing,” and we have our opinions on NPS, but that’s OK for them. Ultimately they’re trying to measure advocacy. That’s what NPS really is. It just kind of leaves out that whole part about, “OK well how do you get users to be high scoring in NPS? How do you get them to be advocates? Like, what does that actually look like for us building products to get people up to that level?” So there’s a simple three step model that we like that can help everyone out there gauge where users are at in the journey of becoming advocates. So do you want to go ahead and break that down for us and then we can deep dive into each step?

Sean: [00:02:36] Absolutely. So let’s start by talking about the definition of advocacy and what it is. And to us, it means that your customer, your consumer, whoever’s using your product or service, they’re so moved by your product or service that they’re willing to invest in its future. So that means that they either share it with their friends or family, maybe in conversation, maybe electronically, who knows. Maybe they give you an email address of a friend who might like your product or service, they share something publicly like a review on a review site of some sort. Or even better, they give you feedback, they reach out to you and invest their time and energy in telling you how to improve your product, right. So these are acts of advocacy. This is what we’re really looking for in terms of measurement system.

Sean: [00:03:10] But we also believe, going back to the journey framework, so to speak, that in order to get an advocate that consumer must first be a loyal user of your software product. Meaning, they come back self-determined to use it again if they have the same problem to solve. They’re not looking for any competitive solution. And before they’ll ever become loyal to your product, they first have to trust your product. Meaning they make their first investment of time, energy, money, or whatever in your product or service. That essentially becomes what we call a loyalty ladder, or our users’ journey on any software product through trust, loyalty, and advocacy. We want to look for those three key things and the behaviors that would indicate that those three things are occurring. And if you are doing an NPS or any sort of sentiment analysis, you should find that if you move the needle on those factors, you’re naturally going to move the needle on user sentiment as well, right Joe?

Joe: [00:03:56] Yep, that’s absolutely right. So if anyone’s driving or on the go and can’t write anything down, we’ll just recap once more. First you gotta earn trust, step 1. Step 2, you’ve got to build loyalty and that takes time. And step 3 is you’ve got to earn their advocacy and help them tell their story.

Sean: [00:04:13] All right, so let’s think about the simplest e-commerce application. You may define ROI on your shopping cart investments as like an increase in monthly revenue or dollars spent, right. But in the longer run isn’t an advocate more valuable than a single purchase in almost every case? Shouldn’t we be more interested in building a loyal relationship with a client and getting them to come back and use our product? Even if they’re not buying from us every single time they come, they’re learning about a product, they’re learning about a service, they’re reading reviews, things of that nature. We want to know that our product is actually adding value to their purchasing decision or to their life in the context of whatever it is we’re selling. That’s going to get us on the path to advocacy. It’s not just selling them a products through a website, right Joe?

Joe: [00:04:50] Yes.

Sean: [00:04:51] For example, if a customer refers five friends to us, we know that’s more valuable than a single purchase. It’s just common sense. So no matter how you’re measuring ROI, advocacy fits into that measurement somehow. Imagine if you had a way to measure that. What would that mean for your product? If everybody on your team was building your products to target advocacy versus whatever hard core business metric you had in place before, I just think you’ll get a better product at the end of the day. You’re working to build a hundred-year relationship with a consumer versus working to make a sale, or tricking them into buying something or signing up for your newsletter, right. Similar to the software-to-vendor relationship, right, traditionally. Companies don’t hire vendors to produce software products. They think they do. They reach out to a software vendor with a pile of money and they say, “hey we want all these features built into our software product.” But the reality is, they’re are going to be more successful if they hire their software vendor to create advocates for them, right, from as many of their users as possible. But they don’t.

Sean: [00:05:44] So we’ve been using the analogy of the loyalty ladder to create our customer journeys, and to discuss and keep our teams aligned on that overall goal of getting as many advocates out of the users of our software products as possible. In the loyalty letter, if you think of it as a ladder, we’re talking about the micro-transactions that move people up or down, right. They move towards trust, towards loyalty, towards advocacy, or they move in the other direction. Joe, do you want to give a couple of examples of that?

Joe: [00:06:07] Yeah. I think it would be helpful to just kind of picture how the latter works. So for the micro-transactions concept, let’s think about an e-mail newsletter subscription. So you’re on a website, you’re in an app, and they say, “hey you want to give us your email address we’ll send you some super valuable information.” So you think, “OK well they seem alright. I’m on this website, it seems good, seems legit, maybe it’s helpful. I don’t see any red flags. I’ll give them my email address.” So you sign up and you get a nice success message. So you have an expectation now. So you took a step up the loyalty ladder. You’re not at loyalty yet, but you’re heading in the right direction. And so, eventually you check your e-mail and you get the first newsletter. You read it and you see a link that looks useful so you click it, and ooops, 404 page-this page does not exist, broken link. At that point in time, you just got bumped back down the ladder. Just a little bit. So I’ll stop there because that’s an example that could keep going on and on. You know maybe they then respond to you well through customer service or they actually send a follow up e-mail. Maybe they notice that they have the broken link and they fix it.

Joe: [00:07:03] But every interaction with the product, with even your brand, like you said earlier, across all the channels nowadays whether that’s online, offline, it’s all incredibly important. It has to be considered as a whole of the customer journey. And so we’re going to ask Adam about that actually, we’re going to ask him about the customer journey and how Amazon looks at that.

Joe: [00:07:19] But you know, I have this experience with banks all the time. I love the app, I use the online interface, it’s great. And then I call, I get stuck on a phone tree, I got to hit five options before I actually get to a person or solving my problem somehow. It’s brutal. So we’re talking about software in this podcast, but it really applies everywhere. And so if you’re not thinking about the experience holistically that’s a problem. And that’s why we’re seeing chief experience officers, that title popping up a lot more these days. The whole concept that customer service is not a department.

Joe: [00:07:45] And something that’s really interesting to think about related to e-mails: what are the quality of the e-mail addresses that you’re collecting? I’m sure everyone here has got like a primary one they use. Maybe it’s their full name, you know, at Gmail or something. You’ve always got this secondary one as well that you use for junk mail. So it’s always been interesting for me to think about, “are you collecting someone’s primary e-mail address that they check all the time, or are you talking their junk e-mail address?” And that’s really hard to measure, but I just think it’s an interesting concept to think about.

Joe: [00:08:08] So back to loyalty ladder. I had someone recommend to me a year ago to try to a FitBit. So I’d been trying the brand and people around me are wearing them, so you know, I just decided to buy one. And then I began using the app, and I really enjoyed it. And it would prompt me to give feedback and overall improve my data, make sure it’s correct and everything. So I wore it enough, I invested time into it to try it. I then thought, yeah I’m getting use out of this, I like looking at how I’m doing. So I started to go in and adjust my data. I had earned their trust started tweaking my numbers, entering water consumption, all those kinds of things. So at that point, I knew I wasn’t gonna buy another health/fitness watch or wearable. I was loyal to it. And then finally, since I was enjoying so much I decided I wanted others to try it. So it was right around Christmas time. I started to give it as a gift. And I thought to myself, “wow I’m really an advocate FitBit because I’m willing to give it to friends and family and tell them that this is worth them investing their time in and trying.” So that’s more of a hardware product example. Software played a large role in that, but it doesn’t always have to be something major, either it’s the little things along the way that move you up and down the loyalty ladder.

Sean: [00:09:10] Totally agree, Joe. Each interaction you have with any business, it’s a step up or down. I’m a little disappointed, I have to say because I clearly wasn’t on your Christmas list. But I’m impressed.

Joe: [00:09:19] No no no. When’s your birthday by the way? I gotta…

Sean: [00:09:24] Hahaha. So I got another story. So this is not just a software product problem, this is also a cultural problem for business, and the reality is if the business culture is not aligned with the software product, you’re not going to be able to truly earn a loyal customer because they have to have a consistent experience across all of your channels, in your store or wherever they are, not just with the software products. But the software is a good place to start.

Sean: [00:09:44] So I have a little story to tell about my mechanic. So I bring my car in for standard brake pads. I’m not, you know, expecting anything major to happen, but at the tail end of the day he’s working on my car, my truck actually. It’s a GMC Acadia and I’m on my way to pick it up and get a phone call. He broke the brake lines. And you know, in my younger years I had a similar event happen and the mechanic told me that the brake line was broken and he found it when he got in there, you know, and created a very bad experience and I never went back to that mechanic again. This guy says, “you know what you’re on your way in. Why don’t you just come on in and I’m going to let you borrow my truck so that you’re not without a car because I know that your wife’s driving you in and you need the car. I’m going to take care of this. We made the mistake, it was a total mess that we created, we’re going to take care of ya.” So just think about that think about how that made me feel. And we all have this experience where we have a service disruption in what’s expected from a business that we’re hiring. In theory, I will never use another mechanic, right. I’m always going to go back to this guy because he knows to be honest, because that’s a big key piece of trust formula, which we’ll talk about another episode. He knows that if there is a problem and he stands behind it, I’m going to trust him forever into the future. I mean, service disruptions are possibly the biggest opportunity that you have for generating that story that your customers will tell when they ultimately become advocates, right.

Sean: [00:10:53] People do not expect you to be perfect, but they do expect you to have integrity. And the reality is, integrity is rare. It’s rare, it’s memorable, but it’s cultural too, you have to fit it in your business’s culture. Software should support that, but you still have to build it into your business culture around your products and services. I’d even go as far to say you could stage bad experiences to help boost your clients over that advocacy threshold.

Joe: [00:11:18] Hmmmmmm….

Sean: [00:11:18] But that would be a little unethical. Probably a bad idea. So anyway, we believe in the power of this cascade: trust, loyalty, advocacy. We’ve even built workshop exercises dedicated to getting everybody on the team kind of aligned with getting our customers to advocacy as quickly, as effectively as possible, without manipulation. But we’re going to save those conversations for future episodes as well. All right, so Joe, let’s talk about trust. Let’s talk about how we define trust and what it means in the context of a software product.

Joe: [00:11:44] So yeah, this is trust in the context of the product we’re talking about. It’s not trust in the way a lot of people think about it, like, “I trust you so I’m okay with marrying you,” or, “I trust you with my life.” This is trust in the context of our users behaving and using my products in the sense that show that they trust it. We usually look for this in, are they willing to make an initial investment in the product? And this can come in a variety of ways. It may be an investment in a monetary term, like they’re willing to give money for the product. It could be an investment in data or information.

Joe: [00:12:19] And so some examples of this would be, you know, signing up for e-mail list. Your email address is very personal to you. People live in their email all day now. And to get spammed by something is a big deal. People actively avoid it, and if they don’t trust a brand or a product, they will not give them their email address or they’ll give them a fake one. But in this case that is an investment. Facebook, for example, you are trusting them to some extent because you’re willing to give them certain information about yourself. Your birthday, for example, there has to be a trust aspect there because your birthday is used across many other software products to be part of your identification. So we are looking for that also in e-commerce products with, are you willing to buy a product, even one, even if it’s for five dollars? Are you willing to do that and take that leap? And then as we talk through loyalty, which is coming up next, you know, we’ll learn about, how do we see that trust evolve into loyalty? What are we looking for there? And I know Sean, you’ve been thinking about loyalty lately and how that transforms from trust, and what that looks like.

Sean: [00:13:14] I also think time is one of the things that you could measure trust for. Like they may tell you who they are but it doesn’t necessarily mean that they trust you yet. Like if you have a product that it takes a long time for you to earn enough trust for them to actually sign up for your service or sign up for your product. It may be the amount of time spent learning about something. The key is that it’s an investment. It’s that first initial investment that we think should be the measure of trust for any given product, and it’s generally time, money, or data.

Sean: [00:13:42] On the loyalty side, I think it’s a little more complicated and a little more simple at the same time. When I say that, I mean it’s a little more complicated because it’s so different for every different product. Daily active users, for example, is one of the measures that Facebook looks at for loyalty. And for other things, like online research portals, they often use a monthly active usage metric of some sort. But it’s more complicated because you have to figure out what is the appropriate regular expected usage from that particular user that would indicate that we think they’re loyal. You may have to measure it over time and try to figure out what we think that expected usage is. You know, and every software product is different. So you have to figure out like how often would we expect them to come back and reuse our service to indicate that they are loyal users?

Sean: [00:14:25] Again, our definition of loyalty means that they come back to our product and use it again because they want to. Self-determined: not because they have to use it, not because we’re forcing them in order for them to get some discount or something manipulative down the road, but they’re coming back to use our product because they choose to, right. That means something important, it means that they’re not looking at any other competitive solution for this particular problem in the world that they’re solving. So that’s how we measure loyalty. It’s an indication of their recurring expected usage of our software product. And we want to get the most people from our ecosystem into that regular state of usage. Right, Joe?

Joe: [00:15:01] Yeah. And then finally, you know, if you’re lucky enough to get them that far, which a lot of companies are not; they don’t know how to turn loyal users into advocates, well you have to give them tools, you have to empower them to be advocates. And by advocates, they’re essentially a mini sales force for you. They’re out there, spreading the word, basically. You know, word of mouth, it’s negative. But for when it’s positive, it’s typically an advocate-type situation where you’ve just had a great experience and you want to tell the world. You want to tell your friends, your family, you want to recommend them on Facebook. You want to tell whoever you can, and the second you hear someone asking for a product or service. If it’s a need, you have that example ready to tell them about. And not only that, but if you ever hear anyone badmouthing that product or service or company that you’re an advocate of, you defend them. You step up to the plate and you say, “that’s not true, I actually had a good experience with them.” And you try to fight that fight for them. And so, they become an extension of your company essentially. They don’t work for you technically, but for all intents and purposes, you want to be treating them as if they do. And collect feedback from them, give them the first chance to give feedback. These are prime candidates for beta testing and collecting feedback from, so pull them in and empower them to be a voice for you.

Sean: [00:16:03] Right. They clearly care about your product at that point, right. If they’re an advocate, they’re self-motivated to invest in its future. Feedback, sharing, all those sorts of things. Those are clear measures of advocacy and that’s what we’re after. Like, for every product. Like we’ve talked about ad nauseum at this point. So what are some examples of some specific products that you use, Joe, or that you’ve thought about, and what kinds of measures would you think you’d put in place that would indicate that the user trusts the product, the user is loyal to the product, and ultimately that the user is an advocate of the product?

Joe: [00:16:33] Sure. So for trust, you know, we like to look at kind of first line metrics. You know sign ups, are they willing to create an account? You know, regardless of how much information the account is asking for. But you know, that’s another thing you can look at is, “are we more likely to get more accounts if we just ask for a first name and an email rather than a first name, last name, and email?” That’s all things you can work out, but we’re looking for, “are they willing to invest their data in that sense?” In terms of time, “are they willing to use the product for a few minutes and invest into it to get better results out of it? And then monetarily, we look at, “how can we make that first purchase the easiest?” It can be, “how can we get them to do a trial and then sign up, or just make that first purchase maybe at a discounted rate or some other way to at least pull them in and get them to try it?” That’s all something you learn through the user feedback mechanism of figuring out why they’re not taking that first step. Because before trust, there is that awareness factor of, “do they even know about your product?” So once they get there, we can figure out, “how do we get them to build trust with the product?” And make sure we’re approaching it properly.

Sean: [00:17:32] Right. We were in an interesting conversation yesterday with a very large insurance company, and that question came up. Like, “all right so we want to earn trust, we all agree this is important. But where do we start with awareness? Like how does marketing fit into this whole thing?” And the reality is, if you’re running a business, you’ve got to be thinking about that. What does your lead generation funnel look like? But what you’re doing out there in the market to bring people to the product is not quite in the domain of like, what can the product do to actually convert these people, right?

Joe: [00:17:59] Yeah, and in certain industries even, they come with these misconceptions attached to them because they’ve been beat up in the press, or they’ve had something happen years ago that’s no longer the case. And you may have to work harder to build trust with your users. If that’s the case.

Sean: [00:18:12] Right. But I think it’s really important to distinguish the difference between marketing and when you get that customer to your product and actually earning trust with the product.

Joe: [00:18:21] Mhm. You know, when we’re doing sales pitches, we hear that there’s maybe a competitor to us, but they’re better SEO, or they’re better at this or that. We say, you know, it’s important: you’ve got to get people to your product. But once you get them there, the experience and how you move them into being advocates is even more important. And that’s what we try to focus on is that actual product, that user, and getting people ultimately to be advocates.

Sean: [00:18:42] Right. So let’s talk about incentives and paying people to become advocates. Right. A lot of companies are doing this these days. It’s like, “here’s five bucks if you prefer a friend in.” “We’ll give you free product for a month if you sign up five your friends to do this.” Multi-level marketing stuff. So lots of companies are finding success and the ability to grow through that but then are seeing a lot of fall off after they get these customers in the door. Or they’ll switch to another company because they’re not actually thinking about and earning and keeping trust and loyalty, right.

Joe: [00:19:10] Right. So you know, I’ve got friends who, they’ll refer me to things and it’ll say, “you’ll get 20 bucks and so will your friend if you sign up for this.” And the first thought I get is, “well my friend didn’t really refer this because they think this is a good fit for me and they just want the 20 bucks.”

Sean: [00:19:25] Or the free product, right.

Joe: [00:19:27] Exactly. Exactly. It does come with strings attached, essentially. So it kind of reduces the authenticity of that referral in that case.

Sean: [00:19:35] Yeah and I think that’s where there’s a key distinction between what’s marketing and what’s actually building trust with the consumer.

Sean: [00:19:42] Awesome. So we’ve talked about ways to build trust, loyalty, and advocacy, and ways to measure trust, loyalty, and advocacy, and we plan to dedicate future episodes to each one of those specific things. Today was really about being cognizant of the fact that users do go on a journey with you, and you do have ways to measure and influence how they move up the loyalty ladder into advocacy. And we have to always take into account, and again, we’ll have another episode on this at some point in the future; it’s their journey and not your journey. And this framework, this trust, loyalty, advocacy thing, is a way for us to have a healthy, thoughtful approach to measuring our long term relationships with the customers that we’re bringing into our ecosystem.

Sean: [00:20:17] We believe that brand advocates are the lowest cost path to your next hundred-year relationship, right, that long-term customer relationship that you are really after. Not just the short-term sale, or the short-term lead gen, or you know, getting that customer in the door. You shouldn’t take it for granted. We have to treat each and every advocate as though we intend to maintain a hundred-year relationship with them, because we believe in our heart of hearts that an inspired customer willing to defend and promote our brand is the best investment that any firm can make.

Joe: [00:20:44] Yeah, so any time you see any sign of advocacy or potential advocacy, someone’s wanting to be an advocate but can’t for some reason, your spidey sense should go off and you should pursue that with everything you’ve got. Once you get those true fans, those advocates, that is how these Silicon Valley companies scale so well, is they take that and they run with it and they build advocates very, very fast.

[00:21:09] (music).

Joe: [00:21:09] So Adam, thanks for joining us today. We have Adam Bates here from Amazon.

Adam: [00:21:12] Hello.

Joe: [00:21:13] First thing we should really just say is congratulations to Amazon on a hundred million Prime subscribers. That is pretty insane.

Adam: [00:21:20] Yeah, they’re getting up there. Hahaha. No end in sight.

Joe: [00:21:24] So Adam, can you tell us a little bit about what your specific role is at Amazon and what you’re working on day-to-day?

Adam: [00:21:29] So currently, I’m head of product management and UX for a business unit of Amazon that is focused on the household consumables shopping experience and how we come up with ways to really grow users’ participation in that set of products. By consumables, we’re really talking about things that you might typically buy at the grocery store or other stores related to that. I think folks often think about, like fresh products, you know, produce that sort of thing, and that’s certainly one aspect that’s available in certain markets to Amazon customers, but there’s also a much wider set of products that’s available. Snack foods, pantry type items, household cleaning supplies, your personal care supplies, you know, beauty and grooming sorts of things that Amazon offers. So yeah, we’re looking at what we can do innovation-wise to get people to partake in more of that from Amazon.

Joe: [00:22:16] So Adam, how did you get your start in building software? Was it something that you consciously made a choice to go, do you kind of fall into it, how did that all happen?

Adam: [00:22:23] You know what, I’ve always been a nerd. I’ve loved messing with tech and using it to solve tough problems, or simplifying various aspects of life. I definitely was around back before the days of Spotify. So you know, like things like getting your music files into MP3’s, moving them around, organizing them, sharing them, making party playlists, you know that sort of thing. Those are the sorts of things I would find myself just piecing together my own solutions or scripting or doing things to just make that optimal. So, you know, even before it became my profession, you know, I’ve always heard myself doing things like that. I think in general, I love creating experiences that transform how we live and work. I love diving in a new domains and, you know, getting into the minds of the users, understanding what’s really, truly important to them, things they don’t even know how to ask for. And then with, that sort of clarity in mind about what you can make their lives. Then from that, designing, building, and refining products that continually blow their mind in terms of what they do for them.

Joe: [00:23:19] Cool. So Sean, in our episode today we’ve been talking about trust and loyalty and advocacy and how we build that. And everyone thinks about Amazon as one of their favorite products, typically, and you know, there’s a lot of hard work. That is not an overnight success. And you know, we wanted to start to hear firsthand how they make that happen.

Sean: [00:23:36] Yeah, so Adam, you guys are arguably one of the best business models in the world out there in terms of earning advocacy from your users. You guys have figured it out. I mean you guys have so many great things going on. I know you have a corner of the Amazon world that you’re responsible for, but you know we’re hoping to get a little bit of insights and maybe share with our audience some of the things that you guys have learned about earning advocacy, earning loyalty from your users, earning trust, and maybe even learning a little bit about how you guys measure that stuff.

Adam: [00:24:01] Yeah absolutely, it’s critical Sean. You know, I think that’s one of the things that Jeff has really emphasized, even from the early days in the 90’s there. It’s like, “hey puy the customer first, build that trust, and maintain it, protect it at all costs.” And you know, that’s really endured today. Every team, every person working on it really has this drilled in and is focused on, first and foremost, protecting that trust. And I think that’s really how Amazon’s been so successful at building that loyalty.

Sean: [00:24:28] Certainly shows in your results. So you’re familiar with the loyalty ladder, right.

Adam: [00:24:35] Yeah, yeah. I’ve heard you talk through it before.

Sean: [00:24:38] So the goal is to earn advocacy. And if everything we do is kind of lined up with that, there’s things that you have to do to get to that point, right. So one of those things is loyalty, which we believe means that they’re using your product when they have that problem to solve. They’re not even searching for a competitor. They just, “solve this problem, I’m going to your product.” And before that ever occurs, you have to have trust, and you just hit the nail on the head there with the Bezos strategy of making sure, no matter what we do, we never fracture trust, right. Always take care of the customer. So when you think about your individual products, how do you guys measure, like what do you consider KPIs, how do you measure those things?

Adam: [00:25:13] I think that there’s a lot of ways to look at measurement. Certainly the business outcomes we’re looking to get, but, you know, that always translates to some CX outcomes, customer experience outcomes. And I think every undertaking we have has that defined very early in the process. So that when we’re pulling in the broader team, everyone knows what behaviors we’re going to look at, what cues from our customers we’re going to look at, that are telling us that we’ve hit success. We can get into a few specific examples now, or later in the discussion, that show you some of the specific instances of this.

Joe: [00:25:47] Yeah, we can definitely do that. So how do you actually use metrics? You know, a typical kind of question is, “what’s your day look like? What’s a typical day like for you?” So how are you looking at metrics on a day-to-day basis, or keeping in tune with them frequently with what the customers are doing?

Adam: [00:26:00] Sure. Well, every project has what I call a product realease score card. So, you know, we’re always releasing, and each release has some objective in it. Whether it’s a CX change that’s intended to drive more add-to-cart, or to generate more subcategories shopped, or whether it’s a behind-the-scenes technical improvement that’s designed to adjust latency in the experience so our customers are getting a more speedy response, every release has those. So we’ve got dashboard setup for each of our products that has these metrics in it. So everyday it’s one of the first things I look at. It’s like, all right how are these changes trending…? Some of these are hard to see from a day-to-day perspective, you really need to get a trend over a week or longer. It’s basically woven into every release plan and our daily routine.

Joe: [00:26:49] So Adam, just, you know, piggybacking off of that, a lot of customers, kind of a fad out there is Net Promoter Score. That tends to lean more towards advocacy because, you know, the whole idea of the question is, “would you recommend this to a family member or a friend?” What is your opinion on Net Promoter Score?

Adam: [00:27:05] Well, I think one thing that is undeniable is that you need some form of sentiment measure. But, you know, sentiment measure alone is not enough. You’ve also got to have a large-scale behavioral measurement to go right alongside that. You need both. We’re talking about loyalty here, but, you know, speaking of product loyalty, I’m a big advocate of Kerry Rodden’s HEART metrics framework that she put together while she was at Google. It’s a framework for defining metrics to measure your UX. You know, you’ve got you know behaviors or things that you think about looking at, and you put measures in place very intentionally, things like task completion rates, error rates, adoption retention figures, you know these are all bread and butter. But I find that they’re incomplete if you don’t know how your users are actually feeling about your service, so I tend to start with behaviorial first. But a sentiment measure is the best way to find gaps in those behaviors that you’re measuring or how you’re interpreting them and to verify that what you think of as success in your behavioral metrics actually translates into user happiness.

Adam: [00:28:00] So NPS is one way to capture that and I’ve found that it can be very effective if there is close alignment between your overall product and service and the specific sentiment that you’re looking to measure, and one specific example might be, say, Google’s inbox product. It’s something that a lot of people use and are familiar with. Now the vast majority of a users impression of that product come from a narrow set of experiences: a feature set that’s well bundled and contained within something that they clearly understand as inbox and they don’t confuse with adjacent Google products. Now if you pop an NPS survey there routinely, you can get a pretty good confidence that longitudinally, you’re understanding the effects of feature service changes to sentiment. However, take a very different example: you know prior to Amazon, I worked for a company called Paychecx, and they had a broad suite of service and product offerings, a mix of technology and live service. And we found that it was very difficult to isolate user sentiment of the tech products from their sentiment of the live experiences they had with service givers. So to capture the sentiment on the digital experiences we were creating, we had to turn to other means. You know, one thing that we used were micro questions placed in the context at the moment of usage. For example, “hey, how satisfied are you with this new feature, scale of 1 to 5,” so that we’re sure that we’re getting input that’s really tied to the sentiment that we’re looking to measure.

Adam: [00:29:26] Another method I’ve used with success and that provides much more nuanced sentiment quantification is UEQ, the User Experience Questionnaire. It’s basically a series of questions that users can go through on their own and it’s designed to get quick top-of-mind responses on things like how intuitive is a product, how attractive is it, how innovative does it feel? So it goes a bit deeper than simply, you know, “would you recommend it?” Now the tradeoff is that it’s multiple questions. So it’s designed and its instructions say clearly that respondents should respond to the questions quickly with the first thought that comes to them, so it doesn’t actually take long to finish. But you know when you go from one question like in an NPS survey, to several, you’re limited in terms of when you can present it. But again, that said, clear feedback on sentiment around some of those things I mentioned: attractiveness, intuitiveness, ease of task completion, dependability, is it exciting to use? Etc..

Adam: [00:30:21] You know, I might add one more point of sentiment measurement in like single air quotes here, and that’s anecdotes. This is certainly a very Amazonian thing and something that I identify with is that PMs and designers should make it a regular practice to seek out anecdotes, you know, to ask users or create mechanisms in-app to collect what they think about something specific, a specific experience, and to collect unstructured, open feedback that can be looked at. This is a highly effective way to find blind spots in your other measurements and really to say if what you’re hearing in an anecdote after anecdote is not supported by your other large scale measurement, you may have a measurement or interpretation gap and you really need to start digging to figure out what it is.

Joe: [00:31:02] Awesome, awesome. And for anyone who doesn’t know, HEART is an acronym. It stands for happiness, engagement, adoption, retention, and task success. And we’ll put a link to that in the show notes.

Joe: [00:31:13] I was at a conference recently and they were talking about, “how do we measure our software experience, or you know, how do we measure it across all our channels?” Because in the loyalty ladder, we talk about moving people up and down the ladder, but moving up on the ladder can also be online and offline because you could have a great online experience with that and have a terrible experience when you call into customer service. So measuring across, it matters. It’s great to pinpoint the experience metrics, but you can also use that feedback from the customers. You know, “I love your online product, but my god I try so hard not to call in and avoid that.” It’s kind of the whole customer service is not a department of philosophy. You know, looking for feedback everywhere. So that’s awesome.

Adam: [00:31:50] Yeah exactly. So you know NPS can be good to give you the big overview of your service and where to dig from there, and then you know, if you’re looking for more focused feedback on aspects of that service, that’s where some of these other ways of measuring that sentiment could come in handy.

Sean: [00:32:07] Cool. Just out of curiosity, I’ve often heard, So NPS measures what they might do in the future, right, so this is what their sentiment is around what they might do in the future. I’ve heard a lot of arguments about it being also interruptive, right. So the only way to really capture it is to interrupt your consumer in whatever they’re doing to collect that feedback. What are your thoughts on that?

Adam: [00:32:26] It’s certainly a balance. You need feedback, but yeah, it’s certainly possible to nag users too much to the point where they get annoyed. There’s a variety of ways to present, you know, when you get that feedback. There’s times where users can choose when they give it if you’re delivering it via email or some other notification mechanism to trigger that. If you’re going to a mode where you’re collecting that, like in the midst of an experience, you know, you’ve certainly got to be mindful of how you’re doing that. I think the days of these interstitials that take over your experience and try to get you to dive in and fill out the survey are probably gone. That’s probably a little too disruptive. But you know, you see any number of experiences now that raise it in a bit more subtle way, a more diminutive way that makes it feel less imposing, less interruptive, and personally with those sorts of mechanisms, my teams have had good success with upping our participation rate in the surveys.

Sean: [00:33:19] So you had also talked about culture, sort of. You know and how it can’t just be the whole service side of the business and the whole touch point, the human aspects of the business. How do you implement those sorts of, or it is something that you are even responsible for thinking about, how do you implement those cultural changes? Or how do you use the data that you get in context to make an impact on the cultural changes in the organization? Is that something that you have had any experience with?

Adam: [00:33:44] You know, I think in talking about some of the reasons for these measures and large-scale quantification, whether it’s of behavior or of sentiment, you know not only to direct you and your teams in terms of what strategy and adjustments you’re gonna make or innovations you’re going to go after, but also in influencing other partners. And obviously Amazon is a very large organization, and each business unit, each team has their priorities; the things that they’re focused on. The best way to influence those priorities to affect a much larger change is with data. I think that’s one thing that Amazon really values. It’s like, “hey, put the customer first, do what’s right for the customer.” But in explaining what’s right for the customer, you’ve really got to come to the table with some data that supports that. So I think it’s pretty much expected that you’re going to come up with a data-backed case to support your viewpoint of what’s best for the customer.

Adam: [00:34:32] Very cool. So just for our audience, you know, if they’re newer to metrics and they’re new to these kinds of concepts, let’s go back to your earlier offer of, what are a couple of metrics that maybe are peaking your interest rate now that you’re looking at, or maybe were quite surprising the way the feedback came back. How are you using that to make a couple different decisions now?

Adam: [00:34:51] So yeah, you were asking about some of the metrics that we were looking at recently, perhaps even tied to loyalty. How do we measure loyalty? Now typically the best metric for this is a retention metric, and these can come in various forms. You’re typically looking at this for specific cohorts, so customers that first started using the product during a certain window of time on a specific version of your software, or maybe tied to a specific campaign that brought them in and it made them customers. It’s the sort of metric that can lose meaning if it’s too highly aggregated, you know, so like across your entire user base it may get diluted.

Adam: [00:35:27] But in one example, we are looking at how do we create more of an ongoing dialogue with shoppers to better help them maintain the inventory of things that they need in their household? So as you might imagine, something like that not only is an experience you encounter when you come to Amazon deliberately, but also when you’re off Amazon and we’re reaching out to you to engage you in employing to engage tactics. So in that sort of example, I think some of the things that you might look for are, “how many notifications are they actually responding to, and does that get degraded over time?” You know, that might tell you something about the quality of the messages you’re putting out there when you’re notifying them, you know, “are the things you’re notifying about just not that important to them?” Those are the sorts of inferences you begin to make.

Adam: [00:36:12] In another example, we are looking at, “hey how do we go out and help specific brands acquire new customers to what they’re doing?” And one of the tactics we use in that is by sending unsolicited products to them on the behalf of brands. Now in that sort of example, it’s pretty straightforward. We look at the activity of those customers really by a few different groups, you know, by the targeting strategy, by the campaign that went out to try and acquire them. And then once they’re on Amazon, how many brand orders are they placing in a 90 day window. That’s a sign of their loyalty. What does the frequency distribution look like for the number of orders from folks that receive the sample? And that really begins to give us a feel for, “all right, yeah. How effective is that particular tactic in attracting loyal customers versus ones that may just try a product once or twice?”.

Joe: [00:37:00] Yeah that’s awesome. So what do you typically see it takes to earn someone’s trust to order something like that that maybe they used to get from the grocery store but to start to get it from Amazon?

Adam: [00:37:10] Great question, and it’s something that a lot of people look at here. Some things that are probably pretty intuitive when it comes to people’s tendencies to buy these sorts of products online versus at the physical store. I think some of the things that we see from users is their, “I’m already at a physical store. It’s easy for me to just throw this at my cart.” Or, “I ran out of this and I need this right now. Maybe I don’t have Prime Now available on my market, so I can’t wait a day or two for Amazon to send it to me, so I am going to go down to the store to use it.” So I think two of the big things that come out of that are, one, you know, how do we make our customers more aware that Amazon carries these things in addition to books and electronics and other things that they might traditionally turn to us for. And then, two, how do we help them with that problem of not running out of things? Are the things that we can do to help them more intelligently manage their household inventory and keep them out of those situations where they’ve got no choice but to drive down the store and pick some stuff up? So to generalize, it’s really identifying the unique problem that online consumable shopping can solve for them and coming up with unique experiences to address it.

Joe: [00:38:18] It’s such an important problem because if you really think about it, these are real problems for families. You know, running out of core household items, you know, maybe for their kids or really reducing stress day-to-day. I know there’s things that we run out of all the time and it’s like, “if we were a little more proactive of knowing we were going to run out of them, it would save us a lot of time and stress.”.

Sean: [00:38:36] All right. So let’s switch gears a little bit. Ask some questions that are related to the topic but a little off of Amazon. What other software products in the world are you personally an advocate of, and why?

Joe: [00:38:44] I think the examples that come to mind, Sean, are a combination of software product and sort of physical experience. You know, one I recently used was called Bird. It’s an on-demand the electric scooter service that was recently piloted in the Santa Monica, you know, Venice Beach area, I think San Diego as well. I’m a huge advocate of it now, a big fan of them. I like to tell anyone that will listen. It’s fun because they created a whole new fun way to get around a community. It’s so easy to sign up, easy to find a scooter, either with the app’s MapGeo feature or just by walking around, like Venice Beach, because they’ve invested in ample scooter inventory and have these things spread out like all up and down the coastal neighborhood there. And the nice thing about it too is when you’re done using the scooter, you can just leave it whereever. A combination of an app experience that just makes it so simple, and then the physical experience that’s blended pretty seamlessly with that.

Sean: [00:39:36] Oh that is a very cool idea. I can’t wait to try that out. Are they in other cities now or just Southern California?

Adam: [00:39:42] They landed a huge funding round, I forget exactly how much and who led that, but their plan is to rapidly expand from there. Another thing just kind of being a product guy and whatnot. The scrappy way that they put together that pilot, how quickly they executed it, and how quickly they are going from that pilot to expansion is amazing. You know, I think they really did this in a manner of 90 days or something like that. I really have got a lot of admiration for doing something on that scale and being able to move so quickly from it.

Sean: [00:40:12] That seems to be the way the world these days.

Adam: [00:40:14] Yeah. Hahaha.

Sean: [00:40:14] Gotta keep going faster, right. Faster and faster.

Adam: [00:40:17] Yeah, yeah. You know I’m a Fitbit Ionic advocate. So that’s their first foray into smartwatches in addition to fitness tracker. I got to admit, it’s probably not the most aesthetically pleasing watch that’s out there, but man, they really solved the problem I have with iWatch. And that’s the way that the battery lasts the whole week. You know, and it’s still got GPS. It’s got the ability to stream music without my phone from services other than Apple Music. It’s a very fitness focused smartwatch, so you know I like that they’re taking their experience in the fitness focus and branching out into a smartwatch that really matches with that lifestyle. It doesn’t make me carry a charger around with me, even for a day or two business trip. So I recommend checking that one out.

Sean: [00:40:59] Neat. I’ve had a Blaze for quite awhile, so I’ve been looking at the Ionic as a stretch into the more smartphone functionality and features. So good.

Adam: [00:41:08] One other example that comes to mind, and it’s not new by any means, it’s been around a while longer, but it is again a combination of app experience and real-world physical experiences. I’m a big advocate of Lyft, specifically over the other popular options like Uber. You know, from an app and driver experience perspective they’re not too different, right. But there is a set of other factors that have me advocating for them in favor of an alternative like Uber. I make a point of talking to the drivers, you know, whether I am in a Lyft or Uber, and drivers that drive for both almost to a person report that Lyft treats them better, has better incentives and training. And also, my personal experience with Lyft customer service is that it’s incredibly responsive and they seem to, again and this is with every interaction consistently, err on the side of the customer if there’s a question of dispute in terms of like what you were charged or whatnot. Whereas with another service, you know, I might not even get a response. So you know, that’s all there. I won’t even go into the management cultural issues that were popularized over the last year.

Sean: [00:42:07] I think that goes back to what we were talking about earlier that the culture is just as important as the software product. And if you don’t have a way to give the company that feedback and create a circular loop and really make sure that the culture is also creating loyalty, is building trust, is earning advocacy….

Adam: [00:42:24] Yeah.

Sean: [00:42:24] Lyft has clearly figured that out. I’ve had very similar experiences and have pretty much exclusively switched my preferred ridesharing service over to that, and again, it is largely because, I mean they’ve changed the market. That whole ridesharing thing has changed the market for personal transportation in so many ways, but the culture is just as important as the quality of our software products for sure.

Adam: [00:42:42] Agreed, yeah.

Joe: [00:42:44] And again, for anyone who may be just getting into software development or is maybe trying to become a product manager or somwthing like that, the number one piece of advice is probably, you know, experience overall for your product, and your service, and your company, you know, experience at every level is what’s winning the customer battle these days. And something I’ve been wanting to ask you Adam is, what do you think is the most single reason products fail, and what can be done to avoid that?

Adam: [00:43:09] I think the single biggest problem with product failure is focus on the part of the team that’s building that product. You know, a brilliant idea of this innovative business model but it having so many moving pieces and complexities that you’ve really got to align assumptions stacked on top of assumptions to get this thing to work. So for someone that’s leading a product, it’s like, hey make sure you’ve got crystal clear focus on what problem you’re trying to solve with that product for your customers and for your business. And you really kind of take the, “hey what’s the one thing we could do with this product this week, this month, this quarter, this year such that everything else that we might have to do is easier, or or maybe even unnecessary.” And man, I find myself quoting from a book I’m an advocate of here in that, um. But I think that’s critically important and teams that don’t have that focus are spreading their attention and their time too thin.

Adam: [00:44:05] I also find that for teams that don’t have that clear focus from leadership, the individual participants on the team building that product, you know what, they may be the most genuine, most skilled folks, but if they don’t have clear focus, what they’re seeing as the priority for the team could be vastly different from the other parts of the team. You know, you may have your software lead’s head in one space while your design lead’s head is in another, and you know, you may not be running in the same direction. I’ve found that when teams can focus, get on the same page, and be very specific and clear about what success means, that the results you get from that have a much higher probability of making the impact you want.

Sean: [00:44:39] All right. So I got a question for you. What do you think is the next industry that’s most ripe for disruption through experience?

Adam: [00:44:46] I think medical care is one that if you know anyone that is anything but a 100 percent healthy, you’re acutely aware of just how difficult it is to find quality providers, get them to listen to you, and have some structure around the programs that they’re putting together for you. And if your care spans multiple providers, which it usually does, again if you’re anything but 100 percent healthy, getting those providers to coordinate, collaborate, look at each other’s work and come up with something cogent for you is a huge problem. So just because of some personal experiences…

Sean: [00:45:20] That’s a big one.

Adam: [00:45:20] That’s huge right. And not just from a monetary perspective, but man talk about how you could change someone’s life that’s dealing with chronic issues or issues that are really hard to diagnose. You know like, several friends or family members that have been dealing with issues over years still don’t have clarity on like what the path to the happiest life is when it comes to their health. So I think from a customer problem perspective and from a business opportunity perspective, just so in need of disruption there. And I think there’s a number of services that are starting to attack it, you know, particularly in some of the metro areas, but I’m very excited to see what can be done there.

Sean: [00:45:58] My theory is that, because this is probably the way it needed to evolve, but the healthcare system, everything about the healthcare system is focused so intensely on the health event, right. You have a health event and everything is engineered to solve that health event. The staff is trained, the culture’s built, the facilities and the tools are all built, just to manage you through that event. And they don’t really take into context the fact that we’re human beings, we’re born and we live a life, and we’re gonna die someday, but hopefully way out in the future, and really where we need the most help in healthcare and in our health is in all those in-between spaces. In between when we have health events that the systems are set up to support.

Adam: [00:46:34] Yeah.

Sean: [00:46:34] The first one that set out is going to be the one that wins the game, I’m sure. It’s got to be about putting the patient and their life at the center and not the health event, and only to the degree at which the health events have to be solved for patients. We have to change how the entire system is structured. It’s a big change but it’s probably the most ripe industry. Looks like Amazon’s getting into that space now.

Adam: [00:46:54] That’s not one in terms of details that I’m really in a position to talk about. But you know, yeah, from the inside view in terms of the way that they attack these sorts of problems, I certainly am excited to see what we’ll do.

Sean: [00:47:03] Fascinating.

Joe: [00:47:04] You alluded to a book earlier that you quote from a lot. What was that book? Do you give it out as a gift, give it to your team?.

Adam: [00:47:09] I think the book has been around, for a little while at least. It recently came into my sphere and I’m totally caveating here because it’s one of those books that I think has a lot of kind of fluffy marketing kind of speak to it, but is still absolutely worth reading because focus is still such a problem. It’s a book called The One Thing by Gary Keller and it gives some very practical ways to achieve focus. You know, all the way from long-term strategic planning to what you’re going to be working on today to make those long term plans come to be. Again, some very practical, teachable ways to align groups of people to think in this focused manner. So yeah, It’s one I definitely advocate and say it’s worth sticking through use of the fluffier parts.

Joe: [00:47:50] Yeah, we’ll put that in the show notes as well. But you know, again, just to talk about focus again and how it relates back to our lives in software. We see our scrum teams trying to fix their problems thhrough retrospectives, not trying to tackle their problems all at once but one by one, focusing on that. And then in the product world, we see a lot of our clients or customers, you know, maybe they’re great at getting customers but they’re not great at keeping them, the retention, the loyalty part of it. So I’m going to check out that book and maybe give it was a gift this Christmas, so thank you.

Adam: [00:48:18] Yeah, no worries. To get a customer experience right, like holistically, throughout the entire phases of a customer journey is hard. It’s not something you can spend a week figuring it out, and then ship, and then you’re done. You know, like things come up and you’ve got to respond to them and it can often take a lot of time from the principal members of your team to figure out like, “hey why is this a problem?” Plus, you know, then figuring out what you do about it. So that’s why the clarity and focus from a product leader is so vital.

Adam: [00:48:46] You know, like how many different areas are you asking your team to spread their attention across? The more aspects of your strategy or the more diluted that your focus is, the more distinct problem areas come up and then you just don’t have the time to give the quality of thought into them. And so what comes out the other end is oftentimes not a great experience.

Joe: [00:49:08] A quote by Ron Swanson from Parks and Recreation, “you can half ass two things or whole ass one thing.”.

Adam: [00:49:16] Hahaha… Sage advice, sage advice.

Sean: [00:49:21] So I’m curious Adam, how do you guys handle customer journeys and customer journey mapping? Because obviously, as you deploy things, you learn that the customer’s journey is their journey, it’s not your journey. We tend to think that our customers will follow this pretty little path to get things done in the context of our software products, but the reality is that their lives are complicated; they’re busy. So how do you guys handle mapping and keeping those journeys updated?

Adam: [00:49:43] You know, I think there’s a few primary ways to look at it. The first one I think kind of maps well to the loyalty letter concept that you’ve talked about before here. These are the phases the customers go through with you over time. They’re researching a product or a solution to use, they become a new customer, they become a repeat user. Then they go through a stage where they deepen their usage and are using more of your service, not just in repeated fashion, but in a broader fashion. And then, as you mentioned, you know, they advocate which can happen along that journey. And also generally, there are users that don’t follow that happy path. So I look to make sure that we not only understand those phases, but understand why users aren’t following what we think the natural progression is, find out why that’s occurring and then to make the most of it. And then, I think once you’ve got a good handle on that overarching framework, you find that that overarching framework does not change routinely. Like if you’ve got a good understanding of your customers, you know, at that level, you kind of understand those major phases. So you spend a lot of time looking at the sub-journeys that transition them from one phase to another and what really affects those changes. What are those inflection points that happened with them, and then how do you improve those? Then of course, you can get more micro. What are the individual tasks within each phase and the steps with each? And that’s where you really start to get some nuanced behavioral metrics that show how you’re doing on that, so you optimize there. You’re then kind of bubbling up and helping speed the transition from some of those more macro phases.

Sean: [00:51:06] I have one more key question and it revolves around that end goal. So we all agree, I mean the Holy Grail, any software product, is to have your customers talking about it as it’s the nuts. This is the thing you’ve got to use, right. So I define advocacy as the point at which they invest personally in the future of your software product. That’s a key indication that this person’s an advocate. And that could come in many different forms. It could come in the form of a simple Facebook like, it could come with them actually having a conversation with their friends and family, right. NPS is kind of designed to keep some sort of sentiment measure on that. You know, we look for the actual behaviors and try to help trigger them; that’s going to come up in a future episode.

Sean: [00:51:44] Another key thing that we’ve found is an advocacy behavior is feedback. Like if your customer takes the time out of their day to give you a feature idea, or a product idea, or just to simply say, “thank you, this product has improved my life,” that’s also an advocacy behavior. Is it something that you guys measure, is it part of your product success metrics? Just curious.

Adam: [00:52:03] Yeah well when you look at the Amazon retail experience, one of the most fundamental blocks is a review. And that’s so fundamental to the Amazon ecosystem, right. And just like you said, yeah, if a customer will take the time to leave a review, it’s a signal that they’re invested in a couple of ways. You know, one they’re invested in the Amazon platform to leave a review on it. They’ve got some investment or or strong response, you know, that could be a negative one, on the product itself. But those are so valuable and they feed into the algorithms in terms of how people discover new products, when products are recommended and surfaced, and that sort of thing. And they’re signals to us behind the scenes of who are really customers that are engaged at Amazon as a platform? And I will say you maybe too, you know, a less structured way, I’ve done this at various products including Amazon.

Adam: [00:52:47] You know, looking at activity that can happen in social media is also interesting, you know, usually less quantifiable than something that’s within your ecosystem that you could measure, but you know, healthy signs that people are reacting strongly enough to take the time to tweet about that and have that be something that becomes part of their online identity.

Sean: [00:53:06] Right. We’ve talked about the investment that they make. So they make this investment in the future of your brand, they’re making an investment, use the example of reviews, the more investment they make, the more invested they are, right. Because they’ve sunk this cost in; they’re showing that they care about this platform. It’s powerful. The value you get out of an advocate is immeasurable as well. Think about Apple versus Android, right. They defend you in the marketplace. They are your army of salespeople helping you to talk about your product and sell for you. They are also, and I think we tend to underutilize this, they are going to be the best source of feedback for how we can make the product even better.

Joe: [00:53:40] True.

Sean: [00:53:40] And it’s an opportunity. You know, they honor when you reach out to them ask them for that sort of help in a non-aggressive, non-interruptive way, just as an invitation.

Adam: [00:53:49] Yeah, agreed, agreed.

Sean: [00:53:51] If you think about it, another way to look at it is they’re putting their personal name and their own seal of approval on your brand. It’s so valuable.

Sean: [00:53:58] Thanks for joining us Adam. You never disappoint. You’re an unbelievably valuable human being in the world and I value the relationship that we’ve been fortunate enough to have over the last several years.

Adam: [00:54:08] Thanks Sean and Joe. Appreciate you guys having me, always a pleasure to chat, talk shop, and especially about products and tech and customers and that sort of thing. So thanks.

Joe: [00:54:18] You got it. Thanks Adam. Talk to you later.