Episode 26 – Helium 10’s CTO Shares Insights Into Our Amazon Sales Estimator and Amazon Email Automation Tools

In episode 26 of the Serious Sellers Podcast, Helium 10’s Success Manager, Bradley Sutton, welcomes our Chief Technology Officer, Bojan Gajic back to the microphone for the third installation of “Tech Talks With Bojan” where Bojan shares insider insights into Helium 10’s Amazon sales estimator and Amazon email automation tools amongst other data-driven tidbits.

At Helium 10, we have always prided ourselves on the highly accurate nature of our data. Given the complexities of the inherent nature of sales estimates and other related data, coupled with our focus on serving our core seller demographics, we wanted to give you a peek inside the strategies and approaches we use to create a comprehensive data model that best serves what our customers need.

In episode 26 of the Serious Sellers Podcast, Helium 10’s Success Manager, Bradley Sutton, and Bojan discuss:

• 00:40 – Sales Estimates – The Mind Shift That Is Needed
• 01:15 – Sales Estimates, BSR, And The Process Of Data Analysis
• 02:50 – The Complexities Of Sales Estimates Data
• 04:10 – Our Approach To BSR
• 05:05 – Why We Have Never Published Case Studies – Validating Our Model
• 09:00 – Our Approach To Building A Model
• 11:10 – The Data That Our Tool Focuses On
• 14:15 – Validating Our Estimates – What Sellers Should Do
• 15:20 – Other Uses Of Our Model – Validating Its Success
• 16:20 – Potential Future Plans For Our Tools
• 16:55 – Steps We Have Taken To Alleviate Abuse Of Our Tools
• 20:10 – The Roadmap For Our Follow-Up Tool
• 24:05 – Requests For Features We Haven’t Implemented – The Rationale Behind These Decisions

Enjoy this episode? Be sure to check out our previous episodes for even more content to propel you to Amazon FBA Seller success! And don’t forget to “Like” our Facebook page and subscribe to the podcast on iTunes, Google Play or wherever you listen to our podcast.

Want to absolutely start crushing it on Amazon? Here are few carefully curated resources to get you started:

• Freedom Ticket: Taught by Amazon thought leader Kevin King, get A-Z Amazon strategies and techniques for establishing and solidifying your business.
• Ultimate Resource Guide: Discover the best tools and services to help you dominate on Amazon.
• Helium 10: 20+ software tools to boost your entire sales pipeline from product research to customer communication and Amazon refund automation. Make running a successful Amazon business easier with better data and insights. See what our customers have to say.
• Helium 10 Chrome Extension: Verify your Amazon product idea and validate how lucrative it can be with over a dozen data metrics and profitability estimation. 
• SellerTradmarks.com: Trademarks are vital for protecting your Amazon brand from hijackers, and sellertrademarks.com provides a streamlined process for helping you get one.

Transcript

Bradley Sutton: Today we’re going to hear from Helium 10’s CTO, Bojan, who’s going to tell us everything that we wanted to know what goes into sales estimation tools such as that Helium 10 Xray Chrome Extension has. We’re also going to find out what upcoming features we have in store for Follow-Up.

Bradley Sutton: How’s it going, everybody? Welcome to another episode of The Serious Sellers Podcasts. This is our third edition of tech talk with Bojan. Bojan is our CTO here at Helium 10. And he always brings us some great insight from the development side, and today I definitely wanted to talk about something that’s been on my mind. I actually just did an AMA about it. It is about sales estimates. A lot of different tools are out there, including Helium 10 that gives sales estimates as far as what sellers are doing on Amazon. And, I think there’s a mind shift, Bojan, you and I have talked about that kind of needs to happen. And it goes back to one of your things that you’ve always tried to teach people in different aspects is ask yourselves why or what is your goal? What are you trying to do? And a lot of times the questions that people are making is going to change if they have this mindset difference or shift, right? So, can you talk a little bit, first of all, we see people coming out with different tools, they’re coming out with different case studies? For example, saying XX tools is XX percent accurate. Now, here is Helium 10, is this accurate? Now, we’ve had Xray for quite a while, but Helium 10, as far as I know, has never come out with some kind of case study or some kind of a comparison chart with other tools as far as the accuracy of sales as missing. Why is that?

Bojan Gajic: Other tools and people look at the problems spacing, and it can be deceptively simple. So making that the sales estimate is essentially attached to bestseller rank via seller number. Since BSR number is exposed by Amazon, when you look at the BSR and when you look at your own sales in relationship to that BSR, you feel as a seller, you might feel that it’s relatively easy to make an estimate. How BSR translates to a number of sales. So, BSR number is exposed. And if you look into it, the way Amazon calculates BSR is documented publicly. It’s not on amazon.com, but if you know where to look, you can find more details about how BSR is calculated. So on the surface, it sounds like a serial problem, and now more people are getting into data analysis and data science and the statistical methods are getting more evolved, and the computational power is cheaper. So it seems like something that’s really easy to achieve and it can be very deceptive. You look at your numbers, and you look at the BSR and say, okay, it’s so easy, I’m going to build a model that’s very successful and has high accuracy. The problem with sales estimation is not the model, the model that the science around that is pretty subtle, so that’s the trivial part. The tricky part is actually defined that problem spacing and recognizing what needs to be estimated. So, let’s say you have two products that are fairly stable BSR, and sales for those two products. You look at the third product that sits somewhere between the two that you are familiar with, and BSR with that product is right there. So say, okay these guys are selling this much. The problem is that BSR is not published in history, BSR is not published by Amazon. So unless you have access to that historical data, you won’t be able to make that accurate estimation. And on top of that, even if you do have access to historical data, some products will change categories, some products will be more, some products will– there’ll be changes on the Amazon side in the marketplace size that will make your history inaccurate. And that’s where the estimation kind of falls apart. So if you look at the entire marketplace and you try to make that estimation, you’re more likely to fail than to succeed on most of the products. So what we did, we focus on products that are relevant for our user base. So, Apple is unfortunately not our customer at the moment. So, making accurate estimation on products that are in top 10 or top 20 best sellers for a specific category like electronics, it would be nice, but it doesn’t really bring any value for our customers. So what we focus on is space between, let’s say BSR 1000, and 2– 300,000. That’s where most of the successful Helium 10 customers reside. If your product is not in top 200,000, you’re likely not making more than a sale or two a day. So to run internally, we do run the test going all the way back to your question, why don’t be published any studies. I can publish study today, I can run a study and I can make Helium 10 most successful tool out there. Obviously, I can do the same thing with the five, or six, or ten competitors. Publishing a study is a trivial task, and we internally have a– obviously, we look at our competitors, they have that friends and family, they have some accounts and some customers that are helping us validate our model. So we don’t– that’s another important point. We don’t use customer data to build our model. So–

Bradley Sutton: So that means we have tons of users here at Helium 10, but it’s not we’re going into everybody’s seller central account and checking what their sales are in order to build our model. That’s pretty much what you’re saying. We’re not doing that. Right?

Bojan Gajic: Correct. That’s would not be compliant with the intended use of that Amazon data, and we don’t really need that kind of data. We would have to build our model on a subset of our customer data. And then we wouldn’t be able to validate the model essentially. Plus it doesn’t really scale, cause you need to build a new model for each category, for each marketplace safe, and new marketplace comes up if all of the sudden you decide to run the estimation for baby category in Italy, we would need customers core for sellers who are successful sellers in Italy in baby category. And they would have to have decent spread in those sales. So, it’s just not just realistic to do that again unless you’re Amazon. So instead, we built different statistical models and we do use our customer who is participating in the program to validate our results. So those internal tests are performed daily and their scoreboards that are showing our results. And at this point, they’re fairly comfortable. I’m very comfortable with what we have. If I do spend time to create a study and publish that study, I’m essentially wasting my time and the reader’s time. Obviously, I will not publish a study that shows somebody else’s tool performing a lot better than my tool. So sure, let’s pretend that this study is already out there and then there’s a study, and we did a study and published it, and we’re the most successful and amazing tool. It’s just as valuable as no study.

Bradley Sutton: I had been working guys with Bojan for a while so I can pick up on his dry humor. Hope you guys basically understood that point though in case you guys didn’t pick up on it. Rewind it 15 seconds if you didn’t. But basically what he’s saying is, no other tool is going to go print some extensive study that shows that their competitors are more– performing better than their tool. But it very well could be the case. So let’s just pretend that we did make some kind of study like that. Well, what is that even mean? Doesn’t mean anything. But as everybody knows, Helium 10 is not going to produce some study that’s going to show Helium 10 coming in 2nd, or 3rd, or 4th place on accuracy. So, what’s the point? Everybody knows that any study that somebody is going to put on sales estimation is there going to be right there at the top. But again, that being said, where does accuracy play a role in their usage, Bojan? Because we’re not just going to say, hey yeah, we’re just going to throw any number as an estimate. These are just estimates, so who cares? I don’t want to say that accuracy is not important. Of course, it is important because, for example, if the actual sales are 50, you don’t want to be in a tool that says it’s 1000 because yeah, you’re going to make different decisions based on that. So is there actually a line that you guys on the development team have as a goal as far as what you try and keep? I’m sure you guys– there are the team test things internally. I’m sure every now and then, you may have mentioned there are some friends and family that show the information, so we can validate the data, but do you guys have a target that you work with as far as how accurate you want Helium 10 numbers to be?

Bojan Gajic: So, in data science. You don’t want to– it’s called overfitting. So, you were training your model on some sample. Let’s say you have a hundred results, and you’re training your model on those hundred results, a hundred estimates, under actual data, sales data points. You don’t want your model to be hundred percent accurate on that data because that usually leads to greater err for when you build your model. So instead you’re trying to build a model that’s relatively flat, relatively predictable, and you’re trying to minimize the error without overfitting. So on one end, I mentioned the not very successful products in the marketplace. So, if you have a product that sells between zero and one units a day, our model could say, okay, the likelihood is 60% that this product sold one unit. So let’s say, it’s 0.6 units a day. We can round it up and say, okay, this product sold 1 unit on that day. Or we can say nowadays sold zero. So, that in terms of percentage, that margin of error is large. In terms of units sales, that’s fairly insignificant. So when you look at the products like that, but they sell 10 a month or 25 a month, it’s not really that relevant if that’s not what you’re trying to emulate. You’re trying to find successful product, but also not products that sell hundred thousand units a day because you’re not, as private label seller, you’re not very likely to find that opportunity. And, build it to 200,000 units today. So, we focused on that farther markets where products are selling, let’s say between 5, and a few hundred units a day. And then we tried to minimize average error, so if you look at the ranges outside our target, you’ll likely find larger in our tools and some other tools. That’s– the thing is that you’re doing that intentionally because you’re trying to reduce the error on that middle segment that caters our customers. Also in some cases, you will want to look at specific product, but most of the time when you’re doing products research you’re looking at niche, you’re looking at that segment of products, collection of products and then you’re trying to analyze average sales or average reviews, or some kind of aggregated number. So we are not, by design, we are not trying to make our prediction hundred percent accurate on specific data set. That’s not the way you’re using the tool. One way where you might want to use the tool to estimate specific sales for a specific product is if you have one competitor that’s outperforming you. So you want to see how much effort, how big the investment you want to make to match that competitor. Maybe you want to spend more on PPC, or some other form of advertising. So if they’re selling more than you are selling, and you’re trying to validate if it makes sense to go head on and try to beat them. That might be a use case where you want to look at a specific product as the sales. Again, if you have a competitor like that, that probably means that this pretty stable product, with not much isolation in BSR. And that also means that our model, it will be very accurate for that use case. So, the early you’re looking for a specific product and if you are looking for a specific product, not out of curiosity, but out of business necessity. Our tool is very likely to give you a very accurate estimate.

Bradley Sutton: Okay. So basically in a nutshell what you’re saying is we know what our users are searching for. We have a lot of history on that and what spaces they’re looking at. And for those, it’s usually very highly accurate. Now that the times you might see Helium 10’s number might be off by a larger percentage, maybe, for example, there’s a product that only sells 0.5 units a day, but maybe Helium 10 is saying 2 a day, or something like that. So technically that’s 200, 300% off or whatever that is. Right? And again, in that situation, our users are not really searching for products, whether they’re doing product research, they’re not really searching for products or niches where they’re only selling half of a unit a day. So who cares? But on the flip side, you talk about Apple iPhone, or some Bluetooth wireless headphone, maybe in Xray and Helium 10 is saying it’s selling $1.5 million a month. But actually who knows, it could be 1 million, so oh my goodness, $500,000 different. But the same token, it’s not something that our users use or the vast majority of our users are trying to break into that exact category and be selling something that’s going to scale to one and a half million dollars right off the bat. So basically, by focusing on the great majority of the used cases, it actually allows us to be more accurate for our users. Is that pretty much the case?

Bojan Gajic: That correct. And, we’re not trying to think for our customers. So we’re just building a tool that supposed to provide another data point to Amazon sellers and not just Amazon sellers. Users still have to apply common sense. It’s kind of weather forecasting. So the forecast might say that there’s a 30% chance of rain tomorrow. You do want to look through the window, and figure out if you need to take the umbrella with you or not. So, in some cases, we are providing estimates, and those estimates should be validated, especially in extreme circumstances. If a customer sees that something is unrealistic, it’s probably data that’s coming out of various sources that are making that estimate look unrealistic. Again, we are validating our performance against our competitors and against actual marketplace results for the targeted population. So internally, if you’re not looking at Amazon bestsellers, we are not looking at Amazon echo, or iPhone’s, but we’re looking at a population that we actually want to help. We’re also validated by users who are using our tools in various ways. I keep hearing about new use cases almost every day. There are people who are not even selling on Amazon who is using our tools to validate market demand on Amazon, and then use that to source products for different marketplaces. So that popular demand for our tools is actually a validation of the quality of the tool. Also putting some pressure on our backend. So there’s a lot of attempted abuse of our platform. So it’s causing some issues for me on the backhand, but it’s also validating the success of our models. I’m pretty comfortable with where we are right now. Obviously, we do have people looking at it and they’re measuring and very constantly trying to improve. So, I’m not saying, Hey, this is the best it can be done. There’s always something that can be done. And, since the marketplace is changing, you want to stay vigilant. So, we do have people looking at that data and trying to build a more sophisticated model. We are trying working on forecasting models. We are working on various things around this space. So there’ll be more and there’s more that we can do. But the actual sales estimation that looks back requires a lot of data that we do have and requires a sophisticated model that we did build. So–

Bradley Sutton: Okay. Now you mentioned that the increasing popularity of Helium 10 and there’s actually, because of this, there’s been some abuse of the tools. Now, what are some steps that the team has taken to help alleviate the risk of some of that on our side?

Bojan Gajic: Right, so recently we had to go out a requirement for MWS token for one of our tools, Keyword Tracker. It was not a very popular move, and we debated that internally but it was essentially a requirement. So we’re trying to cater to mostly private label sellers but in general Amazon sellers and Professional Amazon sellers. So I mentioned that we have customers who are not Amazon sellers, and those customers are generating some load on our platform, which is fine. There are customers who are trying to abuse our platform, which is not as fine, so we’re forced to introduce these kinds of constraints and requirements to make sure that we are catering to our user base to the main target population, which is again, professional Amazon sellers. So if you’re not professional Amazon seller, we have tools, and we have programs and training programs that are designed to help you get to the status where it makes sense for you to register as professional Amazon sellers, but it was like $40 a month, which is not insignificant. But in the grand scheme of things, that’s the amount of day that Amazon provides and additional services that you get access to with that status justifies the investment. So, moving forward we might have to do a similar thing for some over the other tools. Mostly for that abuse prevention reason. In the case of Keyword Tracker, we did not really, we did not require customers to select products from their catalog. So you can– if you’re using Keyword Tracker, you can still do that if you want for whatever reasons, track performance of your competitor, whatever that might be. You can still do it, but you have to authenticate as a professional seller, and that authenticates you as a unique customer to Helium 10.

Bradley Sutton: Okay. So the companies who are agencies or consultants, maybe they don’t sell on Amazon themselves. A Helium 10 customer who fall into those categories. Perhaps they just manage keyword research or keyword tracking for their clients. Now they still can use our tool. It’s not that they have to be actual sellers on Amazon actively. They used to have a professional account with Amazon, but they can go ahead and add any products to Keyword Tracker whether it’s on their account or not. Correct?

Bojan Gajic: Correct. Yeah, that’s right.

Bradley Sutton: Okay, so now we have a few more minutes here. So it’s been a couple of months since we rolled out the Follow-up tool for email automation. Now we usually, at Helium 10 we don’t talk about what’s imminent or what’s kind of in store as far as new features or new tools, but I know you’ve already talked about this on Facebook and customer service is also mentioned this to our customers. So it’s okay to mention one thing that we are going to add soon is the use of images. Users will be able to input their own image. And I believe the reason why we’re implementing this is because that was probably one of the most or the number one thing that was asked for by our customers. Now is there anything else that you can talk about at this time? As far as, for our Follow-up users, our current ones or the ones who are considering using it. Is there anything that is also imminent that we are going to be rolling out soon as far as a feature in the coming weeks?

Bojan Gajic: So yes, that was I believe number 1 request, and we intentionally release Follow-Up with not all of the features fully functional in order to give our customers, and especially our paying customers a chance to influence the development and the roadmap of that tool. So we do that by asking questions during the design phase, but also in this initial rollout phase. The ability to insert custom image and ability to add a custom attachment. Those are the two updates that will be going out this week. The request that comes often is the ability to email to target existing customers with email marketing. And we’re looking at that. There are two issues with the ability for that feature that will allow you to email your existing customers. One is that data exposed by Amazon is supposed to be used for order fulfillment, and not retain past the 30 days. The other issue is that Amazon limits the number of messages that you can send. So, there’s a pretty hard limit in a number of messages that you can send per day to the Amazon platform. So we are trying to find a way to address that customer request without causing any issues, and getting Amazon to suspend an account for abuse. That issue of Amazon enforcement then, and especially privacy concerns are often overlooked by tools in this marketplace. But there is a real risk of getting customer account suspended by and not following the rules, and guidelines exposed by Amazon. So yes, image and attachment, those two things are coming.

Bradley Sutton: PDF attachment?

Bojan Gajic: So it’s documented and we will update our FAQ as well. There are few file types that you can use to attach, that you can attach to your message. So we will add support for that in the next few days and the plan to open it up to our free customers or not paying customers in the next several weeks. There’s more to come. The roadmap, again, our Elite and Diamond and Platinum customers, put a lot of items on our roadmap, but it’s open ended, so any feedback, any suggestions are welcome. There is no guarantee that they will fulfill those requests.

Bradley Sutton: There’s an example, a lot of people who ask us, hey, for review, can you let us know who is, which order that this review came from? We get requests like that, but just because it’s asked for a lot, it doesn’t mean we would actually do it because it’s actually, that would be something that could potentially hurt our customers, put our customer’s account in jeopardy if they’re going out trying to find who is the one who left them reviews. Right?

Bojan Gajic: Correct. And that might be a topic that abuse, and risk, and awareness in that space might be a topic for next month for us, because we looked at some of the requests, and we decided not to do certain things in order to avoid jeopardizing our customers’ accounts. Hopefully, there are things that Amazon is doing to strengthen the enforcement and enhance those privacy safeguards that they have, and that’s welcome. Most of us are Amazon customers as buyers as well, not just sellers or tool providers. So, as a buyer, you would not want your personal information used by third-party sellers to call you at night or send an endless stream of emails asking you for feedback, or to change whatever you– whatever review left. So, when you look at that, try to think about it as Amazon buyer or as a customer buying on the marketplace that’s perceived as a safe and secure. What are you willing to accept?

Bradley Sutton: Okay. So one last question. You, like myself, we’re pretty much native San Diegans now, but you actually originally hail from Serbia. So I’m a big basketball fan. You’re a big basketball fan. I would like to know who are your top three Serbian basketball players in history because, for those who don’t know, Serbia has one of the richest histories as far as basketball goes outside of the USA in the world. So who are your top three Serbian basketball players of all time?

Bojan Gajic: So Vlade Divaj, Dejan Bodiroga

Bradley Sutton: Bodiroga. Which NBA team did he play for? I don’t remember him.

Bojan Gajic: I’m not sure if he ever came to the US, but he was, they call him white magic. Just kind of racist. But, he was really creative flair.

Bradley Sutton: Yes. I think I know who you’re talking about.

Bojan Gajic: Now, I have Jokić in Denver.

Bradley Sutton: Oh, I didn’t even know he was a Serbian. That’s the joker, right?

Bojan Gajic: Yes. So those three guys will leave a mark. There are a few other players that have a big personality that I like. You had who used to play in the US, and Peja Stojakovic. So they are players that I like, they are players from that farther world that left some mark. Maybe even a bigger mark. But these three that I liked.

Bradley Sutton: Okay. But biggest personality right now as far as serving basketball players has got to be Boban Marjanovic. I’m going to have to say he’s the most interesting personality out there, right now.

Bojan Gajic: Yeah. He’s a fun guy.

Bradley Sutton: All right. So, thanks to Bojan for your time here. Actually, anybody listening who is going to the upcoming SellerCon conference, you guys are going to be able to talk to Bojan on about basketball or anything on the technical side as well. He’s going to be at our social event and walking around the conference. So please look for good looking Serbian guy and a Helium 10 t-shirt and maybe it might not be Bojan though.

Bojan Gajic: Yeah, I’ll be right next to that.

Bradley Sutton: You’ll be right next to that. But anyways, Bojan. Thank you as always for coming on here. And we’ll be in high expectation of next month’s Tech talk with Bojan.