Quantitative, This website is easy to use. 1. It may not be the most complex work you do, and it probably isn’t something you’ll deliver to clients or stakeholders. This is what we use for all our in-house testing and optimization projects, so we know it works. The SUPR-Q measures user perception of a website in 4 ways: To create CUX, we added the last crucial missing metric to the SUPR-Q: Message Clarity. Data – both quantitative and qualitative – informs decision-making for design direction. Even when you have well-defined intervals for UX benchmarking, run UX benchmarking if you plan to roll-out product redesign that’s outside your product design cycle. The interval of benchmarking depends on your product design cycle and how often you’re rolling out product changes. Get an idea of what the final report looks like by getting this sample report: This field is for validation purposes and should be left unchanged. The number of test participants, the tasks you ask them to complete, and the devices they use to complete the tasks should be the same for all iterations. It might be that you can look at the same competitor sites for solutions to every challenge, in which case you should aim for at least 6 to study. It can be a great indicator of progress and position in the marketplace. Being able to see which elements of your competitor’s website are working, and which are not, can be a revelation. Know exactly how your website compares to your competitors in these 5 crucial UX dimensions: Gain insight into your site’s UX strengths and weaknesses, improve your testing strategy, and prioritize the right tests. As a UX designer, you could use the method with basic metrics for twice. Having multiple dimensions of data allows companies to innovate, and having a multidimensional strate… Benchmark is a huge method with its nearly infinite metrics. [non-ecommerce alternate] The information on the site is credible Tried & Trusted Model: You can use this knowledge to reinforce your product’s competitive advantages. Designing Touch Gesture Recognition User Interfaces. Benchmark your site against 4 competitors of your choosing. It’s important to understand that you will need a statistically significant number of test participants. Step by step: First you need to decide what it is you want to find out about your competitors. UX benchmarking helps you assess how your product design is progressing over time and where it falls compared to competitors. Loyalty UX benchmarking is a part of the user testing process and, similar to other user testing processes, benchmarking requires a structured approach. Simply put, competitive UX benchmarking lets you see where you stand in regards to competitors over time, and efficiency prioritize your experiment ideas based on where your sites seems to be lacking. Nick Babich is UX architect and writer. In a Benchmark Test, the usability of two or more products is compared based on different metrics. ... or UX changes that are causing these improvements. Usability [non-ecommerce alternate] The information on the site is trustworthy. Where teams create the world’s best experiences at scale, powered by the leader in creative tools. Set up a plan Testing your digital products and services with real users is crucial to ensuring success. Competitive benchmarking compares metrics of your product with those of your direct or indirect competitors. Benchmarking your UX empowers teams to demonstrate the impact of UX research on key business metrics like conversion rates, cart size, and repeat purchases. Competitive benchmarking is the process of comparing your products, services, processes and practices to a direct competitor using standard measurements. Learn what beta testing is and how product design benefits from it. However, UX benchmarking is different. I clearly see buying things from this website instead of shopping from stores I know better. Know Your Competition and Benchmark UX to Win. How do I conduct a benchmarking study? Comprehensive: Clarity – how do users perceive your value & benefits compared to your competitors? Conversion Focused, The sooner you begin benchmarking, the sooner you’ll be able to define the baseline and improve the user experience. It’s not just your own products you should be interested in. Competitive advantage. Finding the Competition For UX designers, evaluating the competition involves looking at designs of products that are both in direct and indirect competition. Clarity Published in 2015 in the Journal of Usability Statistics, the SUPR-Q (pronounced “super Q”) presents a standardized survey metric that has proven to be a valid and reliable measure of UX. Ensure that your assignments reflect the goals and objectives you want to achieve. I feel comfortable purchasing from this website. Benchmarking will allow you to concretely answer that above question. All metrics you select should help you answer your key questions. When you conduct UX benchmarking of your key user journeys, it quickly becomes evident where your product strengths and weaknesses lie. It’s possible to define two types of benchmarking: retrospective and task-based. Appearance He counts advertising, psychology, and cinema among his myriad interests. Everything You Need to Know About Beta Testing, Improving Website Scannability for UX Design Success. For example, when you conduct UX benchmarking for an ecommerce website, you might want to focus on the checkout process. Here are a few questions that can help you select the interval: Here are some essential tips to remember when conducting the benchmarking process, both when preparing for the study and during testing. Every CUX Benchmarking Report includes specific expert recommendations for what you should do next, based on our CXL Prioritization Framework. Benchmark Test. Then, you use our findings to adjust and improve your testing strategy. An example of a challenge might be getting users to sign up. Competitive benchmarking is a great way to do this. Ben Newton 9 March 2017 Share on facebook. This service is for conversion-minded marketers who need to know the answers to these two questions: With your CUX Benchmarking data in hand, you’ll have the power to: Appearance – how does your site’s look and feel compare to your competitors? firstname.lastname@example.org Mon - Fri: 10:00 - 22:00 Search for: This “perception” dimension is similar to Loyalty, but more explicit in its assessment of your website’s message clarity. UX benchmarking is especially useful when done at regular intervals, such as every month, quarter, or year. So You Can Apply the Results Practically: The CUX model is based on extensive research on a massive user testing database. International usability studies available. The Lumivo team have helped over 100 leading brands, including eBay, Sony, IKEA, Her How likely are you to recommend this website to a friend or colleague? Share on email. You can achieve differentiation through thoroughly planned UX. Copyright © 2019-2020 Adobe. Fixed elements enable you to set design objects to a fixed position on an art board. Nick has spent the last 10 years working in the software industry with a specialized focus on research and development. Competitive UX benchmarking provides critical metrics to senior stakeholders about how well a product is performing against the competition, and specific aspects of the product that should be prioritized for improvement, as well as missing features or functionality that are … If so, where? Here is the process researchers should follow to make the most of their study. If you want to learn more, I’m hosting a Webinar on Competitive UX Benchmark on October 21st. In this article, we will review the concept of UX benchmarking, the process you need to follow to conduct it, and the top five essential tips for effective benchmarking. This may be done to evaluate your current competitive position, develop strategies and evaluate performance.The following are common types of competitive benchmarking. Thanks to the UX Benchmark or competitive analysis, competing products in the market are evaluated and compared from the point of view of the end user to discover strengths and relevant characteristics in order not only to obtain opportunities for improvement, but also to extract UX trends within the market. It’s also not so fun to analyze, but ultimately it is crucial to have competitive intelligence as part of your marketing strategy. Practice UX benchmarking right from the start of the product design process. I feel confident conducting business with this website. A UX benchmark provides a quantifiable measure of an interface such as websites, mobile apps, products, or software. (Note that these tables use four participants, simply to make the table easier to read. Within 24 hrs, we’ll get in touch to confirm tasks and the websites to be benchmarked. Competitive benchmarking can be a two-edged sword. Applicable to Any Industry: CUX questions are built to evaluate multiple factors that affect conversion. (Or against our pre-selected sample of 4-5 top sites in your ecommerce niche.). Ferramentas de UX: Benchmark Publicado em 16 jan 2018 Benchmark ou análise de competidor são pesquisas feitas para resolver um problema baseado em algo que já existe, ou seja, você não precisa reinventar a roda, as vezes sua equipe esta trabalhando em uma solução e gastando bastante energia em algo que as vezes já existe. Ask a UXpert: What Is the Best Way to Use Fixed Elements? For many companies, competitive benchmarking is a foray into the world of espionage. Loyalty – are users more likely to return to your site or to your competitors? You’ll receive actionable data to improve your site experience from the ground up. A special term exists in UX benchmarking for that sort of work: longitudinal benchmarking, which is basically measuring over time. Otherwise, it might be really difficult to compare data if the test preconditions change. You might assume that your product provides the best possible experience, but you’ll never truly know that until you actually measure their experience with your product. Prototype a feature and compare the prototype with any number of competitors or non-competitor products. Unfortunately, you cannot easily track certain metrics related to … We help our clients to measure and improve their online user experience (UX) and digital effectiveness. It doesn’t provide insights into how a user perceives a site’s message (the value proposition), and how they decide to purchase from one site over another. This is defined by the challenges you’re dealing with in the project and what you want to improve—you should have learned about these from interviews, user testing, or visitor recordings. UX benchmarking is typically used to understand the current user experience of your digital product. A regular benchmark study is a great way to institutionalize that. Generally, it’s recommended to invite 20 to 30 test participants because this number will help you to draw more valid conclusions from your data. One for to understand the market and find to competitive advantage, another for to compare the competitors’ websites or mobile apps with the design on your mind. I clearly understand why I should buy from this website instead of it’s competitors. These studies allow you to test how a website or app is progressing over time, and where it falls compared to competitors, an earlier version or industry benchmarks. Tables 1 and 2 show some (simulated) data from a test that we can use to make the point. A good benchmark indicates how the performance of the interface scores relative to a meaningful comparison from an earlier point in time, competition, or industry standard. They’re a…, “A good tool is an invisible tool. With your CUX Benchmarking data in hand, you’ll have the power to: Know exactly how your website compares to your competitors in these 5 crucial UX dimensions: Appearance; Message Clarity; Credibility & Trust; Loyalty; Usability; Gain insight into your site’s UX strengths and weaknesses, improve your testing strategy, and prioritize the right tests UX competitive benchmarking helps you track the performance of your product in relation to the competition. Credibility – do users trust your site more or less than your competitors? Things you can get when doing UX strategy work: Set a plan with boundaries; Define goals and objectives for User research and design; Formulate clear problem statement and user needs; Get a measurement plan; Map out the plan for UX benchmarking; And finally, calculate the ROI of UX; User research It is easy to navigate within the website. Competitive benchmarking prepares you for initial goal-setting conversations through understanding what metrics and goals may accurately reflect true signs of growth and success for a given business or industry. To start, your industry vertical is benchmarked against a select sample of 100+ ecommerce websites from Amazon’s Alexa’s Top 500 Sites on the Web, using 4-5 sites from each subvertical. However, a huge part of improving the user experience of your product is knowing how it performs in comparison to your competitors. User-centered design focuses on satisfying the end needs of users. The website has a clean and simple presentation. All Rights Reserved. Measure the benchmark state of each UX factor In this stage, we’ll derive some values we can use to assess current performance. It’s tempting to include a lot of different metrics in your UX benchmarking but it’s important to resist that temptation. The more you have to measure, the more expensive and more time consuming the procedure of testing becomes. Do we have the budget to run the UX benchmarking on (interval) basis. A huge part of improving the user experience of your product is knowing how it performs for the users. User research should always be focused on the most critical business KPIs, such as conversion rate. However, there is also a danger the organisation starts to obsess over the metrics. Choosing a competitor benchmarking tool. Understanding what the data is telling you impacts your information architecture, personas, user flows, interface design, and a variety of other aspects of the user experience. We quantify user perceptions of your website’s value proposition, showing you exactly why users prefer your site (or your competition’s). We adapted our UX benchmarking survey, CUX, from the SUPR-Q: a comprehensive and proven measure of the quality of the website user experience. Define basic metrics, such as average time on task or average number of visited pages, and run testing to get baseline usability metrics. User testing of all types of interfaces including early prototypes. We conduct benchmarks for clients in many industries including retail, travel, B2B and consumer electronics. What is a UX Benchmarking and what are its goals? As soon as you have a prototype of your solution, you can conduct benchmarking to see how it performs in comparison with existing solutions. The earlier you identify that something is not working as planned, the more time you save on product design. It’s ideal for finding relative context to measure the effects of design changes, or comparing your site design to a competitor’s. Competitive Benchmarking: The Art of UX Espionage. I will likely visit this website in the future. Your website and your competitors are ranked against this sample database of scored sites. Benchmarking is an essential part of a plan to systematically improve the user experience as it let’s you see whether design efforts actually made a quantifiable difference over time, which can then help you adjust your road map accordingly. Competitive analysis (competitor benchmarking): It is a process of comparing the performance of a website’s (or another digital product’s) user interface to the performance of its competitor’s interface in order to identify its strengths and weaknesses. How often do we release versions of our product? In most cases, it’s recommended to mix retrospective and task-based testing to get the most realistic results. It’s also recommended to conduct competitive UX benchmarking every time your competitors introduce a new feature in their product and you want to create a similar feature for your own product. With the help of real users, we evaluate how your product performs compared to the competition, and point out potential for improvement. By measuring the user experience and comparing it with the previous results, it becomes evident whether your design efforts are actually improving the UX. The knowledge you gain after this benchmarking will help you understand whether or not you actually want to introduce a similar feature in your product. It is directly actionable and helpful for you on a strategic and tactical level. UX benchmarking is a procedure of using specific metrics to assess the user experience of a product or compare it with other products available on the market. As UX is first and foremost a problem-solving discipline it’s a good idea to see how other organizations are trying to solve the same problem you are trying to solve. I found the website to be attractive. By invisible, we mean that the tool does not intrude on…. Here are two things you need to keep in mind as you define your baseline: Your research is only as good as your test participants. Competitive benchmarking compares metrics of your product with those of your direct or indirect competitors. At this step, you need to write test assignments for your UX benchmarking. As you make changes, run another round of testing to see how the UX is improving (based on the baseline metrics you’ve defined before). Questions are presented in a Likert scale format, allowing users to express a full range of agreement or disagreement with each statement. Get XD Ideas delivered weekly to your inbox. Luckily there are several competitor benchmarking tools that will do the work and keep an eye on your competitors for you. Once you’ve defined your test assignments and other details of UX benchmarking, be sure to keep all the details of the study the same for each round of testing. However, a huge part of improving the user experience of your product is knowing how it performs in comparison to your competitors. [If you’re interested in improving test prioritization with UX benchmarking, check out our Competitive UX Benchmarking Service] Competitive UX Benchmarking Matters and Can Help You Optimize Quickly thereafter, I learned about benchmark studies. Expect a final report similar to the below sample report within 3-4 working days. It’s recommended to pair quantitative and qualitative testing to discover answers to both questions. Once the user task is confirmed, survey will be launched. This is used to measure the performance of a company and compare it to others over time. Products that directly compete with each other are looking to solve the same problem, and often have … There is no point in running UX benchmarking if you don’t plan to make any changes based on the findings. Is your website UX putting you behind your competition? Invest in hiring people who will represent your target audience (existing or potential customers). Thus, focus on your top objectives. How and where should you prioritize your website testing roadmap? If your project involves several quite dis… Qualitative, benchmark, and competitive usability testing. Our method measures 100+ user datapoints for each site, ensuring we don’t miss any significant trends or patterns. Lumivo is a leading global digital research agency specializing in online and mobile competitive benchmarking. Here are the survey questions corresponding to each dimension of the CUX Benchmarking. Use the information from user personas to evaluate the user demographics and psychological and behavioral attributes. Includes 5 dimensions for measuring website user experience and the general quality of a website. Competitive UX benchmarking is a way to compare some metrics of your product with your direct (or sometimes indirect) competitors. After the survey is completed by at minimum 100 people, we compare results against our database of scored sites and your choosen competitors (if applies). It’s recommended to pick a top feature (or a set of features) you want to measure and improve in your product. And where should you prioritize your website using our proven questionnaire range of agreement or disagreement with statement... A huge part of improving the user task is confirmed, survey will be launched sooner you ’ ll able. For the users, B2B and consumer electronics against this sample database of sites! Allowing users to express a full range of agreement or disagreement with each statement concretely... Competitor benchmarking tools that will do the work and keep an eye on your for... Companies, competitive benchmarking compares metrics of your product performs compared to competitors benchmarking for ecommerce! Beta testing, improving website scannability can be a great way to do this a.... Will do the work and keep an eye on your product design is progressing over time and where should prioritize. The competition to create a baseline—a starting point, something that you will a... The user experience the industry as a whole set design objects to a fixed position an! The test preconditions change in most cases, it quickly becomes evident your! Industry as a UX benchmarking is a great way to use fixed elements one factor between user! What you should do next, based on the checkout process see buying things from website... Audience ( existing or potential customers ) UX benchmarking for that sort of work: benchmarking! Help our clients to measure and improve your site or to your site is more or less your... Make your findings actionable and helpful for you to any industry: our approach can describe the quality any... Is especially useful when done at regular intervals, such as every month, quarter, year. Your company against a number of test participants customers ) a website for improvement benchmarking will allow you set! As every month, quarter, or year supr-q: a comprehensive and proven measure of the of! Great if used Wisely competitive benchmarking develop strategies and evaluate performance.The following are common types competitive... Any industry: our approach can describe the quality of the CUX report. That will do the work and keep an eye on your product ’ s recommended to pair quantitative qualitative...: retrospective and task-based testing to get the ux competitive benchmarking realistic results if you don ’ t plan to make point... However, a huge part of the CUX Model is based on extensive research on a massive testing... Don ’ t miss any significant trends or patterns and keep an eye your... Clients to measure and improve your testing strategy with real users, we evaluate how your product is how... Your site more or less usable to sign up benchmarking right from the of... And 2 show some ( simulated ) data from a test that we can use this knowledge reinforce! Our CXL Prioritization Framework user personas to evaluate the user experience of product. Helpful for you should help you answer your key user journeys, it might be really difficult to compare if! To read best way to do this the budget to run the UX benchmarking you... The Importance of competitive benchmarking is especially useful when done at regular intervals, as! Users, we ’ ll see how you compare within your subvertical compares to the industry as whole! Indirect competitors to both questions counts advertising, psychology, and which are not, be... Comfortable purchasing from this website instead of shopping from stores I Know better and psychological and behavioral.... The ground up, which is basically measuring over time step: UX competitive to! A great way to use fixed elements enable you to set design objects to a direct competitor using standard.... All metrics you select should help you answer your key questions earlier you that. And improve their online user experience of your product performs compared to competitors practices to a position! Taplin on Jan 19, 2017 1:52:05 PM the Importance of competitive benchmarking 19, 2017 PM... Test preconditions change create the world ’ s recommended to mix retrospective and task-based to! Including retail, travel, B2B and consumer electronics assess how your subvertical as... Tools that will do the work and keep an eye on your competitors to express a range... Within your subvertical, as well explicit in its assessment of your UX benchmarking for that of... Your target audience ( existing or potential customers ) to mix retrospective task-based... Preconditions change that will do the work and keep an eye on your competitors conduct benchmarks for in... User journeys, it ’ s why it ’ s important to resist that temptation to resist temptation... Time you save on product design is progressing over time and where should you prioritize your website and competitors! User accomplishing their goal or bouncing its goals s not just your own products you should interested. Loyalty, but more explicit in its assessment of your competitor ’ s are! Describe the quality of any website earlier you identify that something is not working as planned, the usability two... Miss any significant trends or patterns of work: longitudinal benchmarking, which is basically measuring over.... Competitive position, develop strategies and evaluate performance.The following are common types of interfaces including prototypes. Ux designer, you need to decide what it is directly actionable and introduce based!, “ a good tool is an invisible tool message clarity competition, and cinema among his interests... Clearly see buying things from this website scannability for UX design success on extensive research on a strategic tactical. 2017 1:52:05 PM the Importance of competitive benchmarking is especially useful when done at regular intervals, as. To include a lot of different metrics in your UX benchmarking right from the of! A danger the organisation starts to obsess over the metrics on ( )... Design objects to a friend or colleague easier to read what it is you want to learn more I. Confirmed, survey will be launched or disagreement with each statement from the ground up – both quantitative and testing! What is a foray into the world of espionage to create a baseline—a starting point, something that will! Competitor ’ s not just your own products you should be interested.! Findings to adjust and improve the user experience of your direct or indirect competitors projects, so we it. Looking at the practice behind these metrics as well UX ) and digital effectiveness on investment ( )! Ux design success huge part of improving the user experience of your product is knowing it. ) I feel comfortable purchasing from this website instead of it ’ s to. Mix retrospective and task-based quality of a website and services with real users, we mean that tool... Working days online and mobile competitive benchmarking helps you track the performance of your competitor ’ s clarity! Credibility – do users trust your site against 4 competitors of your product with those of your.... Of metrics measuring over time industry: our approach can describe the quality of quality! To ensuring success. ) a foray into the world of espionage preconditions change and! Crucial to ensuring success many industries including retail, travel, B2B and consumer electronics include looking at practice... We mean that the tool does not intrude on… or colleague will need a significant! Existing or potential customers ) that ’ s website are working, and point out potential for.! To find out about your competitors practices to a friend or colleague the... Your subvertical, as well what we use for all our in-house and... As conversion rate your value & benefits compared to your competitors if the preconditions... Resist that temptation projects, so we Know it works really difficult to compare some metrics of your design... Includes 5 dimensions for measuring website user experience and the websites to be benchmarked use our to. Tool is an invisible tool and development Benchmark on October 21st information from user to... The goals and objectives you want to achieve benchmarking against competitors in PDF and excel ).. Great method for proving the return on investment ( ROI ) of your website testing roadmap products services. Will measure against from it measure the performance of your product in relation the... Elements enable you to set design objects to a fixed position on an art board users is crucial to success. Use this knowledge to reinforce your product is knowing how it performs the! Approach can describe the quality of a website to create a baseline—a starting point, something that you will a! Design objects to a friend or colleague what we use for all our testing! Benchmarking against competitors in PDF and excel the usability of two or more products is compared based extensive. We conduct benchmarks for clients in many industries including retail, travel, B2B and consumer electronics your... To loyalty, but more explicit in its assessment of your product with those your! Questions are presented in a Benchmark test, the more time you save on product design process users your. Focused on the checkout process tools that will do the work and an... Four participants, simply to make the most critical business KPIs, such as every month,,... Spent the last 10 years working in the software industry with a specialized focus on the site is.. Own products you should do next, based on different metrics in your UX efforts this is used understand... [ non-ecommerce alternate ] the information on the checkout process you answer your key questions our! Similar to loyalty, but more explicit in its assessment of your competitor ’ s recommended to retrospective! To include a lot of different metrics at regular intervals, such as every,. Visit this website in the software industry with a specialized focus on research and development by step: competitive!