Tips for starting your segmentation

Step One: Understand your Problem

Figure out what you want your segmentation to do by asking yourself about key business issues that need to be resolved:

• Do I need a tool to help my sales staff match products and services with clients?
• Do I need to build a more effective prospecting or lead qualification tool?
• Can I build a tool to help plan my media spend?
• What does my marketing collateral have to say to resonate with different niche customers?
• How do I assess strategies for new market development?

These needs and more can be addressed using segmentation research as a foundational resource for a multitude of business solutions.

Step Two: Do Your Homework

Use existing syndicated market intelligence and market experts to assess the extent to which a market is undergoing rapid change or is susceptible to volatile shifts in behavior:

• Highly fluid markets require current primary research to develop effective segmentations;
• More stable markets mean you have an opportunity to repurpose existing research or adapt pre-existing segmentation research to new business solutions.

Step Three: Moving Forward

How to get your segmentation project started once you know what you want to do:

• Start with conceptual or qualitative segmentation exercises to leverage internal expertise among your sales and marketing staff. This will give you ideal-types that inform research designs, and will help you understand what information you need for your business solution.
• Identify and test key segment differentiators; those characteristics that enable you to determine to which group a customer or organization belongs.
• Think through how segment profiles or characterization will support your business solution. Understanding the concrete links between what you need a segmentation to do and the content of your segmentation is the greatest predictor of success.
• Interview consultants and research companies to understand how adept they are in designing research and constructing segment based tools that are informed by research (i.e., primary or secondary).

Originally presented as part of the Oregon Chapter of the American Marketing Association Marketer's Toolkit Lecture Series on September 11, 2007. The original presentation can be found at the following link:

Powerpoint

Monte Carlo Simulation Workshop on Price Elasticity

For those of you who have requested information on my user workshop at the March 2007 Decisioneering Crystal Ball conference, you can find the orginal powerpoint and spreadsheet documents at the following links:

Powerpoint
Spreadsheet

Unfortunately, simulations in the spreadsheet require a licensed version of Crystal Ball.

For additonal resources see the following references:

R. Scott Evans, David Dickinson and Aaron Lee. "Checking those qualitative pricing inferences: Using simple Excel-based Monte Carlo designs to build confirmatory pricing curves from focus group results." In Management, Accountability and Research – The Quest for the Objective Truth. ESOMAR Publication Series, (CD-ROM). ESOMAR: Prague, 2003

Van Westendorp's PSM

A Black Box Diatribe: Caveat Emptor

In a competitive market, business savvy research consultancies package and template much of the work they offer clients. They do this because it lowers the cost of their services, offering better prices to clients and motivating their own productivity through higher margins. Efficiency is gained by having a simple boilerplate solution that can be pushed down to less experienced consultants and requires no customization. However, what is good for the consultancy is not always good for the client.

Designing boilerplate tools that generate solutions or output often makes great sense from the perspective of efficiency. Why reinvent the "wheel"? Who can complain as long as some of this efficiency is passed along to the client at a discounted price. However, caveat emptor - buyer beware. You are completely beholden to your consultant or market research vendor unless they are WILLING and ABLE to "open the hood" on the methodologies used to generate your business solution.

As a client, you should insist on the right to critically evaluate the solution offered by your consultant. Just as importantly, you should be able to work closely with your consultant to evaluate the extent to which a boilerplate design really meets your need. In other words, methodological openness on the part of the consultant permits the client to negotiate an appropriate level of customization that may be required to fulfill expectations. More succinctly, the client should never allow "blind faith" in a black box to over-ride the right to critically evaluate an approach offered by a researcher or consultant.

In my experience, refusing to show one's methodology often indicates something suspect. When I have delved into such secret methodologies, I have usually found that the almost mystical qualities purported by the owner are nothing more than simple mathematical operations. These carefully guarded secrets claimed to give accurate readings of brand value, price elasticity, or win/loss drivers. In instances where statistical functions fed the tool, the statistical applications were often so convoluted or misaligned that any interpretation of the results was highly suspect or even blatantly misleading.

Much to my annoyance, I have had to go through this process on a number of occasions over the past 15 years. In many instances, those that hide behind the secrecy of a "proprietary methodology" really have something to hide.

Good methodologists and statisticians are like most professionals. They like their work to be seen. Like any other professional at the top of their game they want their brilliance to shine among peers and clients alike. Good research companies that pride themselves on their research expertise will want their methodologists to shine as well. They will also welcome informed criticism from their clients - which only results in methodological improvement - and hence a more satisfied client. Moreover, this is part of a consultancy or research company's intellectual capital, upon which is built credibility and capability.

So if you are willing to accept the "black box", then caveat emptor.


Consultants and the Black Box

1. Black boxes often come with inexperienced consultants who cannot assess the relevance of their tool or even know how to modify their product to meet a client's special needs;
2. Zealous consultants using a black box design, of which they have little understanding of the fundamental premises and assumptions underlying its use, may be prone to draw unwarranted or misguided conclusions;
3. Senior consultants who suffer from analytical atrophy because of a longstanding dependency on black box designs created by someone else, are more concerned about a sale than having a keen interest in the client's business problem.

caveat emptor.

Measuring "Importance": Issues with Predicting Behavior

A recent article assessing measurement techniques created some interesting discussion in GCR's advanced analytics group. The article by Keith Chrzan and Natalia Golovashkina, published in the International Journal of Market Research (Vol. 48: 6, 2006), purports to assess a number of methods for measuring "stated importance". As with much of this type of academic literature, it is simply the reification of what many advanced practitioners have already deduced in the course of their daily endeavors. So what came out of our discussion within the advanced analytics group was a more disturbing insight as to how easily the discussion of measurement and inference becomes obfuscated by poorly thought out relationships.

For our clients, who often have little time for the pedantic debates of methodologists, the issue raised is the extent to which measured "importance" provides legitimate predictors of customer behavior. In response, Dr. Cruz has written an open post illuminating a number of issues that Chrzan and Golovashkina fail to understand in their original article. His response I have posted in full.

____________________________________________________________


Open Response to Chzran et al.

By Dr. Mike Cruz, GCR



Market researchers frequently try to measure importance, e.g., importance of attributes to a brand or product features to a purchase decision. Chrzan and Golovashkina (2006) assessed which methods of measuring importance perform the best, and their results are interesting and informative. They report that maximum difference scaling (MaxDiff) is the best, a conclusion I agree with based on my own experience with the other methods tested: Rating scales, Q-sort, constant sum, unbounded scaling and magnitude scaling.

Chrzan and Golovashkina assessed stated importance. One measure of performance was the correlation between stated and derived importance which is the extent to which attribute or feature performance predicts satisfaction (or customer loyalty, product quality, etc.). In other words, derived importance is the ability to predict outcomes of interest. Chrzan and Golovashkina implicitly assume that the goal of stated performance measures is to replicate derived importance measures, that the two are different paths to the same result. Hence, the higher stated importance correlates with derived importance, the better the measure of stated importance.

I disagree—stated and derived importance are fundamentally different concepts. Here’s an example: TVs come with remote controls. Is a remote important to consumers, i.e., would anyone buy a TV without a remote? Not likely—a remote would have some stated importance. Inclusion of a remote, however, would not predict which TV is purchased because inclusion does not vary. Put another way, a remote is a necessary but not sufficient condition to TV purchase decision. Necessary conditions will always have stated importance but may not have derived importance.

A second reason to doubt that stated and derived importance should correlate highly is the inherent irrationality of human decision making (the academic literature on which is vast, but see Gilovich, Griffin and Kahneman, 2002 and Schneider and Shanteau, 2003 ). Research on decision making has long shown that attitudes and beliefs are often only weakly correlated with behavior, if at all (for a review, see Wallace, Paulson, Lord and Bond, 2005). What’s important in a car? Performance, safety, price. What car did you buy? The red one! In short, stated importance reflects what people believe and feel, whereas derived importance incorporates every other factor that determines behavior.

Finally, stated importance is affected by norms and social desirability. Socially acceptable attributes tend to have higher stated importance, but drive behavior only weakly. For example, good gas mileage and low emissions may be rated as important in a car, but that has not stopped many Americans from buying SUVs and sports cars.

Why should anyone care about the difference? The easy answer is that may be harder to order the measures of stated importance from best to worst than one would think. More importantly, differences between stated and derived importance are informative, not diagnostic of the effectiveness of the measures. When interpreting data and making business decisions, understanding the interplay between stated and derived importance is critical. To make sense of that interplay often means going beyond the importance data, whether to other variables in the study, past data or relevant qualitative data.

I should also point out that Chrzan and Golovashkina compute derived importance using rating scale data. To the extent that rating scales lag behind other methods for measuring stated importance, they almost certainly lag behind those methods when measuring performance as well. In fact, the shortcomings of the rating scale data are noted extensively by the authors who had to abandon standard regression techniques to get good derived importance scores. As the authors rightly point out, though, MaxDiff takes longer than other methods and is not always practical.

References

Chrzan, Keith, & Golovashkina, Natalia. (2006). “An empirical test of six stated importance measures.” International Journal of Market Research. 48:6, pp. 717-740.

Gilovich, T., Griffin, D.W., & Kahneman, D. (2002). Heuristic and biases: The psychology of intuitive judgment. New York: Cambridge University Press.

Schneider, S.L. & Shanteau, J. (Eds.) (2003), Emerging perspectives on judgment and decision research. New York: Cambridge University Press.

Wallace, D. S., Paulson, R. M., Lord, C. G., and Bond J. (2005). "Which Behaviors Do Attitudes Predict? Meta-Analyzing the Effects of Social Pressure and Perceived Difficulty." Review of General Psychology, 9, 214-227.



____________________________________________________________


Dr. Cruz's Bio

Michael Cruz, PhD, Managing Director, Analytics. Michael Cruz directs both the qualitative and quantitative aspects of custom research engagements including proposal development, survey design, analysis and presentation. He is responsible for innovation in research design, the development of analysis plans tailored to client needs and the translation of analytic output into actionable recommendations for business. Michael has expertise in a broad range of analytic techniques including segmentation, structural equation modeling, nonlinear models and curve fitting, bootstrapping and Monte Carlo simulation.

Michael came to GCR from Gartner where he was a research director and worldwide head of forecasting for GartnerG2. Prior to joining Gartner, Dr. Cruz was a faculty member in the Department of Communication Arts at the University of Wisconsin-Madison. He has published numerous articles on leadership, group processes, decision making and statistics. Michael holds a Bachelor of Science degree in mathematics and masters and doctoral degrees in communication from Michigan State University.

Small N Designs

I was recently asked if I had any ideas on how businesses could do effective market research, both inexpensively and without jeopardizing the predictive power of the data. My response is that for some business questions there is in deed a more cost effective methodology, and one that does not sacrifice analytical power. This approach falls under the general rubric Small N designs.

Rationale for Small N Designs

For traditional research, the greatest expense is data collection and project management. This is ironic, given that the best prescriptive elements of any research come out of the analysis, which often represents a small fraction of the overall cost. This traditional formula has only changed slightly with the advent of cheaper data collection options, such as online panels, offshore call centers and more effective CRM resources. The general expense ratio of data collection to analysis remains roughly the same. Moreover, in the competitive fixed price universe of the current market research industry, the substantive efforts to remain competitive often means skimping on the analysis. Any real change in the cost ratio must rely on more advanced statistical techniques that can ensure a rigorous predictive capability when using smaller sample sizes.

Small N designs use specialized techniques such as bootstrapping, hierarchical Bayesian modeling and Monte Carlo simulation to determine the statistical significance of patterns in smaller samples. These are not new techniques and are currently found in a wide array of disciplines and are frequently embedded in many standardized market research methodologies that require data imputation (i.e., newer clustering and SEM algorithms, choice-based utility calculation, missing value imputations). Now these techniques are not only feasible for traditional analytical approaches. For those who understand the fundamental difference in how these statistical techniques handle uncertainty, there is a great opportunity to extend their application to business problems that would normally be relegated to costly traditional market research solutions.

Specialized techniques like Monte Carlo simulation are particularly germane as a market research strategy for companies that have very focused or narrowly defined markets and business questions – where traditional large sample designs are not feasible or far too expensive. The Small N design permits companies to model behavior or forecast trends in a quantitative and statistical environment, which is not open to more traditional statistical techniques that are very sensitive to sample size.

In other words, you get the rigor and power of statistical analysis, without the cost associated with larger sample sizes – you get the analytic power without the data costs.

Key to understanding this approach is knowing in what situations Small N designs are appropriate. Mapping a highly variegated or heterogeneous market usually requires large sample sizes. However, working with consultants who understand advanced analytical techniques can help determine how to adapt these techniques to a particular business problem. They can help you focus on key homogeneous market slices where a small N design would be both useful and meet the need for strategic information.

The bottom line: you can get more bang for your research dollar by making sure you are talking with consultants who can leverage many of the powerful analytical tools that permit effective Small N designs.

SMB and IT Financing: IBM's 2007 PartnerWorld

I recently attended IBM's 2007 PartnerWorld as an analyst. While giving me special access to some of my sponsor's internal clients, it also offered constructive insight into the direction IBM is pursuing, and by extension a direction that may have broad impact in the industry.

As these events go, PartnerWorld was both entertaining and informative. As much as the abandoned buildings of St Louis and the muddy Missisippi were depressing, Sam Palmisano (IBM's CEO) and his leadership team were energizing. As IBM's press release suggests, the next few years will continue to see Big Blue drive forward as a key connector and facilitator. Contrary to the media buzz following Palmisano's speech in the Tuesday general session, the real insight is not about targeting SMB (small and mid-sized businesses). The story is that IBM is finally getting its act together in the SMB space after knowing for so many years that this is a fertile market.

Apparently Steve Solazzo et al. have finally transformed the IBM selling culture into something that can attend to SMB. Now partners and IBM sales teams value the 30k deal and expect to focus on higher volume and smaller contracts. In conjunction with this transformation, the product development teams have purportedly scaled products that are suitable for SMB needs and budgets. I will let the more specialized analysts comment on whether or not the products and sales effort are really where they need to be.

An important sub-theme emerging from this focus on SMB (and emerging markets as a secondary note), is the role of IBM's Global Financing Services. Acting as a development bank, financing services will play a key enabling role in targeting both SMB and emerging markets (i.e., Eastern Europe, India and China). Similar to the path forged by the financial services divisions of the leading automotive manufacturers, IBM will bridge the needs of business and the often inadequate services of more traditional commercial banking. IBM's financing services may just be the engine needed to be drive a burgeoning SMB strategy to fruition. Once the sales teams and product lines are in place, IBM's financing services will be the key deal broker. With financing services acting as lender, leaser and retrofitter, IBM will tailor deals in a manner that surpasses many of its competitors. With an appropriate financing plan, deals can be as expansive or as narrow as the client desires. The financial situation of the client can be effectively managed, which is critical for encouraging earlier adoption patterns that build traction in the SMB space. Finally, the vast partner network and combinations of services, software and systems (hardware) are more manageable with an IBM financing service brokering the contract.

The Rest of the Message

Lest this sound like an infomerical for IBM, I would like to comment on the rest of IBM's leadership message: collaboration, integration, flexibility and innovation. The evidence of Partnerworld itself suggests a concrete collaborative thrust. The genius of IBM is that it is so effective at developing partnerships with those who should be its natural competitors. The irony is that the beast is slow moving, but like a monstrous tanker it picks up significant momentum in whatever direction it goes.

The collaborative pace for IBM on SMB has been slow. I having been studying SMB for more than five years, for a wide range of clients in all sectors of technology. It has always been clear that there was an untapped lucrative market, which required a more targeted focus. Yet the persistent lag in an appropriate sales and product development orientation within large technology vendors like IBM has been perplexing. What is the research feeding the leadership team? It should have been telling decision-makers that this was a large untapped market. Why has it taken this long to steer the SMB course? Has IBM's partners given Big Blue the push it needed?

On another front, Palmisano rehearsed the mantra of integration and flexibility, referencing the software-services convergence and paramountcy of SOA and Web 2.0. But the past few years have not demonstrated a convincing IBM lead that cannot be tied to its basic sales machinery and its ability to sell at the senior managment level. Is IBM really more advanced in its promotion of integration and flexibility than Microsoft or Intel? Who are the SOA champions and Web 2.0 protagonists? IBM does not come to mind as an innovator - great promoter perhaps.

IBM is still the "safe choice" - but not the innovative choice. After dozens of technology brand studies crossing services, software and systems, I am convinced that IBM is still not coming across as a technology leader despite its claims to the contrary. Nick Donofrio's (IBM EVP Innovation) speech, while highlighting impressive innovation stories, did not offer up anything more than what I would expect to hear from all of the large technology vendors.

Even Palmisano's allusions to a brave new world of virtual social networking (i.e., Second Life) seemed specious. I did sally forth in a newly outfitted avatar and found this virtual world promising, but hardly ready for "prime time". Heather Clancy's blogging buzz on IBM's Second Life (SL) has a more positive spin, but my ongoing experience with Second Life and IBM's SL properties suggest little more than graphically rich springboards to IBM websites. This suggests that the SL environment and basic interactive infrastructure is still far from being a substantively new or effective business tool.

At this point the real insight into IBM's current direction is not about patent comparisons and technological bravado. It is about a fundamentally powerful business model that combines an effective selling organization with a dynamic partnership environment. The glue that holds these two parts together and makes technological adoption viable for both large and small organizations is the ability of IBM to bring together a broad portfolio and ensure that all of the financial instruments are at play, giving clients the where-with-all to invest in technologically driven business enhancements. In this sense, IBM the "promoter" will continue to have a major impact on the expansion and direction of technology markets.

Support is Central to Linux Desktop Success

An interesting survey on Linux desktop usage popped up on my list of news updates. This is an area that I have been mulling over for some time. The key question lurking in the back of my mind is why are larger organizations not flocking to Linux-based desktops. The vast majority of knowledge workers in these organizations would find the basic desktop solutions in SUSE or Mandriva more than adequate. Specialized applications can run on web-based application servers, leaving desktops for basic work tools. Given the potential savings in Microsoft licenses, and given the relative stability and functionality of Linux applications for word processing, presentation and spreadsheet - this seems like a very sensible approach to cost reduction. Moreover, the development of more user-friendly interfaces (e.g., KDE or Gnome) makes these application suites accessible to a wide range of users.

So what is the adoption barrier?

The key lies in the response to the battery of items in Q22 of the SUSE_user survey. When asked to rate how well OpenSUSE fulfilled user requirements, the tension between support and all else remains a consistent pattern. While not having access to the raw data limits statistical testing, a basic observation in the mean scores and grouped distributions suggest that OpenSUSE excels in price, security, stability, usability and selection. By most standards such ratings would constitute a winning proposition. However, one of the key weaknesses lies in product support. Respondents provided lower ratings for support, in the areas of multimedia, documentation and hardware. This pattern extended to the broader issue of "support services".

The problem of support is underscored in a subsequent question, where participants were asked what changes they would like to see in future versions of OpenSUSE. The vast majority of respondents (68.9%) indicated a need for improved hardware support. This is in contrast to the next area of recommended improvement, where 44.6% of users wanted more software packages.

Despite ineffective support, the vast majority of OpenSUSE users are using their desktop for all the basic tasks common to a Microsoft desktop environment. The growth and durability of the OpenSUSE user population also suggest that mastering a profitable support solution could open a significant enterprise opportunity. While more than 30% of respondents stated that they had been using OpenSUSE for more than 4 years, over 35% have begun using this platform within the last 12 months. Such a pattern is suggestive of an easy sell into user populations - once IT decison-makers are convinced that a viable support solution is in place.

The crux of the problem is the fundamental weakness in Linux desktop support capabilities that scares IT decision-makers and keeps them wedded to the Microsoft solution. While insiders will argue that Microsoft support fails in its relationship with individual users, attested to by its consumer satisfaction scores, it has successfully demonstrated a very viable desktop support capability for enterprise users. In fact, there is an entire cottage industry supporting Microsoft desktop solutions. Even for those who are enterprise "power users", but not necessarily Linux savvy, the yearning for a well priced alternative to Microsoft will not overcome the substantive issue of support. As desktop solutions become increasingly integrated in all we do, the need for quick and reliable support becomes increasingly paramount for the ongoing productivity of knowledge workers. As a result, do not expect enterprise adoption of the Linux desktop until support solutions can rival the current hegemony.

For additional discussion of the SUSE_user survey see: Attention desktop Linux users - Are you a typical user? by ZDNet's Adrian Kingsley-Hughes -- Are YOU a typical desktop Linux user?