A New Evaluation Framework for Customer Service Products

Posted Thursday, January 25, 2018 in Customer Service by Mitchell Kramer

We used a new Evaluation Framework for our latest Product Evaluation Report, which is about Salesforce Service Cloud. We introduced the new Framework to make our reports shorter and more easily actionable. Shorter for sure, our previous report on Service Cloud was 57 pages including illustrations. This one is 16 pages plus illustrations, shorter by more than two thirds!   

We don’t yet know whether the Report is more easily actionable. It was just published. But, our approach to its writing was to minimize descriptions and to bring to the front our most salient analyses, conclusions, and recommendations.


Our Product Evaluation Reports had become increasingly valuable but to fewer readers. Business analysts facing a product selection decision, analysts for bankers and venture capitalists considering an investment decision, and suppliers’ competitive intelligence staff keeping up with the industry have always appreciated the reports, especially their depth and detail.

However, suppliers, whose products were the subjects of the reports, complained about their length and depth. Requests for more time to review the reports have become the norm, extending our publishing cycle. Then, when we finally get their responses, we’d see heavy commenting at the beginning of the reports but light commenting and no commenting at the end, as if they lost interest. Our editors have made the same complaints.

More significantly, readership, actually reading in general, is way down. Fewer people read…anything. These days, people want information in very small bites. Getting personal, for example, I loved Ron Chernow’s 800-page Hamilton, but I have spoken to so many who told me that it was too long. They couldn’t get through it and put it down unfinished, or, more typically, they wouldn’t even start it. I’m by no means comparing my Product Evaluation Reports to this masterpiece about American history. I’m just trying to emphasize the point.

Shorter Reports, No Less Research

While the Product Evaluation Report on Salesforce Service Cloud was two thirds shorter, our research to write it was the same as our research for those previous, much longer Product Evaluation Reports. Our approach to research still has these elements, listed in order of increasing importance:

  • Supplier presentations and demonstrations
  • Supplier web content: web site, user and developer communities
  • Supplier SEC filings, especially Forms 10Q and 10K
  • Patent documentation, if appropriate
  • Product documentation, the manuals for administrators, users, and developers
  • Product trial

Product documentation and product trial are the most important research elements and we spend most of our research time in these two areas. Product documentation, the “manuals” for administrators, users, and developers provides complete, actual, accurate, and spin-less descriptions of how to setup and configure a product, of what a product does—its services and data, and of how it works. Product trials give us the opportunity to put our hands on a product and try it out for customer service tasks.  

What’s In?

The new Framework has these four top-level evaluation criteria:

  • Customer Service Apps list and identify the key capabilities of the apps included in or, via features and/or add-ons, added to a customer service software product. 
  • Channels, Devices, Languages list supported assisted-service and self-service channels, devices attachable to those channels, and languages that agents and customers may use to access the customer service apps on those devices.
  • Reporting examines the facilities to measure and present information about a product’s usage, performance, effectiveness, and efficiency. Analysts use this information continually to refine their customer service product deployments.
  • Product, Supplier, Offer. Product examines the history, release cycle, development plans, and customer base for a customer service product. They’re the factors that determine product viability. Supplier examines the factors that determine the supplier’s viability. Offer examines the supplier’s markets for the product and the product’s packaging and pricing.

This is the information that we use to evaluate a customer service product.

What’s Missing?

Technology descriptions and their finely granular analyses are out. For example, the new reports do not include tables listing and describing the attributes/fields of the data models for key customer service objects/records like cases and knowledge items or listing and describing the services that products provide for operating on those data models to perform customer service tasks. The new reports do not present analyses of individual data model attributes or individual services, either. Rather, the reports present a coarsely granular analysis of data models and services with a focus on strengths, limitations, and differentiators. We explain why data models might be rich and flexible or we identify important, missing types, attributes, and relationships then summarize the details that support our analysis.

“Customer Service Technologies” comprised more than half the evaluation criteria of the previous Framework and two thirds of the content of our previous Framework-based reports. These criteria described and analyzed case management, knowledge management, findability, integration, and reporting and analysis. For example, within case management, we examined case model, case management service, case sources, and case management tools. They’re out in the new version and they’re the reason the reports are shorter. But, they’re they basis of our analysis of the Customer Service Apps criterion. If a product has a rich case model and a large set of case management services, then rich case model and large set of case management services will be listed among the case management apps key capabilities in our Customer Services Apps Table and we’ll explain why we listed them in the analysis following the Table. On the other hand, if a product’s case model is limited, then case model will be absent from the Table’s list of key capabilities and we’ll call out the limitations in our analysis. Just a reminder, our bases for the evaluation of the Customer Service Apps criteria, the subcriteria of Technologies for the old Framework are shown in the Table below:



Customer Service App

Basis for Evaluation and Analysis

Case management

·    Case model

·    Case sources

·    Case management services

·    Case management tools

Knowledge management

·    Knowledge model

·    Knowledge management services


·    Search sources

·    Source management

·    Search query syntax and language support

·    Search analyses and matching

·    Search results management

Social customer service

·    Social sources

·    Instrumentation of social sources

·    Identification of social posts

·    Alerts and notifications

Reporting and Analysis

·    Instrumentation

·    Packaged reports

·    Reporting tools

Table 1. We present the basis for the evaluation of the Customer Service App criteria in this Table.

Trustworthy Analysis

We had always felt that we had to demonstrate that we understood a technology to justify our analysis of that technology. We had also felt that you wanted and needed our analysis of all of that technology at the detailed level of every individual data attribute and service. You have taught us that you’d prefer higher-level analyses and low-level detail only to understand the most salient strengths, limitations, and differentiators.

The lesson that we’ve learned from you can be found in a new generation of Product Evaluation Reports. Take a look at our latest Report, our evaluation of Salesforce Service Cloud and let us know if we’ve truly learned that lesson.

Remember, though, if you need more detail, then ask us for it. We’ve done the research.


  • jonashex
    jonas hex on November 5, 2020 at 6:09 a.m.
    Thank you for the informative text. The improvement of CRM products does not stand still, which allows companies to regularly increase the number of sales. Salesforce consulting companies also constantly receive updates from us. Only in conditions of constant development can you stay afloat
  • mariannamay
    Lucy March on January 17, 2021 at 11:18 a.m.
    I also agree with you about the importance of CRM products, which allows companies to regularly increase the number of sales and control their quality by 100%. The sales dashboard for salesforce also receives constant updates from us and you can track all the processes in real time. Only in conditions of constant development is it worth improving and finding new ways for sales.
  • JeffSimons
    Jeffrey Simons on January 25, 2021 at 6:14 a.m.
    This is the reason why I keep a constant check on your website because you'll never know when this next list of great links are going to pop up. At least you have finally uploaded the lords mobile online and lokicraft on pc because a lot of members are asking for it. You've got some great new additions in there as well and I can't wait to discover what's in them. I hope that they won't cause any problems though because the last batch had a couple of them though they were not game breaking or anything.
  • Michelle08
    Michelle Roe on February 5, 2021 at 7:54 a.m.
    This post is highly helpful for me. Massive thumbs up for this! Regards https://www.mizzmarketing.com/
  • JerryP133
    Jerry P on February 10, 2021 at 8:38 p.m.
    Wow.  Couldn't be more spot on!  Great information.  It's interesting to see for sure.  Wonder how it will impact the world as we continue to move forward in these pandemic times.  The whole world is changing, whether it be for conservative people or people who like to live on the edge like swingers.  Put no matter what people are always going to want to meet real swingers!  
  • jackslesly19
    Lesly Jacks on February 16, 2021 at 3:56 a.m.
    Greetings from custom home builder bendThis is interesting. I'm impressed. You're truly well informed and very intelligent. You wrote something that people could understand and made the subject intriguing for everyone. I'm saving this for future use.
  • Michelle77
    Fuhra Leshia on February 16, 2021 at 4:24 a.m.

    Great work! I love how you made this thing so detailed.

    handyman services st. petersburg

  • cmsalva
    Carlos Salvatore on February 16, 2021 at 9:54 p.m.

    Excellent table as basis for your evaluation on a customer service app that you are testing. Thank you for sharing this info and looking forward to its results.



    commercial cleaning auckland

  • azcleaningpro
    Elvis Cervantes on February 23, 2021 at 9:36 p.m.
    Thank you for the article! This is definetely something we will use in our house keeping company to improve our customer service!
  • LucilleM
    Lucy Morning on February 25, 2021 at 5:38 a.m.

    Yay! This framework will surely help a lot of businesses! Amazing.concrete contractors greenville sc  lawn fertilizer

     stamped concrete charleston sc

  • johnsneek
    john sneek on March 1, 2021 at 1:05 a.m.

    Thanks for sharing this amazing source of information especially the EVALUATION OF CUSTOMER SERVICE APPS. 

    junk removal twin cities

    junk removal

  • jaxoli4324
    Jaxon Olive on March 1, 2021 at 2:03 a.m.

    Great! thanks for this very informative evaluation of the framework for CSP !we appreciate it from Office Cleaning

  • johnlead
    john lead on March 1, 2021 at 6:06 p.m.
    It was really nice framework and no doubt about increasing the evaluation. Thank you for sharing your information. Cheers from plastersydney.com.au
  • petermartin85404
    Peter Martin on March 4, 2021 at 3:20 a.m.
    1. Understand current customer journeys: Use this checklist of questions to understand typical customer journeys to help determine what customers are looking for.  Extenze
  • wilnewell7122
    William Newell on March 4, 2021 at 11:03 p.m.

    Remind me again why someone would even want this. It is so commendable hence I am going to bookmark it. You can end of tenancy oven cleaning service here.

  • Handmanbloomingtonil@gmail.com
    Dylan Cherko on March 6, 2021 at 8:29 a.m.
    Would you suggest that patents are needed for most business activities or are they just for inventions. Bathroom Remodel Bloomington
  • Handmanbloomingtonil@gmail.com
    Dylan Cherko on March 8, 2021 at 5:16 a.m.
    This is by far the best place on the topic. Surveyors Tampa
  • brielleluna
    Brielle Luna on March 8, 2021 at 12:13 p.m.
    I believe there are a lot of available of sites that could help issues such as this one.
  • Kent
    Kenneth Murphy on March 11, 2021 at 9:17 a.m.
    Thank you for posting this. Very informative. richmond mold removal services
  • Steve01
    Steve Sanders on March 13, 2021 at 3:17 a.m.
    Thank you for sharing such a great information. IT services west palm beach 561-475-1169.
  • fritzsara5225
    Sara Fritz on March 16, 2021 at 7:12 a.m.
    Alongside the action of human via online media stage expanding, web-based media investigation has been generally utilized in different fields' examination, including estimation investigation. In this paper, we propose a structure to assess consumer loyalty based on the information from web-based media stage and the innovation of conclusion investigation. Assessing consumer loyalty dependent on the remarks from online media stage is exceptionally true and solid for the explanation that individuals like to tell the genuine inclination in web-based media instead of in business review.
  • petermartin85404
    Peter Martin on March 18, 2021 at 6:55 a.m.
    The new evaluation framework for the customer service product is more straightforward and easily understood therefore it is easy to analize. Tethered Drone
  • corys88
    Cory Starling on March 18, 2021 at 3:12 p.m.
    Thank you for this. Was thinking of putting a package togther after getting inpsired by stump grinding Victoria BC companies and so here I am. Will put this info to good use, I hope.
  • Steve01
    Steve Sanders on March 27, 2021 at 12:32 a.m.
    Glad to found this website. Keep up the great work. Fence tampa 813-532-3771.
  • johnlead
    john lead on March 29, 2021 at 3:05 a.m.
    You’ve got some interesting points in this article. I would have never considered any of these if I didn’t come across this. Thanks!.  Please do visit also our Website
  • rdixon
    Robert Dixon on March 30, 2021 at 10:28 a.m.

    Ease and convenience are the things that I can see why you have made this new Framework. Anyway, thank you for sharing this update with us, and looking forward to its success. 

    Robert - Landscaper at El Paso Landscaping

  • lawncareguys
    Dan Messina on March 30, 2021 at 5:05 p.m.

    This could prove to be very useful. Having an effective CRM is everything.

    Senior Care Home

    Pole Buildings Oregon

    Bend Fencing Contractors

  • bobedward
    Bob edward on April 2, 2021 at 1:58 a.m.

    Wonderful blog post. You’ve got some interesting points in this article. 

    SEO company California

  • johnrico022117
    Jhon Rico on April 2, 2021 at 8:21 a.m.

    This is very interesting! Thank you for sharing this post and I have learned some new information in the evaluation framework for customer service products.

    [url=https://www.concretecontractorsyuma.com]concrete yuma az[/url]

  • johnrico022117
    Jhon Rico on April 2, 2021 at 8:25 a.m.
    Thank you again for this information! concrete yuma az
  • Connorrr
    Connor Roland on April 2, 2021 at 9:09 a.m.

    Looking for ways to diversify your practice and offer new services to meet emerging needs often means stepping outside your comfort zone and into unfamiliar territory. Using a systematic approach to evaluate potential opportunities can not only help you consider a variety of important issues related to launching a new service, but also facilitate effective decision making and increase the likelihood of success in new ventures. roofing cincinnati


  • getnada.com
    xalelpipyle pipyle on April 3, 2021 at 9:56 a.m.
    Nice knowledge gaining article. This post is really the best on this valuable topic.
  • bulbivbali
    bulbiv bali on April 3, 2021 at 11:59 a.m.
    nice information,  <a href="https://bulbiv.com">Bulbiv Marketing Solutions Bali</a>
  • bulbivbali
    bulbiv bali on April 3, 2021 at 12 p.m.
  • bulbivbali
    bulbiv bali on April 3, 2021 at 12 p.m.



  • bulbivbali
    bulbiv bali on April 3, 2021 at 12:01 p.m.
  • johnlead
    john lead on April 5, 2021 at 5:19 a.m.
    'm very grateful to the author, this really changed my thought. I'm looking forward to acknowledge this website as one of the best for new information. painting Newcastle
  • johnlead
    john lead on April 5, 2021 at 5:20 a.m.
    I'm very grateful to the author, this really changed my thought. I'm looking forward to acknowledge this website as one of the best for new information. plastering sydney
  • tracypressurewashing@gmail.com
    Justin Chavez on April 7, 2021 at 8:04 p.m.

    Great insight into customer service technologies! Thanks for sharing your knowledge on this matter.

    Justin | Concrete Cleaning

  • Connorrr
    Connor Roland on April 8, 2021 at 10:41 a.m.
    I would definitely give this to my employees. women's fashion
  • JimJenkins123
    Jim Jenkins on April 8, 2021 at 6:58 p.m.
    Trustworthy analysis is at the heart of any solid customer service product. Bravo for investing the time and resources into this endeavor. tree removal Fishers
  • getnada.com
    xalelpipyle pipyle on April 13, 2021 at 11:20 a.m.
    An fascinating discussion is value comment. I think that it is best to write extra on this matter, it won’t be a taboo topic however generally people are not enough to talk on such topics. To the next. Cheers
    More Info
  • Olive
    Olive Smith on April 13, 2021 at 1:11 p.m.
    Thanks for sharing this nice article, I like the content. roofing contractors
  • bobedward
    Bob edward on April 13, 2021 at 1:41 p.m.

    Really great framework. Much appreciate your efforts in sharing it. Thanks.

    tree care san Jose

  • magnoliacare
    Julie Dron on April 13, 2021 at 8:55 p.m.

    Excellent framework. Love to see how all of this is constantly evolving.

    roofing companies vancouver wa


  • gofoto2866@ddwfzp.com
    todd dens on April 14, 2021 at 6:29 a.m.
    Thanks a lot for sharing this with all folks you really understand what you are talking approximately bathrooms makeovers
  • gofoto2866@ddwfzp.com
    todd dens on April 14, 2021 at 7:14 a.m.
    I’m glad that you shared this helpful info with us. Thanks for sharing! https://deckingcentralcoast.com.au/
You must be a member to comment. Sign in or create a free account.