Ability to Execute
Vendors were evaluated on the quality and efficacy of the processes, systems, methods and procedures that enable their performance to be competitive, efficient and effective, and that positively impact their revenue, customer retention and reputation. Vendors were also judged on their ability to capitalize on their vision.
Product or Service: This criterion focuses on the core goods and services that compete in and/or serve the IT VRM market. This includes current product and service capabilities, quality, feature sets and skills. This can be offered natively or through OEM agreements/partnerships, as defined in the Market Definition/Description section and detailed in the subcriteria.
The IT VRM use case focuses on the process of ensuring that the use of third-party service providers and IT suppliers does not create an unacceptable potential for business disruption or a negative impact on business performance. IT VRM solutions support enterprises that must assess, monitor and manage their risk exposure from third parties that provide IT products and services, or that have access to enterprise information. In general, IT VRM solutions focus on the risks associated with logical assets, information governance and IT controls, rather than the risks associated with physical supply chain assets.
Overall Viability: This criterion includes an assessment of a vendor’s overall financial health, and the financial and practical success of the relevant business unit. Factors include the likelihood that the business unit will continue to invest in the software, offer the software and advance the state of the art within the organization’s portfolio of products. Evidence of ongoing investment in IT VRM, overall company revenue and revenue from the IT VRM platform determine a vendor’s score for this criterion.
Sales Execution/Pricing: This criterion concerns a vendor’s capabilities in all presales activities and the structure that supports them. This includes deal management, pricing and negotiation, presales support, and the overall effectiveness of the sales channel. A key metric is sales performance in the past year. For pricing, the key metrics are the transparency and ease of calculation of the pricing model.
Market Responsiveness/Record: This criterion concerns a vendor’s ability to respond and adapt to changing competitive forces as opportunities develop, competitors act, customers’ needs evolve and market dynamics change. It also considers a vendor’s history of responsiveness and its ability to quickly address changing requirements. A key metric is the growth of a vendor’s IT VRM customer base over the past three years.
Marketing Execution: The clarity, quality, creativity and efficacy of programs designed to deliver the organization’s message in order to influence the market, promote the brand, increase awareness of products and establish a positive identification in the minds of customers. This “mind share” can be driven by a combination of publicity, promotional activities, thought leadership, social media, referrals and sales activities.
Customer Experience: This criterion covers a vendor’s relationships, products and services/programs that enable customers to be successful with the products evaluated. Customers were asked questions to determine their experience with their vendor and its IT VRM solution. These included whether the product met, exceeded or fell short of their expectations; areas where they think the vendor needs to improve; and their overall satisfaction with the vendor. Key metrics include overall satisfaction, value for money, and positive and negative comments from reference customers.
Operations: This criterion considers an organization’s ability to meet its goals and commitments. Factors include the quality of the organizational structure — including skills, experiences, programs, systems and other factors that enable the organization to operate effectively and efficiently on an ongoing basis. Key metrics are customer satisfaction with support and ongoing upgrades, customer satisfaction with professional education and training programs, and the availability of user conferences and other means by which customers can improve their skills.
Table 1: Ability to Execute Evaluation Criteria
Completeness of Vision
Vendors were evaluated on their ability to convincingly articulate logical statements about current and future market direction, innovation, customer needs and competitive forces. Vendors were also rated on their understanding of how market forces can be exploited to create opportunities for themselves and their clients.
Market Understanding: This criterion considers a vendor’s ability to understand buyers’ needs and to translate those needs into products and services. Vendors with the most vision listen to and understand buyers’ wants and needs, and can shape or enhance those wants and needs. Vendors need to understand the business and regulatory drivers for IT VRM in the short term, and the market’s long-term requirements.
Sales Strategy: This criterion considers a vendor’s strategy for selling IT VRM solutions. We look for the use of an appropriate network of direct and indirect sales resources, partner networks and alliance relationships to extend a vendor’s market reach to both existing and prospective customers.
Offering (Product) Strategy: This criterion considers a vendor’s approach to product development and delivery, with an emphasis on differentiation, functionality, methodology and features as they map to current and future requirements. Vendors are evaluated on their IT VRM roadmaps to advance current capabilities and deliver new ones.
Vertical/Industry Strategy: This criterion considers a vendor’s strategy to direct resources, skills and offerings to meet the specific needs of individual market segments, including industries. Ideally, vendors should have differentiated strategies for the financial services, healthcare and life science sectors, and offer value to customers in less-regulated industries.
Innovation: This criterion concerns a vendor’s direct, related, complementary and synergistic layouts of resources, expertise or capital for investment, consolidation, defensive or preemptive purposes. The criterion also evaluates the percentage of revenue dedicated to both R&D and the clearly innovative offerings that shape the market.
Geographic Strategy: This criterion concerns a vendor’s strategy to direct resources, skills and offerings to meet the needs of geographies outside its native geographic area, either directly or through partners, channels and subsidiaries, as appropriate for that area and market. The primary metrics are a direct sales and support presence in multiple geographies, as well as reseller and service partner support.