How to Choose an AI Vendor for Talent Management

Companies are struggling to keep up with a rapidly changing workplace and the associated shift in how people are managed. Some are turning to artificial intelligence (AI) solutions. The Deloitte Global Human Capital Trends report finds that 44 per cent of the companies surveyed are actively exploring AI solutions to embed into their organizations. 

With many talent management vendors on the market, sorting through the options to find the best product and value can be daunting. This checklist can help navigate the process of identifying vendors that offer high-quality solutions.

Define the problem and understand the solution 

• Define a specific business problem that can feasibly be solved by AI. For example, AI tends to be best at finding and learning from patterns in large data sets.

• Identify someone within your organization who can bridge the gap between technical and business jargon. This person can critically explore the claims of the vendor and explain how the product can solve the proposed problem.

• Enable this technical and business-fluent person to act as knowledge translator between AI experts and business executives.

• Evaluate employees’ specific needs and build relevant personas to ensure all levels of the organization will derive value from the solution.

• Ensure the product’s functionality aligns with the core problem(s) being solved.

• Estimate the quantitative and qualitative value created from the AI solution and the approximate return on investment.

Involve internal stakeholders

• Engage your IT professionals early in the process. Even if they may not lead the charge in selecting and implementing the AI solution, they are vital to ensuring the vendor has access to data that will be used to “train” the AI product.

• Mitigate roadblocks such as platform integration by encouraging collaboration between IT and the vendor. Encourage your IT team to build a sense of ownership by involving them in the vendor evaluation process whenever possible.

• Manage expectations around the current state of AI technology and scope so that the product does not become an ill-fitting blanket solution for all problems across the business.

• Discuss the product with end-users. Have them identify and explain their desired user experience so that meetings with potential vendors have a clear set of expectations for functionality and integration. Involving end-users can also surface potential barriers to adoption.

Research the vendor

• Search a website such as Crunchbase for detailed information about a company’s funding, age, leadership and steady growth. If the vendor is still in initial seed rounds to fund product development, it may be too early for it to tackle a large project effectively.

• Look for series A or B funding rounds. If a vendor has not secured any new funding, but is technically in the growth phase, it could indicate that the company has been unable to convince venture capitalists on its merits. 

• Explore the likelihood of acquisition. A vendor that gets bought by a larger company may be less responsive or have less personalized customer service. It might focus more on profit and less on growth from innovation.

• Ensure the vendor can demonstrate product maturity, a track record of growth and a sustainable business model.

 Research the AI tool

• Verify that the vendor has an expert AI data scientist on staff to assist with ongoing support and updates. Be wary of vendors who merely subcontract data scientists for the initial build.

• Get an estimate of upfront costs associated with the product as well as ongoing support and maintenance costs over the long term.

• Determine how frequently product updates are released and if new features are priced separately from established licensing fees.

• Investigate how employees are incentivized or encouraged to complete their initial registration for the platform. For example, can automated push notifications be enabled to facilitate skills updates? Are there behavioural nudges embedded in the platform?

• Differentiate between “fake” versus “real” AI by asking:

  • Is the solution an existing legacy software rebranded?

  • Was the code built from the ground up?

  • What type of algorithms were used and why?

  • What data set was used to “train” the AI?

 Determine the availability of support and training

• Confirm that structured training courses are offered by the vendor. Initial training reduces over-dependency on the vendor over the long term.

• Discuss how the vendor can help build internal competency within the organization. This could be a “train the trainer” learning model or a SaaS website with learning materials that are updated as new features are released.

• Establish the frequency of updates and maintenance of the technology. An obsolete or old program version that is incompatible with other internal software can be detrimental to the long-term implementation success.

• Explore the vendor’s capacity to support relevant integrations within existing systems of your business, and if the product is API-driven.

Explore the vendor’s track record with previous or current clients 

• Illuminate potential red flags by discussing the vendor and implementation experience with one of the vendor’s existing clients who is similar in size and industry to your organization.

• Explore benchmarking data if available and the success or failure of the product rollout.

• Ask an existing client how well the platform integrates with other applications (such as Microsoft Office or Workday). Do they feel the platform has been scalable, flexible, intuitive and accessible for all users?

• Talk to experts beyond the sales team, such as a software engineer or data scientist, who can explain the decision-making process behind key tech features and algorithmic design.

• If possible, chat informally with end-users to get a candid view of how the product functions day-to-day and how it was received by employees.

Assess and mitigate risks

Talent management leaders should be aware of the potential risks of introducing AI-based solutions. While it is not necessary for them to have a deep understanding of machine learning, they should ask enterprise vendors if they are prepared to address concerns about bias and ethical use.  For an overview of the issues involved, begin by reviewing the responsible use of AI guiding principles released by the Canadian government to understand the ethical and legal deployment of AI solutions.

When assessing potential vendors, consider asking if they are experienced in writing “equal opportunity algorithms” or if they have any best practices in place to mitigate unconscious and machine biases. Be wary if the response is a resounding, “Our AI does not have any biases.” If the response is, “We’ve done a number of things to help reduce bias in our algorithms and we can outline these actions in detail”, then the vendor is likely worth exploring further. 

Dagmar Christianson is a consultant in workforce transformation and the future of work.