Home » Shaping Futures » How To Evaluate AI Software

How To Evaluate AI Software

Buying off-the-shelf AI (Artificial Intelligence) software is a good first step for those companies that are new to the technology. There should be little need to make investments in technical infrastructure or to hire expensive data sciences. There will also be the benefit of getting a solution that has been tested by other customers. For the most part, there should be confidence in the accuracy levels as the algorithms will probably be implemented properly.

But when it comes to a new AI solution, it can be tough to find the right sources of data, wrangle it and integrate it. Thus, when evaluating an application, you need to make sure that there are ways to handle this process.

“The most complex task in an AI solution is not to implement the machine learning algorithm anymore—this is usually available as a set of functions in every tool—but to collect the data,” said Rosaria Silipo, a Ph.D. and a principal data scientist at KNIME. “That is, to connect to a variety of data sources, on premise, on the web, or on the cloud, and extract the data of interest.”

This is why it is important to see if the application is built to handle your particular vertical or situation.

“Take search, for instance, where AI can be used to re-rank results and improve relevance,” said Ciro Greco, who is the Vice President of Artificial Intelligence at Coveo. “When applied to ecommerce, search is searching on semi-structured records, such as products with little text available and we can count on reasonable amounts of behavioral data produced by users who browse the website. A strategy based on user behavior can be very effective, because we can count on having enough data to learn from.”

Yet AI-search for customer service use cases is often much different. It’s often about finding technical documents. “There is plenty of unstructured text, such as Knowledge Articles, and fewer behavioral data points, because customer service websites usually are less visited than e-commerce platforms,” said Greco. “So in that case, a strategy based on NLP for topic modeling will probably be more effective, because we need to maximize the gain from the information we have, in this case free text.”

The article was originally posted at Forbes.

To support for AI positive development in business and society, Michael Dukakis Institute for Leadership and Innovation (MDI) and Boston Global Forum (BGF) has established Artificial Intelligence World Society Innovation Network (AIWS.net). In this effort, MDI and BGF invite participation and collaboration with governments, think tanks, universities, non-profits, firms, and other entities that share its commitment to the constructive and development of full-scale AI for world society. This initiative is to develop positive AI for helping people achieve well-being and happiness, relieve them of resource constraints and arbitrary/inflexible rules and processes, and solve important issues, such as SDGs.