How the Levenshtein Distance shortens the extra mile for sales teams
What’s up in Google Cloud?
11. October 2021
What’s up in Google Cloud? Next ’21 Special
21. October 2021
By Patrice Wuescher,
Cloud & Data Engineer
How the Levenshtein Distance shortens the extra mile for sales teams
Data Analytics: Wabion Cloud & Data Engineer Patrice Wuescher points out how companies can harness Fuzzy Matching to increase the quality of customer data and hence the efficiency of sales teams.
In 2021, cold acquisition is still the hottest iron for the business development team in many companies. Due to a lack of alternatives, sales reps browse the CRM, call potential customers, go the extra mile visiting prospects without prior contact, and often come back empty-handed. Of course, there are highly skilled, very experienced and successful salespeople that rely on cold acquisition and deliver convincing results. Generally speaking, however, the downsides outweigh the benefits:
- Lack of transparency: cold acquisition is a cumbersome process that relies heavily on the individual expertise of sales reps without enabling a transparent assessment of the market.
- High costs: unannounced in-person visits or calls often remain unsuccessful and incur high expenses.
- Poor employee retention: various studies indicate that cold calling entails an undesirably high employee turnover, as it is a rather daunting activity.
- Poor customer retention: companies that focus mainly on cold acquisition tend to neglect existing customers.
- Low conversion: cold campaigns usually end with low conversion rates, not least because they don’t follow a strategic and structured approach.
But in 2021, there are alternatives and improvements to cold acquisition. Successful companies provide their sales reps with resources to tackle the market with a more systematic and thus more promising approach. Hence, in the age of data, there is no way around the availability of high-quality market data. But how do you get there?
Fuzzy matching for better data
Even if you have never heard of Fuzzy Matching aka Approximate String Matching, you have most likely already encountered it. Many databases use a String Matching Algorithm to accommodate misspellings in the search function. For example, when you enter “serch”, “search” will also appear among the results. Calculating the minimum edit distance between two strings as the minimum number of necessary operations (insertion, deletion, substitution) needed to transform one string into another – the Levenshtein Distance– is a well-known metric applied in Fuzzy Matching:
The Levenshtein distance between “INTENTION” and “EXECUTION” is 5, as it takes 5 operations (delete, substitute, substitute, insert, substitute) to transform one word into the other. Each operation has a cost of 1. There is an alternative version of the metric valuing “substitute” as the combination of “insert” and “delete” with a cost of 2.
There are numerous use cases for Fuzzy Matching with the Levenshtein Distance outside search engine frameworks. In Customer Relationship Management (CRM), this proven approach enables companies to improve the quality of their contact database. As a result, sales representatives can come up with new and more targeted business development strategies.
Example of a solution harnessing Fuzzy Matching to improve the quality of a customer’s contact database. The solution based on the Google Cloud Platform (GCP) and developed by Wabion compares CRM entries (name, address etc.) with publicly available geo data to improve the data quality.
Data-driven customer acquisition
Google Cloud Premier Partner Wabion, for example, has harnessed the Levenshtein Distance to help a customer refurbish the CRM system as the first step to rethink the (cold) acquisition policy. Combining its Google Cloud expertise with Fuzzy Matching, the Wabion solution compares CRM data such as name, address etc. with publicly available (geo) data via Google Maps API in BigQuery, Google Cloud’s serverless data warehouse, to improve the quality of the customer’s contact database (see image above). Matching the data via a User-defined Function in BigQuery, developed in JavaScript, includes several steps:
- Data Cleansing, including the removal of common “stop words” in customer names
- Conversion of all string values to lowercase to remove case sensitivity
- Tokenization of strings, i.e. sorting the individual “words” of a string in alphabetical order, re-concatenating them and checking the result for similarity
- Definition of a threshold to distinguish between “true” and “false” matches (visualized with chart)
- Weighing of different algorithms and parametrizations (multi-value approach using multiple characteristics to determine final match)
- One-to-one mapping of customer entities to avoid double matches
The power of BigQuery and Google Cloud helps leverage the benefits of Fuzzy Matching (e.g. matching despite different strings) and overcome its challenges (e.g. minimizing the number of false matches). However, there are also some non-technical obstacles such as data privacy issues when using publicly available data for business development purposes. Nonetheless, Fuzzy Matching with the Levenshtein Distance can lay the foundation for data-driven acquisition strategies.
Source code of the User-defined Function in BigQuery developed in JavaScript.
More hot Leads
Once the CRM is refurbished, the door is open to take the data to the next level. With Artificial Intelligence (AI) and Machine Learning (ML) expertise, companies can structure customer information in a new way, focussing on most promising opportunities instead of contacting prospects randomly or only based on sales reps’ subjective market assessments. Cold acquisition may still be a part of data-driven business development, but with hotter leads, a shorter extra mile and happier sales reps.