We arranged an AI hackathon at Aalto University. I’ll summarise our learnings into this post.

The assignment

We had prepared two assignments:

1. Sales Opportunity Qualification – how can B2B sales teams predict which opportunities they will win?
2. Customer Base Segmentation – how many segments should we have and with features of the data should be used to create segments?

We started with the Sales Opportunity Qualification assignment. The business problem many B2B sales organisations have is low hit rate, especially in businesses with long sales cycles (3 to 24 months) and high cost of sales (thousands or tens of thousands Euro and multiple people participating solution design, proposal development, negotiations and so on).

Companies (e.g. telecommunications, industrial automation, enterprise software, outsourcing, ERP/CRM) with long sales cycle would save a lot of money and win more business if they could pick and choose the “right” opportunities early on in the process.

Typically sales organisations have very little data and facts about the sales process, even less about how customers make decisions (etc. which people participate decision making process, what is the company trying to achieve, do they have a budget, etc etc).

Our business problem for this assignment was:

1. Is it possible to develop machine learning based approach to opportunity qualification?
2. Assuming success, how could we create “adaptive opportunity qualification model” based on machine learning?

 

Data

We presented a data set of 200 sales opportunities; 150 lost and 50 won opportunities.

On top of this, we had a rich data set of “opportunity qualification questions”. Qualification criteria includes 15 questions and each question has 5 pre-defined answers. In total, we had 15 x 5 = 75 individual statements (true or false) combined with the outcome of the sales process, won or lost.

Our starting point was that we knew, for 200 sales opportunities, both the outcome (won or lost) as well as answers to all qualification questions (e.g. is there a budget, customer’s strategy and objectives, decision making process, decision making criteria, customer’s perception of our value add, etc).

The task was to use machine learning to help sales team to pick and choose the right sales opportunities early on in the process.

Approaches

We worked in teams of 2 to 4 people.

Teams we allowed to use which ever AI/ML platforms, programming languages and mathematical approaches they wished.

Some teams started to look at the data and work their way up from there, some teams started to think about to business problem to make sure they understood if fully before going into data and some teams started to visualise the data to understand it better. Guess which approach was best?

It was very interesting to see how the teams worked and how different approaches worked.

Typical tools were R, Python, Numpy, TensorFlow and Keras. Typical algorithms were Decision Tree, Random Forest and Neural Networks.

Results and outcome

Our task was to help sales organisations to choose the best sales opportunities early on in the process.

The reference approach was “analyst, excel and linear regression analysis” which has previously been able to identify chance of winning percentage of 67 % at best for one question and 58 % for another question. The rest of the questions provided lower chance of winning estimates. You can read more about this approach in my book Lean Sales (available on Amazon.com).

Artificial intelligence and machine learning driven approached did better.

One team was able to reach 70 % and 73 % prediction accuracy.

Another team was able to reach 81 % prediction accuracy.

Regarding the second question in the assignment – adaptive qualification model – we didn’t have time to dig in. We’ll do that next time.

Future opportunities

In short, there are lots of opportunities for improvement in sales. AI and machine learning approaches can, with proper planning and careful selection of algorithms and platforms, deliver great value and improvements.

AI based performance improvement assumes there is data. This is an evergreen problem in sales. Sales organisations do not collect enough data. Maybe this situation improves, maybe not. It is very clear that who ever has more data about the sales process is able to outperform competition.

The commercial AI applications are not ready to help out-of-the box so the approach will have to continue to be picking one sales related problem at a time and fixing it with point solution. Overtime these solutions will merge into AI driven products and platforms.

Potential areas for utilising artificial intelligence and machine learning in B2B sales are:

• Customer segmentation and customer base analysis
• Pricing
• Sales process improvement
• Lead and opportunity qualification
• Sales behaviours; both instant feedback as well as longer term skills development and training needs
• Prospecting
• Digital marketing, sales and merging marketing and sales processes

The list is long. As soon as there is data, there is an opportunity for machine learning. At the same time, machine learning alone will not solve all of your problems but it can provide new insights and help backup decisions with data, facts and analysis.

Learnings regarding the hackathon event

#1 Data preparation takes a long time. All team spend 30 – 50 % of the time on preparing the data.

#2 Different approaches deliver different results. Different approaches also provide different types of insights into the data. My conclusion is to take a bit more time to allow playing around with the data. If one is able to play around with data and look at “what the data could tell us” instead of purposefully trying to reach a solution the results and learning are a lot more than one could expect. You never know what the data hold until you spend time analysing it.

#3 Different tools / libraries and platforms deliver very different results. AI and machine learning platforms

#4 Four hours is enough time to scratch the surface but not enough to get to the solutions

#5 Shop at Kotipizza to feed the participants. We made our large order pretty close to the closing time. The local Kotipizza entrepreneur had no problems keeping the shop open after closing time! Good work Tommi the Pizza Guy and Kotipizza.

#6 Don’t let raccoon dogs in. They attack people when afraid.

WTF – Welcome to Finland. Wildlife is close by 🙂

Conclusions

My conclusions from this event are

1. Machine learning is perfectly capable of helping sales organisations and sales people do better.
2. Machine learning vs. traditional excel based analysis is way more powerful.
3. Commercial AI driven products are not ready to work off-the-shelf – this work will continue to be professional services and consultant driven analysis
4. Machine learning approach does provide not only better analysis but also a platform for better solutions (e.g. adaptive opportunity qualification).
5. I’ll continue to arrange both hackathons as well as my search for repeatable AI/ML based approaches to solve problems in B2B sales. The journey has just started.

What’s next

We’ll arrange new “AI meets sales processes” hackathons in the near future. We’ll also continue to apply AI to improve sales processes.

But let’s now enjoy the summer time!