Recommendation techniques
The recommendation systems are built using Machine learning techniques which filter out the products that users would be interested in or would buy based on his or her previous buying history. The accurate recommendations are achieved when we have good data available about a user.
Here we will talk about popular recommendation techniques
- Collaborative filtering
- Content-based filtering
- Hybrid recommendation system
Collaborative filtering
In the real world we seek advice from our trusted people like friends, colleagues, experts…it’s called “word-of-mouth”. This technique automates the process of “word-of-mouth” based on collecting and analyzing a large amount of information on users’ behaviors, activities or preferences and chooses below ways to recommend.
- Recommend items with maximum prediction
- Weight all users with respect to similarity with the active user
- Select a subset of the users / neighbors to use as recommender’s
- Predict the rating of the active user for specific items based on its neighbors rating
There are two commonly used Collaborative Filtering based on similarities among ‘users’ or ‘items’
- User-Based Collaborative Filtering (user-item matrix): Say Peter and Tony have seen the same movies,and they rated movies all almost identically. But Tony hasn’t seen ‘John Wick: Chapter 3’ and I did see and liked it, it sounds logical to think that Peter will also like it. This way we create the similarity between these users to make a recommendation. The applications which implement User-Based CF will generally apply user-based Nearest Neighbor algorithm.
- Item-Based Collaborative Filtering (item-item matrix): Let’s take example, if Peter gave high rating to a “Matrix” movie then Peter would be recommended other movies rated highly by people who rated “Matrix” movie highly too. Note that in Item-Based Collaborative Filtering, we directly pre-calculate the similarity between the co-rated items, skipping Nearest Neighbor search.
Challenges:
- Ratings data is often sparse, and pairs of users with few co-ratings are prone to skewed correlations
- Fails to incorporate agreement about an item in the population as a whole
- Calculating a user’s perfect neighborhood is expensive
- The Cold Start problem – concerns the issue where the recommendation system cannot draw inferences for users or items for which it has not yet gathered sufficient information. (this is addressed by matrix factorization used in Model-based recommendation system).
The above two (Item-Based and User-Based) are Memory-based recommendation systems, which are not always as fast and scalable as we would like them to be, especially in the context of actual systems that generate real-time recommendations based on very large datasets. To overcome these, model-based recommendation systems are used.
The Model-based recommendation systems involve building a model based on the data-set of ratings. In other words, we extract some information from the data-set, and use that as a “model” to make recommendations without having to use the complete data-set every time. This approach potentially offers the benefits of both speed and scalability.
Although the basic idea behind model-based recommendation systems is the same, there are several approaches like Clustering, Association, Bayesian networks, Neural networks…. that we can take to build the model and use it.
Content-based filtering:
Content-based filtering, also referred to as cognitive filtering, recommends items based on user’s actions, such as products bought or clicked on, web pages viewed, time spent browsing various product categories, etc. It then uses this information to create a customer profile. This profile is then compared to the product catalog to make recommendations.
Challenges:
- Scalability is a challenge – exceptional domain-specific knowledge is key requirement, becomes impractical to scale the experts when there are huge additions
- Attributes may be inconsistent – recommendations are only as good as the experts who are tagging items, when dealing with millions of items, there will be challenging to ensure attributes are consistently applied.
- Lack of novelty and diversity – In this technique, the recommendations seem obvious i.e. if we watched “Infinity war”, system recommends “Endgame”; we don’t need a recommendation engine to tell this. What is needed is stuff the user wasn’t expecting and stuff that represents a broad selection of their interests.
Hybrid recommendation systems:
Hybrid recommendation systems are based on the combination of Collaborative filtering and Content-based filtering. Hybrid approaches can be implemented by making content-based and collaborative-based predictions separately and then combining them.
Hybrid recommendation system will overcome the limitations of native filtering approaches. It improves the prediction performance. Importantly, it overcomes the filtering problems such as sparsity and loss of information. Given two or more basic recommendation systems techniques, several ways have been proposed for combining them to create a new hybrid system. However, they have increased complexity and are expensive to implement. Usually most of the commercial recommendation systems are hybrid; example Google news recommendation system.
