Recommendation system

In this post, we visit various approach used in recommendation system. For each approach, i will walk through major points. This post is not in depth explanation, but revision of various approach followed in research and industry.

The content of this post is as follows:

  • Content Based
  • Collaborative filtering
  • Hybrid Approach
  • Matrix factorization
  • Deep Learning Based recSys
  • Graph Based inference model
  1. Content Based Method
    1. Based on user history
    2. Not helpful for cold-start problem
    3. Feature would be likes of product, location, feature of product we bought etc
  2. Collaborative filtering Approach:
    1. Interaction Based feature
      • user-user interaction (I like sitcom tv series, it find the similarity between users and recommend product/movies of users, who have same flavor as me)
      • item-item interaction (Recommend similar product on amazon)
    2. can handle cold-start problem very well
    3. KNN algorithm to measure similarity
  3. Hybrid Approach:
    1. User History
    2. Interaction of user-user or item-item
    3. Most company use this approach
  4. Matrix factorization:
    1. We need to predict the missing entries in matrix for user-item rating, it can be done using SVD as R = U S V', where U is user-feature, S is eigen-value and V is Item-features
    2. It can be done using Alterning Optimization approach with loss function of |r_{ij} - u_i S v_j|^2
  5. Probabilistic Matrix Factorization:
    1. We can include user and item known history/feature as well to learn better latent-representation.
    2. Loss funtion is |r_{mn} - u_n v_m|^2 + |u_n - W_u a_n|^2 + |v_m - W_v b_m|^2 , where a_n is feature or history of nth user and b_m is the history/feature of mth item.
    3. We can even add regularization on u_n and v_m
    4. Work fantastically in Netflix-movies
  6. Deep Learning Based
    1. deep and shallow network approach
    2. Deep network will use word-embedding of product description and shallow-network use the user-history as feature or meta-data
    3. implemented in You-tube recSys
    4. There are many possibile way to build network and use feature, play with word embedding
      • time series content
      • time distributed layer for parsing document on item
      • tfidf feature
      • svd feature
    5. can even used doc2vec for each document. Use gensim doc2vec for training. **Main idea is that, while training we use the same approach as word2vec, except than we add a document tag with it, which maintains the context for each doc as well
  7. Another Deep Learning Approach:
    1. two network, one for users and other for items
    2. Compute user-item interaction by dot or cosine product
    3. Build more dense layer to have more complex representation
    4. use multiclass cross entropy loss for 5-star rating
    5. We can also user sparse implict feedback feature, which are binary in nature (implicit feedback which is generated by system on click-based and explicit-feedback is collected by likes, review and purchasing history)
  8. Graph Based Network
    1. use graph embedding to build NN (deep walk, random-walk)
    2. use each feature as node and interaction as edge. For example, for movie recommendation system, we have 3 user, 5 movies, 8 actors, 5-star rating, 10 genres we can use each attribute as a node and then for each interation, we create an edge as user1 like actor1 and given 4-star
    3. can use node2vec, where each node is represented by vector which is trained on same concept of random-walk
    4. Current state of art recSys platform follows this.

Some practical insights of Recommender System

  1. Netflix uses 100s of different base model, final prediction is the weighted average of all. (Generally non-linear blending is preferred)
  2. Weighted Hybrid: Choose 10 items from users rating for collaborative filtering as well as from content based filtering. Now make a list using 60% weighted collaborative and 40% weighted content list. Finally sort the list.
  3. Mix Hybrid: Take 5 items from content, 5 from user history, 5 from trending and 5 from others
  4. Switching: confidence: If user is logged in, switch to collaborative filtering, otherwise switch to content filtering.