Deep Learning Programming

Programming Assignment: Deep Learning for Content-Based Filtering

Lab Overview

Objective: Implement content-based filtering using a neural network to build a recommender system for movies using rich user and movie features.

Dataset Details

• Source: MovieLens ml-latest-small dataset
• Reduced dataset: nu = 397 users, nm = 847 movies, 25,521 ratings
• Focus: Movies from years since 2000 and popular genres
• Rating scale: 0.5 to 5 in 0.5 step increments
• Movie features: Title, release date, one or more genres
• User features: Limited information other than ratings

Content-Based vs Collaborative Filtering

Key Difference

Content-based filtering generates user and movie feature vectors but recognizes there may be other information available about the user and/or movie that may improve the prediction. The additional information is provided to a neural network which then generates the user and movie vector.

Feature Engineering

Movie Content Features

• Original features: Year released, genre as one-hot vector (14 genres)
• Engineered features: Average rating derived from user ratings

User Content Features

• Engineered features: Per genre average rating computed per user
• Additional data: User ID, rating count, rating average (not used in training)

Training Data Structure

The training set consists of all ratings made by users, with some ratings repeated to boost underrepresented genres. Split into:

• User array: User features for each rating
• Movie/item array: Movie features for each rating
• Target: y_train containing the actual ratings

Key Variables

• num_user_features: User features used in training (excludes user ID, rating count, avg rating)
• num_item_features: Item features used in training (excludes movie ID)
• uvs/ivs: Start indices for genre vectors
• u_s/i_s: Start indices for training columns

Exercise 1: Neural Network Implementation

REQUIRED CODE: Implement User and Item Networks

You need to create two sequential neural networks with identical architectures:

# User Network
user_NN = tf.keras.models.Sequential([
  ### START CODE HERE ###
  tf.keras.layers.Dense(256, activation='relu'),
  tf.keras.layers.Dense(128, activation='relu'),
  tf.keras.layers.Dense(num_outputs)  # num_outputs = 32
  ### END CODE HERE ###
])

# Item Network
item_NN = tf.keras.models.Sequential([
  ### START CODE HERE ###
  tf.keras.layers.Dense(256, activation='relu'),
  tf.keras.layers.Dense(128, activation='relu'),
  tf.keras.layers.Dense(num_outputs)  # num_outputs = 32
  ### END CODE HERE ###
])

Network Requirements

• First layer: Dense layer with 256 units and ReLU activation
• Second layer: Dense layer with 128 units and ReLU activation
• Third layer: Dense layer with num_outputs units and linear/no activation
• Output: Both networks output 32-dimensional vectors

Architecture Note

The networks don’t need to be identical - if user content was substantially larger than movie content, you might increase the complexity of the user network. In this case, content is similar so networks are the same.

Data Preprocessing

Feature Scaling

The lab uses scikit-learn scalers:

# Scale input features
scalerItem = StandardScaler()
item_train = scalerItem.transform(item_train)

scalerUser = StandardScaler()
user_train = scalerUser.transform(user_train)

# Scale target ratings to [-1, 1]
scalerTarget = MinMaxScaler((-1, 1))
y_train = scalerTarget.transform(y_train.reshape(-1, 1))

Train/Test Split

# Split data (80/20)
item_train, item_test = train_test_split(item_train, train_size=0.80, shuffle=True, random_state=1)
user_train, user_test = train_test_split(user_train, train_size=0.80, shuffle=True, random_state=1)
y_train, y_test = train_test_split(y_train, train_size=0.80, shuffle=True, random_state=1)

Model Architecture

The complete model uses Keras functional API:

# User input processing
input_user = tf.keras.layers.Input(shape=(num_user_features))
vu = user_NN(input_user)
vu = tf.linalg.l2_normalize(vu, axis=1)  # Normalize to unit length

# Item input processing
input_item = tf.keras.layers.Input(shape=(num_item_features))
vm = item_NN(input_item)
vm = tf.linalg.l2_normalize(vm, axis=1)  # Normalize to unit length

# Compute dot product
output = tf.keras.layers.Dot(axes=1)([vu, vm])

# Create complete model
model = tf.keras.Model([input_user, input_item], output)

Training Process

# Configure and train model
cost_fn = tf.keras.losses.MeanSquaredError()
opt = keras.optimizers.Adam(learning_rate=0.01)
model.compile(optimizer=opt, loss=cost_fn)

# Train for 30 epochs
model.fit([user_train[:, u_s:], item_train[:, i_s:]], y_train, epochs=30)

Making Predictions

New User Recommendations

# Create new user profile
new_user_id = 5000
new_rating_ave = 0.0
new_action = 0.0
new_adventure = 5.0  # Likes adventure movies
new_animation = 0.0
# ... other genre preferences
new_fantasy = 5.0    # Likes fantasy movies

user_vec = np.array([[new_user_id, new_rating_count, new_rating_ave,
                    new_action, new_adventure, new_animation,
                    # ... all genre preferences]])

# Generate predictions for all movies
user_vecs = gen_user_vecs(user_vec, len(item_vecs))
suser_vecs = scalerUser.transform(user_vecs)
sitem_vecs = scalerItem.transform(item_vecs)

y_p = model.predict([suser_vecs[:, u_s:], sitem_vecs[:, i_s:]])
y_pu = scalerTarget.inverse_transform(y_p)  # Unscale predictions

Exercise 2: Distance Function

REQUIRED CODE: Implement Squared Distance

def sq_dist(a, b):
  """
  Returns the squared distance between two vectors
  Args:
    a (ndarray (n,)): vector with n features
    b (ndarray (n,)): vector with n features
  Returns:
    d (float): distance
  """
  ### START CODE HERE ###
  d = np.sum(np.square(a - b))
  ### END CODE HERE ###
  return d

Expected Results

• Distance between identical vectors: 0.000
• Distance between [1.1,2.1,3.1] and [1.0,2.0,3.0]: 0.030
• Distance between [0,1,0] and [1,0,0]: 2.000

Finding Similar Movies

Extract Movie Feature Vectors

# Create model to extract movie vectors
input_item_m = tf.keras.layers.Input(shape=(num_item_features))
vm_m = item_NN(input_item_m)
vm_m = tf.linalg.l2_normalize(vm_m, axis=1)
model_m = tf.keras.Model(input_item_m, vm_m)

# Generate feature vectors for all movies
scaled_item_vecs = scalerItem.transform(item_vecs)
vms = model_m.predict(scaled_item_vecs[:,i_s:])

Compute Distance Matrix

# Compute all pairwise distances
count = 50
dim = len(vms)
dist = np.zeros((dim,dim))

for i in range(dim):
  for j in range(dim):
      dist[i,j] = sq_dist(vms[i, :], vms[j, :])

# Mask diagonal to avoid self-similarity
m_dist = ma.masked_array(dist, mask=np.identity(dist.shape[0]))

Key Implementation Requirements

What You Must Implement

1. Neural Networks: Complete user_NN and item_NN with proper architecture
2. Distance Function: Implement sq_dist() for finding similar movies
3. Understanding: Comprehend feature engineering and scaling processes
4. Predictions: Create and evaluate predictions for new users

Expected Learning Outcomes

After completing this assignment:

• Understand content-based filtering with neural networks
• Experience with feature engineering for recommender systems
• Learn about scaling and preprocessing for neural network training
• See how to find similar items using learned feature vectors
• Apply the system to real movie recommendation scenarios

Tip

Success Tip Pay attention to the feature engineering section - understanding how user and movie features are constructed is crucial for building effective content-based recommender systems.