Evaluation methods¶
Base Method¶
@author: Quoc-Tuan Truong <tuantq.vnu@gmail.com>
-
class
cornac.eval_methods.base_method.
BaseMethod
(data=None, fmt='UIR', rating_threshold=1.0, exclude_unknowns=False, verbose=False, **kwargs)[source]¶ Base Evaluation Method
Parameters: - data (array-like) – The original data.
- data_format (str, default: 'UIR') – The format of given data.
- total_users (int, optional, default: None) – Total number of unique users in the data including train, val, and test sets.
- total_users – Total number of unique items in the data including train, val, and test sets.
- rating_threshold (float, optional, default: 1.0) – The threshold to convert ratings into positive or negative feedback for ranking metrics.
- exclude_unknowns (bool, optional, default: False) – Ignore unknown users and items (cold-start) during evaluation.
- verbose (bool, optional, default: False) – Output running log
-
evaluate
(model, metrics, user_based)[source]¶ Evaluate given models according to given metrics
Parameters: - model (
cornac.models.Recommender
) – Recommender model to be evaluated. - metrics (
iterable
) – List of metrics. - user_based (bool) – Evaluation mode. Whether results are averaging based on number of users or number of ratings.
- model (
-
classmethod
from_splits
(train_data, test_data, val_data=None, data_format='UIR', rating_threshold=1.0, exclude_unknowns=False, verbose=False)[source]¶ Constructing evaluation method given data.
Parameters: - train_data (array-like) – Training data
- test_data (array-like) – Test data
- val_data (array-like) – Validation data
- data_format (str, default: 'UIR') – The format of given data.
- rating_threshold (float, default: 1.0) – Threshold to decide positive or negative preferences.
- exclude_unknowns (bool, default: False) – Whether to exclude unknown users/items in evaluation.
- verbose (bool, default: False) – The verbosity flag.
Returns: method – Evaluation method object.
Return type: <cornac.eval_methods.BaseMethod>
Ratio Split¶
@author: Quoc-Tuan Truong <tuantq.vnu@gmail.com>
-
class
cornac.eval_methods.ratio_split.
RatioSplit
(data, fmt='UIR', test_size=0.2, val_size=0.0, rating_threshold=1.0, shuffle=True, seed=None, exclude_unknowns=False, verbose=False, **kwargs)[source]¶ Train-Test Split Evaluation Method.
Parameters: - data (.., required) – The input data in the form of triplets (user, item, rating).
- fmt (str, optional, default: "UIR") – The format of input data: - UIR: (user, item, rating) triplet data - UIRT: (user, item , rating, timestamp) quadruplet data
- test_size (float, optional, default: 0.2) – The proportion of the test set, if > 1 then it is treated as the size of the test set.
- val_size (float, optional, default: 0.0) – The proportion of the validation set, if > 1 then it is treated as the size of the validation set.
- rating_threshold (float, optional, default: 1.) – The minimum value that is considered to be a good rating used for ranking, e.g, if the ratings are in {1, …, 5}, then rating_threshold = 4.
- shuffle (bool, optional, default: True) – Shuffle the data before splitting.
- seed (bool, optional, default: None) – Random seed.
- exclude_unknowns (bool, optional, default: False) – Ignore unknown users and items (cold-start) during evaluation and testing
- verbose (bool, optional, default: False) – Output running log
-
evaluate
(model, metrics, user_based)[source]¶ Evaluate given models according to given metrics
Parameters: - model (
cornac.models.Recommender
) – Recommender model to be evaluated. - metrics (
iterable
) – List of metrics. - user_based (bool) – Evaluation mode. Whether results are averaging based on number of users or number of ratings.
- model (
Cross Validation¶
@author: Aghiles Salah
-
class
cornac.eval_methods.cross_validation.
CrossValidation
(data, fmt='UIR', n_folds=5, rating_threshold=1.0, partition=None, exclude_unknowns=True, verbose=False, **kwargs)[source]¶ Cross Validation Evaluation Method.
Parameters: - data (.. , required) – Input data in the triplet format (user_id, item_id, rating_val).
- n_folds (int, optional, default: 5) – The number of folds for cross validation.
- rating_threshold (float, optional, default: 1.) – The minimum value that is considered to be a good rating, e.g, if the ratings are in {1, … ,5}, then rating_threshold = 4.
- partition (array-like, shape (n_observed_ratings,), optional, default: None) – The partition of ratings into n_folds (fold label of each rating) If None, random partitioning is performed to assign each rating into a fold.
- rating_threshold – The minimum value that is considered to be a good rating used for ranking, e.g, if the ratings are in {1, …, 5}, then rating_threshold = 4.
- exclude_unknowns (bool, optional, default: False) – Ignore unknown users and items (cold-start) during evaluation and testing
- verbose (bool, optional, default: False) – Output running log
-
evaluate
(model, metrics, user_based)[source]¶ Evaluate given models according to given metrics
Parameters: - model (
cornac.models.Recommender
) – Recommender model to be evaluated. - metrics (
iterable
) – List of metrics. - user_based (bool) – Evaluation mode. Whether results are averaging based on number of users or number of ratings.
- model (