Evaluation

The objective of the competition

The goal of the participant is to rank the target variable for each stock in the Adia Lab investment universe, from the highest to the lowest, at each given date.

This doesn't require estimating the exact target value for each investment; rather, it involves identifying which investments are likely to perform better than others. Participants can obtain this information from the various features (or Xs) describing each investment at each date in the provided dataset. The features' meanings are unknown to both CrunchDAO and the participants to prevent bias and facilitate sharing of the anonymized dataset.

The scoring metric

This competition is evaluated on Spearman Rank Correlation.

Each row in the test set represents the predictions (X) associated with a stock of the universe at a given date and its target (Y).

$$r_{s}=\rho _{\operatorname {R} (X),\operatorname {R} (Y)}={\frac {\operatorname {cov} (\operatorname {R} (X),\operatorname {R} (Y))}{\sigma _{\operatorname {R} (X)}\sigma _{\operatorname {R} (Y)}}}$$

Where:

• $\rho_{R(X),R(Y)}$ denotes the usual Pearson correlation coefficient, but applied to the ranked variables $X$ and $Y$;

• $\operatorname {cov}(R(X), R(Y))$ is the covariance of the ranked variables;

• $\sigma_R$ are the standard deviations of the ranked variables.