The Objective
This Challenge has two objectives:
- As an industry organization, the DMA focuses on educating the analytic community about analysis, modeling, and data manipulation. The session and Challenge are also intended to teach analytic techniques to event attendees.
- Recognize companies that achieve superior performance in analytics.
The Challenge
The importance of predicting customer lifetime value to drive personalized customer interactions is becoming increasingly important. The 2016 Challenge is to build the most accurate model possible to predict customer expected value. The winners will be selected using mean squared error (lowest) overall, and the quality and clarity of the presentation of the prediction methodology and model development process.
Each participant in the Challenge will receive a data set(s) for model building and validation. Data includes:
- Model development data set which includes 768,168 records
- Model validation data set which contains 84,910 records
- Data dictionary/meta data explaining the dependent and independent variables and the structure of the dataset-elapsed time, listing and time stamp of independent and dependent variables for both development and validation dataset
Participants will be required to post a scored validation data set to the competition FTP site by Friday, September 30, 2016. This will be used to evaluate solutions and identify the top two solutions. Additional details regarding the evaluation criteria will be provided when the modeling data set is released.
In addition to presenting at the &THEN conference in October (October 16-18), winners will be invited to present during a future DMA webinar.
Documentation Requirements
- Overall PowerPoint presentation (no more than ten pages) outlining:
- the technical results showing the accuracy and performance of the model against the validation data set via charts and accuracy diagnostics,
- the explanation of model algorithm/equation- what predictor variables it contains and their relative importance to prediction
- what each variable means/how it was developed, and a non -technical explanation of how the overall model predicts the accuracy of customer lifetime value
- Scoring code must be provided when the scored validation file is submitted
- Variable creation – variables that were used in the model, need to be traced from input to coefficient / node (if using a tree based methodology). This includes all recodes, transformations, and summarizations. This information should be presented in a table, with English labels for variables, and equations (if appropriate) for transformations or recodes.
Judging the Results
- Evaluation results – cumulative gains tables – will be reviewed by EY and a subset of the DMA Analytics Community Leadership Council
- The Judges Panel will identify winning solutions based on criteria that will be specified with the data packet
- The Panel will not know which companies have won, but will know which models were the winners
- Winners will be notified by Friday, October 7, 2016.
DMA Presentation Requirements
- The top 2 solutions will be featured during a session at &THEN, DMA’s annual event as well as during the Analytics Community breakfast all taking place October 16-18, 2016 in Los Angeles, CA.
- Each participating company agrees to have a company presenter available at &THEN for the Analytic Challenge session, prepared to present. Note that you will need to register as an attendee for the conference.
- There will be a check-in for participating companies prior to commencing the session. Any participating company without a presenter will be excluded from the competition at that time.
- The participating company will be allowed to use company logos in the presentation. However, the presentation will be converted to a standard slide template for presentation consistency purposes. The logo will be included.
Documentation requirements for the final Presentation
- 1. Winners will be notified on Friday, October 7, 2016, and must prepare a 7-8 minute presentation for use at the October 17 session and attend the October 18 breakfast, both to be held during DMA’s &THEN Conference in Los Angeles. This presentation must include the following components:
a) The types of exploratory analyses conducted to select variables used in the model
b) The steps taken with the raw data provided to convert into modeling variables
c) Any variable reduction techniques used
d) The steps undertaken in the modeling process
e) How the final model was determined to submit for the competition
- The presentation must be in PowerPoint using the DMA template, and submitted to DMA by Friday, October 14, 2016.
- All presentations will be reviewed and edited for content and educational components. As such, you may be contacted to review changes. Refusal to reveal educational content will be grounds for removal from the competition. By agreeing to participate, you are agreeing to fully share the steps and methodologies used in your model.
Additional Requirements
- As a DMA Conference session, the DMA retains non-exclusive rights to the material presented during the Analytics Challenge session
- All PR releases regarding the DMA Analytic Challenge must include a statement that it is
“powered by EY and hosted by the DMA Analytic Community”
- Data provider EY has exclusive rights and ownership of the model algorithm at no additional charge, including up to 5 hours of support to transition the model to them
Questions? Contact Laura Gigliotti at lgigliotti@the-dma.org or 212.790.1536.
