As a panel judge, I was part of an evaluation team consisting of C-level or VP-level executives, and several founders from a wide range of industries in "exceptionally good company."
The goal of the evaluation process was to hunt for "needles-in-the-haystack" and "diamonds-in-the-rough."
Research shows that picking novel early-stage innovation projects typically goes very poorly (Boudreau et al. 2016).
Even when referring to an expert there remains the following challenges:
Finding needles-in-the-haystack
Understanding ideas when they remain half-baked
Dealing with profound uncertainty
Overcoming biases (esp. of what worked in the past)
Not just picking ideas—but directing, steering, and guiding them!
This evaluation panel was designed to overcome these limitations with an innovative approach that combined the inputs of accomplished experts with data science methods to make sense of patterns across diverse views.
Games now include endless possibilities for interactions in physical spaces (ie. Pokemon Go).
The goal of this evaluation was to identify high-quality next-generation game concepts we can further develop, optimize, prototype, and trial.
We were randomly assigned proposals to evaluate from hundreds of game concepts developed in a recent challenge at the IoT Open Innovation Lab.
Each proposal had three parts (use case, technical architecture, business case) and we were to score them on several dimensions (concept quality, prototype feasibility, idea novelty, proposal clarity).