If ideation is an exploration with no limits, concept evaluation should bring the limits back. The goal of concept evaluation is to understand either proposed ideas of how the product can be improved are useful and feasible and choose a single concept.

Preparation

• Set evaluation criteria (defined in a project goal)
• Write evaluation scenarios
• Prepare prototypes
• Prepare evaluation inventory
• Get a budget confirmation
• Find a place where you invite people
• Schedule workshops and interviews
• Invite target audience representatives
• Invite technical experts

Note: You can mix a group of technical people with nontechnical to increase empathy within you project team.

Evaluation criteria

I recommend to reuse defined in the goal UX metrics through all evaluation sessions and track the change over the time.

By this moment in your project you must already have some measures got from benchmark to compare to. If not, you can include one benchmark (competitors product) sample into your evaluation and set a baseline.

Evaluation inventory

Here I call an evaluation inventory on a “One participant set” a set of documents you need to conduct a formal user experience interview with a participant:

• consent form a user study contract
• that can be combined with NDA,
• demographics form,
• description of tasks for a user to perform during study,
• closing questionnaires.

Evaluation sessions

• Run a workshop or a series of user interviews with target audience representatives to review concepts for usefulness.
• Run a workshop with technical experts (engineers) to review the concept for feasibility. If it is possible, invite stakeholders to this session to save some time for explanation later.

Expected results

As a result of concept evaluation you have to be able to:

• choose a single concept or get enough information to synthesize a new one,
• based on end-users feedback list core design principles for the product,
• based on technical review list  limitations for your product.

Note: This days technology developers so quickly that ideas not feasible today can become feasible in a few month from now.

As soon as your design is complete, it is useful to assemble a prototype and to evaluate it.

Why? It will most probably take several month for developers to implement it but you can find and fix some problems. When design is implemented in prototype, give it target users to use.

Important

• Consider another sample so users have something to compare the prototype with;
• Define evaluation metrics to see how much the proposal is different from the reference;
• Define acceptance criteria;
• Choose participants according to personas;
• Choose user tasks from the core use-cases;
• Document discovered problems.

Expected result
Design release and a start of the next step in the project (development) or if needed recommendations for improvement and one more design iteration.

While developers go on with implementation, consider several evaluation sessions along the way. Compare measured metrics. Make sure the design/product is still usable after implementation.