1. What is HCI Evaluation?

Definition: HCI evaluation is the process of assessing the usability, effectiveness, and user satisfaction of an interactive system. It ensures that the system meets user needs, is efficient, and minimizes errors.

Key Idea: Evaluation is essential in iterative design to identify problems, validate design decisions, and improve the overall user experience.

2. Types of Evaluation in HCI

HCI evaluation methods are broadly classified into two categories:

A. Formative Evaluation

• Conducted during the design process
• Purpose: Identify usability problems and improve the design before final release
• Examples: User testing of prototypes, heuristic evaluation

B. Summative Evaluation

• Conducted after the system is developed
• Purpose: Assess overall effectiveness and usability
• Examples: Final user testing, surveys, analytics

3. HCI Evaluation Techniques

HCI evaluation techniques can be grouped into three main categories:

A. Expert-Based Evaluation

• Performed by usability experts using knowledge of HCI principles

• Methods include:

  1. Heuristic Evaluation – Checking the interface against established heuristics (e.g., Nielsen’s heuristics)
  2. Cognitive Walkthrough – Experts step through tasks as if they were users, identifying potential problems
  3. Guidelines/Checklists Review – Using design rules and guidelines to inspect the interface

Pros: Quick, cost-effective, identifies many problems early Cons: May miss problems only real users encounter

B. User-Based Evaluation

• Involves real users performing tasks on the system

• Methods include:

  1. Usability Testing – Observing users performing tasks, measuring effectiveness, efficiency, and satisfaction
  2. Surveys and Questionnaires – Collect subjective user feedback (e.g., SUS – System Usability Scale)
  3. Interviews and Focus Groups – Explore user opinions, expectations, and experiences
  4. Think-Aloud Protocol – Users verbalize their thought process while performing tasks

Pros: Provides real user feedback, identifies unexpected issues Cons: Requires more time and resources

C. Automated / Analytical Evaluation

• Uses software tools or simulations to evaluate usability

• Methods include:

  1. Performance Metrics Analysis – Task completion time, error rates, clicks, navigation patterns
  2. Logging and Analytics – Track user behavior in real-time systems
  3. GOMS / KLM Models – Predict task execution time based on human-computer interaction models

Pros: Provides quantitative data, supports large-scale analysis Cons: May not capture subjective experiences

4. Common Usability Metrics

• Effectiveness: Accuracy and completeness in achieving goals (task success rate)
• Efficiency: Resources used to achieve goals (time, effort, number of steps)
• Satisfaction: User comfort, enjoyment, and perceived ease of use
• Learnability: How easy it is for new users to accomplish tasks
• Error Rate: Frequency and severity of user mistakes

5. Steps in HCI Evaluation

1. Define goals and objectives – What aspects of usability are being tested
2. Select methods – Choose expert-based, user-based, or analytical methods
3. Plan and recruit participants (if user-based)
4. Conduct evaluation – Observe, collect data, and record issues
5. Analyze results – Identify problems, patterns, and areas for improvement
6. Recommend improvements – Feed findings back into the iterative design process

Key Takeaways

• HCI evaluation is essential to ensure systems are user-friendly and effective.
• Techniques can be expert-based, user-based, or automated/analytical.
• Evaluation provides quantitative and qualitative data for improving usability.
• Iterative evaluation throughout the design process ensures continuous improvement.