LIS 5275: Usability Analysis

Fall 2022, Thursdays, 8:00 to 10:00 PM, Online
Assignment Descriptions
Participation Assignments
Weekly Participation (15%)

Class participation is required for everyone. This means you need to engage in the class discussion at least once, in class, each week. You can do this very easily by asking or answering questions in the text chat, but please note that participation means making a meaningful contribution to an ongoing discussion; simply attending class does not count as participation.

You have two free passes for the semester -- no explanation needed -- and of course you won't be penalized for a legitimate, excused absence (just drop me an email explaining what is happening, before class if possible). Each day of non-participation (beyond the two free passes and excused absences) will subtract a point from your class participation grade.

Usability Disaster of the Week (5%)

Building upon our discussions of the importance of usability, each student will individually "present an interface" in class and discuss its usability flaws. These do not need to be lengthy presentations -- think of them as brief, human-interest stories! -- and we will scatter them throughout the semester (selected dates to be determined).

To complete the assignment, you will select a product or design -- physical or digital -- that you believe demonstrates an interesting usability flaw. Think about the "interface" through which someone interacts with this device, and the problems they might have using it. Bring a screenshot or picture to share with the class -- or provide a link if you've chosen something online -- and be prepared to answer the following questions:

  1. What is the object or interface you selected, and what is its intended purpose?
  2. What is the problem (or problems) with the current design? What difficulties could this cause for the user?
  3. How would you fix these problem if you were to redesign the product?

Note that this is an individual, in-class, participation assignment; you do not need write anything up or submit anything in advance.

Written Assignments
Individual Assignment 1: Designing for Users (10%)

Blog Post due Tuesday, September 6. One post per student.

This is an individual assignment that builds upon our discussions of usability and user-centered design, and the difficulties that designers have looking at things from the users' perspective. It provides an opportunity for you, as an expert user of a particular website, to assess that site's interface through the eyes of a novice user.

Select a (publicly available) web site that you use fairly often and know well. The actual site doesn't matter, as long as it consists of several pages so you can observe potential navigation problems.

Find someone (a friend, roommate, relative, etc.) who has never seen or used this site before. Ask this person to explore the site while you observe their activities; you can ask them to complete a specific task if you want, or just let them do whatever they want.

Watch what happens as they explore the site. Try to figure out what they do, what they want to do, and how the two match up. Pay particular attention to what happens when things go wrong. Remember also to look for what they didn't do, but that you might have expected they would do.

Write a blog post covering the following points, in this order, using subheadings to separate each point:

  1. Identify the website you selected (including a screenshot), and briefly -- and anonymously! -- describe your user tester, their technical expertise in general, and their familiarity with this type of site in particular.
  2. Describe your user's actions... What did you observe? What did they do that you found unexpected? What problems did they as a novice user encounter that you, as an expert user, would not have encountered?
  3. Focusing on the difference between expert and novice users, discuss how the interface helped or hindered the user in their activities from a user-centered design perspective, and explain how the site might better support novice users. Where appropriate, include embedded links to appropriate pages so that we can see the causes of the difficulties you observed.

Submit your post to the class blog in the "Designing for Users" category. Length: minimum 500 words.

Group Assignment 1: Group Topic Selection (5%)

Blog Post due Tuesday, September 13. One post per group.

This is a group assignment where you will select a group topic (i.e., type of website, such as universities, e-commerce, travel, weather, non-profits, etc.), and start thinking about the different kinds of people who might use this type of site, the different kinds of tasks they might be trying to accomplish, and the different kinds of problems they might encounter.

Working together with your group-mates, finalize your type of website / group topic selection, and identify a list of websites that serve as good examples of your chosen type of site. Spend some time discussing what your group hopes to learn from analyzing this type of site.

Write a blog post covering the following points, in this order, using subheadings to separate each point:

  1. The name of your group, and the names of your group members.
  2. The type of site your group has selected, along with a short list of sample websites.
  3. A brief statement explaining why your group is interested in this type of site -- why does it appeal to you? why you think it would be useful to explore the usability of this kind of site?

Submit your post to the class blog in the "Group Topic Selection" category. Length: minimum 250 words.

Individual Assignment 2: Heuristic Evaluation (15%)

Blog Post due Tuesday, September 27. One post per student.

This is an individual assignment. Select a website that represents your group's topic, and perform a heuristic evaluation of that site (which can be, but does not need to be, one of the exemplar sites from the group topic selection assignment). Each group member should evaluate a different website for this assignment (as best as possible). You are required to use Nielsen's 10 heuristics for this assignment, available online at: https://www.nngroup.com/articles/ten-usability-heuristics/

To guide your analysis of the site, you are required to identify a scenario of use that describes how a representative user might approach interacting with this site. This can be very general -- if you were evaluating a travel website, for example, you might think about someone trying to cash in all their frequent flier points, or someone trying to trade in a ticket for a first class seat, or someone simply looking for the best deal from point a to point b. Your goal here is to develop a basic scenario that will guide your heuristic evaluation.

Guided by this scenario, you will then systematically evaluate the interface (or a portion thereof) of your chosen site, checking to see which interface elements you encounter (if any), while following your scenario, violate any of Nielsen's heuristics. You should work from your scenario, and NOT work from the heuristics (i.e., don't take each heuristic one at a time, looking for heuristic violations on the site).

Write a blog post covering the following points, in this order, using subheadings to separate each point:

  1. Identify the website you selected (including a screenshot), and briefly describe the site being evaluated.
  2. Describe the scenario you used to focus your evaluation, and explain how you approached the site with respect to the usability heuristics.
  3. Provide a detailed analysis of the usability flaws you found for this site, along with a) the heuristics each flaw violates, and b) an explanation of why this flaw violates those heuristics. Make sure that you connect each flaw you found to at least one heuristic.
  4. Discuss some possible design recommendations that could improve the site, explaining how each recommended change would address the usability flaws you identified above. For full credit, your write-up must provide a thoughtful analysis that draws solid conclusions based on your findings.

Submit your post to the class blog in the appropriate "Group" category for your group AND the "Heuristic Evaluation" category. Length: minimum 750 words.

Group Assignment 2: User Scenarios and Representative Tasks (10%)

Blog Post due Tuesday, October 18. One post per group.

This is a group assignment that requires you to collectively decide on the specific website that all your group members will analyze (individually) for the final individual assignment (representative user test), and to re-design (collaboratively) for the group Iterative Design project. Important: all group members must agree on the same website, and this is the website your group will use for the remainder of the course!

Once you have selected a site for your group, spend some time discussing the kinds of users who would frequent your selected site. Develop ONE user profile or persona that you feel describes the users you have in mind. Feel free to think about your hypothetical users' demographics, backgrounds, education, experience, and overall goals in life. You may develop either generic profiles (e.g., Fortune 500 business executives who are frequent fliers...) or specific personas (e.g., "Steve" is a 52-year-old man from Chicago, IL, who regularly flies around the world (first class) with a cocker spaniel in tow...) -- whichever approach will be most helpful to your group!

Next, think about what these users might be trying to accomplish with your selected website. Based on the user profile or persona you've identified, develop ONE scenario of use that describes what your typical user might be trying to accomplish at your site (e.g., a primary school teacher is planning to take his or her students to a museum on a field trip, and is visiting the museum's website to look for ways of integrating the museum visit into the classroom).

Finally, develop THREE specific tasks that a representative user of your chosen site would likely have to complete as part of that scenario (e.g., someone visiting the Museum of Modern Art in New York for the first time may want to find out what special exhibits are taking place, how to get to the museum, and how much it will cost to see the exhibits). You should try to develop tasks that, in theory, would take users to different sections of the site, and involve a certain amount of complexity (e.g., needing to visit multiple pages). These tasks will form the basis for the representative user tests you will conduct (individually) in the next assignment.

You should feel free to develop tasks that target specific areas of the site (i.e., those sections that you are most interested in testing for usability). Make sure these tasks are explicit, but do not "give away" how they are to be done by using specific terms from the interface you will be testing.

Write a blog post covering the following points, in this order, using subheadings to separate each point:

  1. The chosen website your group will analyze for the remainder of the course, along with a brief discussion of why you chose this site, and what sections of the site you are most interested in testing for usability.
  2. The user profile or persona you created for the type of users who might use the site your group has chosen.
  3. The scenario of use you developed for that user profile or persona, describing how those users might approach the site you selected.
  4. The specific tasks you have developed that your test users will need to complete during the user test, along with a brief description of why you chose these tasks (i.e., why these are the types of things these types of people would do on this type of site, and what you hope to learn about the usability of your chosen site).

Submit your post to the class blog in the appropriate "Group" category for your group AND the "User Scenarios and Representative Tasks" category. Length: minimum 500 words.

Individual Assignment 3: Representative User Test (20%)

Blog Post due Tuesday, November 1. One post per student.

This is an individual assignment, but remember: each student is required to analyze the same website as their other group members.

This is an "actual" user test using an "actual" user (or users) of your site. You may not be able to find a test user that matches the user profile you identified in the previous group assignment, but you can (and should) describe the scenarios of use to your user so that they understand their "motivations."

Choose one of the formal methods of user testing described in class (e.g. Think Aloud), and perform a representative user test of your website where your user(s) try to accomplish the tasks developed in the previous group assignment. Make certain your tasks are clearly identified ahead of time, and that you are very familiar with the site being evaluated. It is perfectly appropriate for individual students to refine the representative tasks and scenarios of use at this stage, if desired, but you should try not to deviate too far from the group's expected scenarios and tasks.

Write a blog post covering the following points, in this order, using subheadings to separate each point:

  1. Identify the website you selected (including a screenshot), and briefly describe the site being evaluated.
  2. Describe, but do not identify, the characteristics of your user(s).
  3. Describe the user testing method you used during the test, and briefly explain why you chose that method for your test.
  4. Describe the tasks you assigned to your user(s), and briefly explain any changes you made to refine the scenarios or tasks from the previous group assignment.
  5. Present a detailed analysis of what happened during the completion of each task, explaining the problems your user(s) encountered and any insights you might have as to why your user(s) encountered these problems.
  6. Discuss some possible design recommendations for improving the usability of this interface, explaining how each recommended change would address issues observed during your user test. For full credit, your write-up must provide a thoughtful analysis that draws solid conclusions based on your findings.

Submit your post to the class blog in the appropriate "Group" category for your group AND the "Representative User Test" category. Length: minimum 1000 words.

Group Assignment 3: Iterative Design (20%)

Iteration 1 due Tuesday, November 15 (10%). Iteration 2 due Tuesday, December 6 (10%). One post per group per iteration.

This is a group project. Drawing upon the results of your heuristic evaluations and representative user tests, your group will work together to re-design (a portion of) your selected website. As this is the final class project, your redesign and accompanying design rationale should clearly demonstrate the usability analysis concepts you have learned throughout the course.

Think carefully about the site you are planning to re-design, and consider how you would change this interface to improve its usability -- what's wrong with it? why? how you are going to fix it? Focus on the representative scenario(s) of use you identified (how might people use this interface, what problems might they encounter, etc.). You will likely not be able to fix everything you want, so be sure to identify the most important features and explain why you chose them. You can do this by page, by task, by feature, or by whatever makes the most sense for your site.

Please note that you do NOT have to create a WORKING version of the interface; you are simply creating a non-working, prototype design. You can use any type of design program (e.g. HTML editors, Photoshop, Visual Basic) -- just be certain you have something visual to show and say "this is what our proposed re-design looks like." If you want to add some interactive features, that's fine, but all you need is a series of static pages or illustrations showing the changes you are proposing.

The point of this assignment is to explore the value of prototyping and iterative design for improving the usability of interfaces. For each design change you make, you will need to develop an illustration or screenshot that shows how each proposed re-design would work, and write a design rationale that explains your reasoning behind that particular interface element. Each time you make a change, be sure to document why you made that change and why you think your changes will increase the usability of the site. Please use the results of your user testing experiences (and any comments on your early designs, as applicable) to support your design changes.

You will naturally go through a variety of different ideas as you do your redesign. Feel free to show me as many "early" designs, crazy ideas, whatever as you want. I want to see your design process so that I can understand your final redesign better. The more you show me about your design process, the better I understand what you've learned from the class.

Please note the importance of providing a design rationale that supports your design changes, and explains why you believe the changes you have made will solve the usability problems you've identified. As you develop your ideas and receive feedback from the in-class iterative design workshops, you should iterative expand your design rationale. While your re-design ideas are important, do not neglect the importance of this design rationale -- the explanations of what you did are just as important (if not more so) than the design changes themselves.

You can have as many iterations as you want (each submitted as a separate post), but at a minimum, you need to have two iterations by the end of term. It is not a good idea, however, to think about this project in terms of discrete iterations. You will have a far superior end product, with many more than two iterations, if you simply keep working on your design, trying out new ideas and writing up your design rationales over the course of the project.

Post your re-design ideas, accompanied by illustrations/screenshots along with your design rationale, as separate posts for each iteration to the class blog in the official category for your group AND the "Iterative Design" category. Length: minimum 500 words of design rationale to explain each iteration of your design.

The idea is that by reading your posts in order, it should be possible to follow the iterative design of your proposed interface from start to finish. Your posts should make it clear that you have understood and can apply the concepts of the course, and that you can explain the value of usability in the design process. How and why your design changed over time is just as important as what your final version looks like.

Notes on Written Assignments

Submissions are due by 11:59 p.m. Eastern Time (EDT or EST, as appropriate) on the dates indicated above. Late assignments will not receive full credit. If you have extenuating circumstances, please contact me ASAP.

When the papers are marked, points will be assigned based on the assumption that you have carefully followed, read, and understood the assignment descriptions. Please do not hesitate to ask for clarifications!

All assignments should be posted to the class blog as separate, individual posts, excepting group assignments, which will have one post per group, per assignment.

You are required to use headings and subheadings to make the organization of your paper clear to the reader. The main headings in your paper must correspond to the main sections of the assignment descriptions, listed above. Subdivide each section to reflect its content as necessary.

Papers must be demonstrably spell-checked, grammar-checked, and proofread for nonsense; failure to do so will result in a lower grade.