Grad Life Website

I collaborated with two other researchers for this project.

We worked with our stakeholder who was the program director for graduate student life to implement our recommended changes based on our empirical research.

This research took place over the span of 2 months as we each balanced multiple demands for our time and attention.

We conducted each research task independently and then came together to optimize our combined efforts. I was particularly involved with things like writing the interview guide and think aloud scenarios and keeping our stakeholder in the loop throughout our research.

Before meeting with our stakeholder, we familiarized ourself with the graduate student life website --a one-stop-shop that directs people to resources that will help them thrive during graduate school.

We formed an initial impression and hypothesis that the current organization of content would be ineffective and promote confusion.

During our stakeholder interview, we discovered our primary goal, which was to easily navigate the website and find resources.

Our secondary goals were to determine if the website was welcoming and inclusive and increase discoverability of resources people didn’t know they needed. We also addressed logistics and constraints.

Before beginning with participant-involved research, we performed a competitor analysis. We looked at similar websites from different universities to see how they organized resources. Every school did it different, but most used around 5 categories. This told us that there was no right way to organize the resources, but we may benefit from using more than 2 categories.

Next, we searched for methods that would suit our research goals and budgetary constraints. We only had enough money to adequately compensate 10 people for about 30 minutes of their time. We decided to have people participate in a think-aloud study to see how fast they could find resources on the website, and discover any pain points in that process. After the think-aloud study, we had them do a usability survey on their own to obtain direct reports on other questions we had, like "how welcoming was the website." In addition, we had them organize the resources into categories in a virtual card sorting task.

For our research sample, we wanted to represent the diverse student body, not just the “majority” of Notre Dame graduate students. We collected relevant demographic information on things like religious affiliation, that would help us better understand our sample.

The think aloud study was conducted using Zoom. Participants shared their screen with us while they navigated the website in response to a scenario like “Ryan is looking for apartments near campus. Where would you direct Ryan to go?” In their thought process and other reports, we observed that many resources were not easy to find under the current organization system. In this qualitative study, we also observed many minor pain points that were then used to make recommendations on the reorganization of the website.

After the think-aloud task, people completed a usability survey that asked them to rate their experiences during the think-aloud, and additional questions to inform our secondary goals. Supporting our findings from the think-aloud study, people reported that the website organization was somewhat unclear and they were not always able to find what they were looking for quickly. In addition, one person did not find the website welcoming. Again, this supports the need to reorganize the content. In addition, this indicates that more research is needed to determine aspects that make the website unwelcoming.

Finally, people were tasked with sorting 26 resources in up to 5 categories and providing category labels. Most people used 4-5 categories with common themes in the groupings. From this, we decided that it would be best to use 4-5 categories and were able to identify appropriate labels for the categories.

Based off our observations, we proposed a new organization, in addition to recommendations for combining and splitting categories as well as renaming several categories for clarity.

We also recommended testing the re-organization of the website with A/B testing in the future when there is budgetary support.

Given the project constraints and our own ux research knowledge at the time, I like to think we did the best we could. In hindsight, there are several things I would do differently in a future study like this. I’d first start off with an assumption slam to address any assumptions or biases that we as researchers have going into this. I’d also ask for any available website analytics to examine website flow both before and after user testing.

We did not have the common issue of stakeholder buy-in. In future studies, however, I would be intentional about recording user quotes for effective storytelling.

I would also improve my survey by asking more indirect questions because responses to direct questions like “Did you find the website welcoming?” are not very informative and are subject to inaccurate reporting.

Of course, with more resources like time and a larger research budget, I would have the chance to follow up this research with more foundational research to understand how people use this website in practice, conduct a/b testing to measure effectiveness of recommended changes, etc.