DELOITTE DIGITAL 2020
I led the redesign of a premier .gov career application flow from discovery to testing. Time to application for target candidates was successfully reduced from several minutes to 20-30 seconds.
7 minutes to read, but feel free to skim. Includes scrubbed research artifacts and models. Scrubbed wireframes on request.
This project is not yet complete, so I can't share finer details or designs (yay gov). Instead, I have provided a light and censored summary of the project and my accomplishments to date below.
If you want to get to know my process more, feel free to contact me. I will gladly nerd out for hours on my strategy. 🤓
Our client was a premier government agency looking to redesign their website. We had two key goals: increase the number of diverse and qualified “top-talent” STEM applicants and improve their perception of the agency's brand.
Before this project was green lit, I was a part of a separate team that conducted stakeholder researcher several months earlier. I served as a UX researcher and was responsible for planning and facilitating interviews with 12 unique offices within the agency. We decided to conduct the interviews in a panel format given that the employees provided within each office were of similar level (no power conflicts) and opinion (homogenous in goals) from prior informal conversations.
After conducting qualitative analysis on over 730 utterances of anecdotal data, I was able to output a list of 15 high-level goals e.g. There are two distinct purposes for the website - to recruit and to educate. We printed out each of these goals onto physical note cards and facilitated a prioritization workshop with all of the offices present. Through this workshop, the offices were able to come to consensus on what was feasible and desired for their new website. The decided upon key goals were to increase the number of diverse and qualified “top-talent” STEM applicants and improve the their perception of the agency.
Fast forward back to the present. Despite having business goals established by our stakeholder interviews, we were still in the dark when it came to actual users of the website. There were also some particularly tricky challenges I had to navigate as the research lead:
Instead of panicking, I began to plan out a research strategy based on my constraints. During the first week of research I:
A top-talent STEM candidate can't be pinned down to a single gender, race, or ability/disability (to mention a few factors). I wanted to make sure we could talk to many different kinds of people so we could build an experience that could say, "hey, you belong here"- regardless of any factors beyond one's qualifications to do the job.
Since we couldn’t use an outside vendor for recruiting, I began by gathering a large list of potential interviewees by making use of the broader team's connections. Since convenience sampling runs the risk of recruiting homogenous groups, I created a live confluence table inspired by the Matrix of Oppression.
I plotted the potential interviewees onto the table to clearly visualize and identify where our unrepresented intersections were. This allowed me to pivot and refocus the team’s recruitment efforts specifically onto those unrepresented intersections until all gaps were filled or directly brought up to leadership (it’s better to recognize where exclusion is than to ignore it outright).
I selected 10 individuals out of a list of 30+ to schedule interviews with for the following week. To double check my unconcious biases, I asked one of our design analysts to create her own list based on the matrix for validation purposes. We promptly scheduled interviews with the potential interviewees that matched on both our our lists (~70%), and talked through our reasoning for the remaining interviewees before coming up with our final selection together.
By the second week, I was ready to begin interviews. I had already finished training my secondary interviewers through an interactive interviewing workshop.
Each interview was split into two parts: a general background interview and a contextual goal-based exercise with prompts to search for a job in order to observe live behavior. The background interview in the first half allowed us to select what prompt to provide the interviewee. For example, if we learned that they had recently conducted a job search on a website, we would ask them to show us how they did it.
At the end of each interview, we asked the interviewee if they would be interested in keeping in touch; I wanted to be able to include them in the design process later on during wireframing and testing. Once the interviewee left, I checked in with the secondary interviewer to discuss our key takeaways. The secondary interviewer would then transcribe the interview into my pre-formatted excel spreadsheet which would set us up nicely for contextual analysis.
I decided to use Hartson’s trusty WAAD (Work Activity Affinity Diagram) method to conduct contextual analysis as it was a good fit given the scale and complexity of the anecdotal data. It also would allow me to get the entire team engaged with the research process. Historically this has helped me sync up cross-competency teams, save time, build empathy with end users, and ease design handoffs.
During the WAAD, we had 20+ participants including product managers, designers, developers, and more in attendance over the course of two days! I prepared a short introductory workshop to explain the rules of the game, and we were off to a quick start. Incredibly, we were able to organize over 600+ lines of trackable anecdotal data in just two days.
Thanks to the WAAD, we were now familiar with what functions and features were most important to top-talent STEM candidates. Although the WAAD naturally categorized these in an easily digestible manner by topic and chronological order, there was still more to abstract in terms of behavioral patterns. For example, could we identify a pattern between filters used (e.g. salary) and make-or-break values (e.g. clout)?
Using Alan Cooper’s behavior mapping method, I drew out several scales across a variety of categories identified from the WAAD including filters, devices, 3rd party services, values, and more. I then led a small team of design analysts to plot the interviewees (numbered 1-10) onto the scales using small colorful stickies. This allowed us to identify reoccurring clusters of interviewees which we named and converted into our first provisional personas.
We continued to flesh out the personas by cross-checking them with our anecdotal data until we felt confident they were well representative of all the individuals we had interviewed. To mitigate unconscious bias and to encourage inclusion, I also opted to use monikers instead of real names and pictures. For example, “The G2G (got to go)” can help focus our conversations on a key behavior trait instead of say “John Gates, the Harvard Graduate” which may unintentionally transform a persona into a stereotype. This process ultimately output 4 final personas.
Our journey maps featured each persona’s expected route to apply to a job layered on top of a rudimentary emotional chart which mapped how those expectations would be met or unmet in the current state of the website we were redesigning. Collectively, this helped us understand which parts of the experience were more important or less important for each of the personas. This in turn allowed us to contextually select a primary persona based on which part of the experience we were designing. For example, one persona- the G2G- was significantly more likely to drop off from the experience early on if they were not provided easily scannable job information. I designated the G2G as our primary for our search results page, but not for our benefits page which was not a highly valued part of their experience.
Due to the nature of work, I cannot publish any wireframes publicly. Please contact me to get the full scoop.
Regardless if you're a student looking for directions, my future boss, or a complete stranger that got lost on this portfolio-- I'm here for it all. No hidden fees apply.
Send me an email at firstname.lastname@example.org and expect a response in 2 days. Or else I owe you a donut.
We can be LinkedIn friends too, whatever that means?
12.2020. Me shamelessly in front of a wall of donuts at Lotte World in Seoul, Korea.