1. Project Overview

• Problem Statement: At U.S. Bank (USB) in mid 2023, we were aiming to improve Customer Satisfaction (CSAT) scores on Bill Pay (BP) for our web interface, called Online Banking (OLB). They were low compared to mobile and to other product lines. The Customer Experience (CX) team delivered multiple analyses showing that there were issues both below and above the glass. It was not clear how to use all that information to inform strategy and make decisions moving forward, and this is where I come in.

• Research Objectives: Tie the existing information together and do any other research necessary to understand users’ needs and come up with a cohesive strategy.

• Business Goals: Retain customers and attract prospects.

• Research Objectives: Identify UI issues and gaps that exist between our experience and customer expectations. Understand how customers and prospects pay bills and approach bill management. Explore what customers and prospects need and expect from the Bill Pay experience.

• Role & Contribution: I created the plan; conducted the qualitative research; worked with the CX team to integrate their findings; worked with the UX team to plan the competitive review, synthesized the findings; ideated, crafted, and presented the findings, user needs, and strategic opportunities, and together with my managers, prioritized the opportunities.
   

2. Context & Background

• Product/Company Overview: U.S. Bank is the 5th largest bank in the United States. Bill Pay is one of its foundational features, allowing customers to take care of their bills on the go or from home.

• Target Users: All users see the same bill pay interface, and this study targeted retail and wealth customers using it on a computer, since there was a new small business team spinning up who would change the Small Business Owner SBO experience. We focused on OLB, not mobile since the OLB CSAT scores were lower.
   

3. Research Plan

• Methodology: conduct qualitative interviews. Also collect and combine from collaborators: 5M+ sessions analyzed in QM, 891 retail customer panel survey participants, 13 competitors reviewed, and persistent feedback surveys. This mix of methods gave a more comprehensive picture than just qualitative or quantitative studies could.

• Participants: the survey work was limited to users who chose to fill out a survey. The QM data was from all visitors. The qualitative research was from a total of 21 customers matching different personas. We screened for those from the retail and wealth segments as that's who this team was responsible for and could impact with their changes. A range of genders & locations around the US.

• Tools & Techniques: We used an agency to recruit for interviews. QM to collect usage data. And QM for surveys. I used Mural to take notes, organize ideas, and collate the findings from different sources. 

• Timeline: About 2 months from starting the research to getting the insights. (2 weeks to plan and get approval from the many stakeholders. 2 weeks to conduct the interviews. 2 weeks to synthesize.) Then it took another 8 weeks for everyone between me and the VP I was presenting to to weigh in. 
   

4. Research Process

• Planning & Preparation: Stakeholder alignment: working even more closely with product than and with a few levels higher than I would on usability testing. Clear alignment on business goals and research goals. A new collaboration with the CX team where I coordinated weekly calls to align on the products needs and the data synthesis so that CX was on board and felt accurately represented. Tested my qualitative script with a coworker. 

• Execution: Data collection details (e.g., number of sessions, key observations). Had team-members take notes on the study in Mural.co.

• Analysis & Synthesis: Affinity mapping to analyze the interviews as well as organizing findings from the many other inputs.
   

5. Key Findings & Insights

• High level insights: One of the challenges that I and my research managers realized was that while leadership was skilled at getting Engineering to focus on improving core functionality, and they were good at having Design work on lofty features and delights, the central piece of competitive parity and complex long-term improvements to the experience was getting overlooked. 

• Findings Presentation: I dug up an existing strategic map and sync’d it to the suggested UX/CX strategy framework. This helped product leaders integrate the ideas and suggestions quickly as they were able to slot the new information into an existing mental model. It used the idea of pieces building on each other which helped convey the delights should come after achieving expected functionality. 

• User Needs & Pain Points: I identified pain points, needs, and current gaps in the product and presented it mapped all to the new product framework/UX strategy combination.
   

6. Recommendations & Impact

• Design Recommendations: Recommendations were grouped in three categories that aligned with the strategic map. Core functionality, Competitive features, Customer delights. They included immediately actionable items as well as long term suggestions that touched on Engineering (e.g. reduce downtime to create an easeful experience) Design (craft a solution replacing customers’ analog checklists to ensure peace of mind), Content (e.g. ease fears about the dispute process to reduce stress), and Strategy (e.g. allow for real-time money movement to increase efficiency, speed, and loyalty).

• Presentation: I created a report in PowerPoint that walked through the problem, methods, and findings, and included multiple appendices to the other relevant reports and studies so that stakeholders could get any extra detail they were seeking all in one place. The user input was included as quotes. I presented this to multiple groups before bringing it to the larger group to make sure we were all aligned.

• Impact on Product/Strategy: It was completed in mid 2023 and it shaped the 2025 priorities and has spun into a few more investigations by other groups, like the call center.
   

7. Reflection & Learnings

• Challenges & How You Overcame Them: A challenge was how to present the changes that included complicated, long-term impacts, like shifting to real-time money movement, a technically complex and long process. See Key Findings & Insights (above) on how I overcame this.

• What You’d Do Differently Next Time: Find a more effective way to allow all the directors to weigh in without slowing the readout down by 2 months. It was a tender topic for many involved and we wanted to tread carefully since it was one of the first times we were bringing these groups together. Next time I would start to form these partnerships earlier. I could speak to each director individually to understand what their fears were of how this research might impact their product and timelines, and also schedule some group working time or group review sessions, rather than doing them all individually.

• Personal Takeaways: I am so grateful for the opportunity to work on this project and with the people I got to partner with. I gained skills in building cross-team partnerships. I worked with many directors and gained their trust and support. I found new and effective ways to frame recommendations. I had an impact on future roadmaps. I would continue to partner across teams when possible and/or invest in mixed methods studies when smaller scale projects are not convincing enough to shift priorities.