Computer Science Professor Looks to Improve Accessible Technologies

Amy Pavel joined the College of Natural Sciences faculty in spring 2022 as an assistant professor in the Department of Computer Science. She was a postdoctoral fellow at Carnegie Mellon University and research scientist at Apple. She obtained her Ph.D. and undergraduate degrees at the University of California at Berkeley.

What is your main area of research?

My lab works at the intersection of artificial intelligence, human-computer interaction and accessibility. I work to understand humans and their goals, and then see what we can do with technology to achieve those goals. In general, my goal is to create new interactive technologies, usually embedding artificial intelligence, that all people, including those with disabilities, can use effectively.

What are your main goals for your time at UT?

One main goal is to determine the types of new interactive techniques we will use in the future. I plan to build artificial intelligence-powered interactions that help answer questions about how people interact with computers and then evaluate them to gain feedback on what we're creating. I also want to research how we can make the digital and physical world more accessible using recent advances in technology. Finally, I'd like to explore what interactions will make it easier for people to create and adapt computational tools to their own needs, because there is a big gap between the people who are creating technology and those who are using it.

What are some examples of projects you've worked on focusing on accessible technologies?

Video is a major medium of communication these days. We use YouTube and TikTok, for example, which are not accessible at all to people with vision impairments. My research has started to look at how we can make videos accessible to people with vision impairments. On online streaming services, like Netflix, you can hear audio descriptions of what's happening in the film. They're an art form, where the silent sections of a film are filled with descriptions of what's visually happening in the film, but these descriptions are really challenging to create and are only done by professionals. One of my projects looked at how we can let novices create audio descriptions easily. Another complication is that the current approach assumes that there will be silence in the video where you can fit in extra descriptions. So, another project considered how we could let the user just pause the video to get more information on demand. In the future, we should also consider how to make videos accessible during production, rather than trying to make them accessible at the end of the production process.

How do you include people with disabilities in the research process?

One standard approach is interviewing people with disabilities about how they do certain tasks. While working on helping people find accessible videos on YouTube more easily, we interviewed blind participants who already use YouTube regularly and asked them how they found videos, and what they considered "good" and "bad" videos in terms of accessibility. We also try to integrate users while collecting data. So, for example, if we're creating an artificial intelligence algorithm to predict if a video is accessible, we might include blind users to label videos as accessible or not accessible, so we make sure our assumptions about what makes a video accessible are informed by the users of the technology. Then, we take the algorithm we created and show it to users of the technology and integrate the feedback we get to improve our tool. It's important to have researchers with disabilities in our lab, so their opinions are included from the very beginning.

What drew you to UT Austin?

One of the things that is really exciting about UT in particular is that it's an excellent public university, so we get a big variety of talented students from across the state. I am super excited for the opportunity to teach and mentor students of all levels at UT. The university also has a top computer science department with strengths in areas that overlap with my own — natural language processing, computer vision and robotics. So, while I'm the first person in my subfield at the department, there will be ready collaborators. Plus, UT has strong departments across the university, like film, media and the School of Information. These strengths are helpful for my research, which often collaborates and evaluates technology with people in other domains.