Research Scientist Intern, Audio Quality with AI

The Meta Reality Labs Research Team brings together a world-class team of researchers, developers, and engineers to create the future of virtual and augmented reality, which together will become as universal and essential as smartphones and personal computers are today. And just as personal computers have done over the past 45 years, AR, VR and MR will ultimately change everything about how we work, play, and connect. We are developing all the technologies needed to enable breakthrough AR glasses and VR headsets, including optics and displays, computer vision, audio, graphics, brain-computer interfaces, haptic interaction, eye/hand/face/body tracking, perception science, and true telepresence. Some of those will advance much faster than others, but they all need to happen to enable AR, VR and MR that are so compelling that they become an integral part of our lives.In particular, the Meta Reality Labs Research audio team is focused on two goals; creating virtual sounds that are perceptually
indistinguishable from reality, and redefining human hearing. See more about our work here: Inside Facebook Reality Labs Research: The future of audio and Filter Out the Noise With Conversation Focus. These two initiatives will allow us to connect people by allowing them to feel together despite being physically apart, and allow them to converse in even the most difficult listening environments.Meta Reality Labs Research is looking for an intern who is passionate about speech perception and audio quality to investigate why processed speech sometimes sounds degraded or robotic. The project focuses on identifying systematic phonemic errors as causal factors in perceived quality degradation, and linking these errors to human quality and intelligibility judgments. A core method is to explore the capabilities of audio vs video LLMs. This is fundamentally a speech-perception research role; multimodal/LLM methods are a supporting tool rather than the central focus. Our internships are twelve
(12) to twenty four (24) weeks long and we have various start dates throughout the year.

Required Skills:

Research Scientist Intern, Audio Quality with AI Responsibilities:
1. Investigate systematic phonemic errors as causal factors in perceived speech quality degradation, and link them to human perceptual judgments.
2. Build and curate datasets and benchmarks of speech for phoneme-level analysis. Explore and compare the capabilities of audio and video (multimodal) LLMs as tools to support this analysis.
3. Relate findings to human perceptual data (quality preference and intelligibility) and translate them into actionable insights for research and engineering teams.
4. Where appropriate, adapt multimodal models to the task in a supporting capacity. Collaborate with researchers, engineers, and cross-functional partners to define goals, communicate findings, and drive improvements in speech quality.
5. Develop tools and infrastructure to streamline and scale the analysis.
6. Stay current with research in speech perception and audio quality and intelligibility assessment, and incorporate best practices into Meta's workflows.
7. Disseminate results through internal reports and presentations, and, when appropriate, external publications.

8. Currently has, or is in the process of obtaining, a PhD degree in the field of Speech and Hearing Science, Auditory Neuroscience, Computational Neuroscience, Computer Science, Artificial Intelligence, Generative AI, Transformer Models, Machine Learning, Signal Processing or Computer vision
9. Experience with Python, Matlab, or similar
10. Experience with machine learning software platforms such as PyTorch, TensorFlow, etc
11. Background in speech perception, psychoacoustics, or acoustic phonetics
12. Experience deploying novel audio computational models and LLMs
13. Must obtain work authorization in the country of employment at the time of hire, and maintain ongoing work authorization during employment, 14. Experience building novel audio computational models and LLMs
15. Demonstrated software engineer experience via an internship, work experience, coding competitions, or widely used contributions in open source repositories (e.g. Github)
16. Experience in advancing AI techniques, including core contributions to open source libraries and frameworks in computer vision or audio processing
17. Experience with audio and speech quality assessment
18. Experience with multichannel audio processing
19. Experience in visual and acoustic scene analysis
20. Experience manipulating and analyzing complex, large scale, high-dimensionality data from varying sources
21. Proven track record of achieving significant results as demonstrated by grants, fellowships, patents, as well as first-authored publications at leading workshops or top computer vision and machine learning conferences such as ARO, ASA, NeurIPS, ICML, ICLR, ACL, EMNLP, CVPR, ICCV, ECCV, ICASSP, InterSpeech or similar
22. Experience in utilizing theoretical and empirical research to solve problems
23. Experience working and communicating cross functionally in a team environment
24. Intent to return to a degree-program after the completion of the internship/co-op

Back to blog

Other Jobs To Apply

No other job posts for this day.

Common Interview Questions And Answers

1. HOW DO YOU PLAN YOUR DAY?

This is what this question poses: When do you focus and start working seriously? What are the hours you work optimally? Are you a night owl? A morning bird? Remote teams can be made up of people working on different shifts and around the world, so you won't necessarily be stuck in the 9-5 schedule if it's not for you...

2. HOW DO YOU USE THE DIFFERENT COMMUNICATION TOOLS IN DIFFERENT SITUATIONS?

When you're working on a remote team, there's no way to chat in the hallway between meetings or catch up on the latest project during an office carpool. Therefore, virtual communication will be absolutely essential to get your work done...

3. WHAT IS "WORKING REMOTE" REALLY FOR YOU?

Many people want to work remotely because of the flexibility it allows. You can work anywhere and at any time of the day...

4. WHAT DO YOU NEED IN YOUR PHYSICAL WORKSPACE TO SUCCEED IN YOUR WORK?

With this question, companies are looking to see what equipment they may need to provide you with and to verify how aware you are of what remote working could mean for you physically and logistically...

5. HOW DO YOU PROCESS INFORMATION?

Several years ago, I was working in a team to plan a big event. My supervisor made us all work as a team before the big day. One of our activities has been to find out how each of us processes information...

6. HOW DO YOU MANAGE THE CALENDAR AND THE PROGRAM? WHICH APPLICATIONS / SYSTEM DO YOU USE?

Or you may receive even more specific questions, such as: What's on your calendar? Do you plan blocks of time to do certain types of work? Do you have an open calendar that everyone can see?...

7. HOW DO YOU ORGANIZE FILES, LINKS, AND TABS ON YOUR COMPUTER?

Just like your schedule, how you track files and other information is very important. After all, everything is digital!...

8. HOW TO PRIORITIZE WORK?

The day I watched Marie Forleo's film separating the important from the urgent, my life changed. Not all remote jobs start fast, but most of them are...

9. HOW DO YOU PREPARE FOR A MEETING AND PREPARE A MEETING? WHAT DO YOU SEE HAPPENING DURING THE MEETING?

Just as communication is essential when working remotely, so is organization. Because you won't have those opportunities in the elevator or a casual conversation in the lunchroom, you should take advantage of the little time you have in a video or phone conference...

10. HOW DO YOU USE TECHNOLOGY ON A DAILY BASIS, IN YOUR WORK AND FOR YOUR PLEASURE?

This is a great question because it shows your comfort level with technology, which is very important for a remote worker because you will be working with technology over time...