While the quality of visual images in video games and virtual reality continues to advance impressively with technology improvements, the sound and music quality of those games haven’t kept pace. A team of faculty and students at the University of North Carolina at Chapel Hill’s Department of Computer Science has been developing sound simulation technologies to overcome that barrier for almost a decade.
One doctoral student, Carl Schissler, has developed new practical solutions that can increase the sound quality using simulation methods and can even run on mobile platforms.
Schissler, an Asheville native working on his doctorate in computer science in UNC’s College of Arts & Sciences, has co-authored three patents and a number of papers based on his dissertation research to improve the quality of sound significantly in virtual environments. His work has drawn attention from Oculus VR, the virtual headset video game company, which has offered Schissler a position with the company.
“What we’ve been working on is simulating audio effects in virtual environments – realistic audio effects,” said Schissler, who also earned his bachelor’s degree in computer science at UNC-Chapel Hill. “I’ve always been interested in video games and video game technology, and one of the areas I felt that was really lacking was audio. I also had a big interest in audio because I was into recording music, and this was an intersection of those fields, so it was an interesting thing for me to work on.”
Schissler said his work developed through finding new ways to solve problems.
“My research is about is how audio waves moves through an environment and simulating that environment,” he said. “Depending on the type of environment you’re in, sound could be reverberant and have lots of echoes, or it can be less reverberant. It’s the difference of, say, being in an office environment, where it’s very still environment, or being in a church or cathedral, where there’s a lot of reverberation. It gives it a sense of space.”
While most of Schissler’s work is aimed for virtual settings, he also has studied how sound moves in real environments. He uses a 3-D scanner to capture the geometry and images of real-world room and uses that data to construct a virtual model of the same room for simulating sound effects.
“Once you have the 3-D model and the material properties, you can simulate how sound should propagate in that model,” he explained. “That’s useful for augmented reality – where you are adding virtual objects into the real world that you are viewing. You wear some glasses and some headphones, and you place some virtual objects in there, like holograms. With this technology, you can simulate how the virtual objects sound in the real world.”
Dinesh Manocha, the Phi Delta Theta/Matthew Mason Distinguished Professor of Computer Science at UNC-Chapel Hill, has worked with Schissler since Schissler’s undergraduate days on campus, and is currently supervising his doctoral dissertation.
“He’s done amazing work on sound simulation, Manocha said. “He’ll be finishing his doctorate in four years, when the average is usually five-and-a-half years, and developed a new set of methods for efficiently generating acoustic effects. Carl is amazing that he’s published so many papers in such a short time in the top conferences and journals. His work is well-regarded and is trend-setting in the field. I expect his work will have a strong impact in industry.”
“There’s very strong interest in Carl’s patents and software system, and the university may be licensing them,” he said. “Carl has really pushed the boundaries in terms of real-time sound simulation. It has applications in virtual reality, but it also has applications in computer-aided design of buildings and urban layouts.”
Manocha said that Schissler’s future job with Oculus is the pinnacle of achievement for someone working in sound and virtual reality.
“Oculus is one of the biggest names in virtual reality,” Manocha said. “They’ve opened a research lab in the Seattle area, which is currently regarded as on the best research labs in virtual reality. Furthermore, they have a strong group in research related to sound simulation. Carl interned there over the last few summers and will join them in a month or so. His work is phenomenal; I’ve had the pleasure of working with many excellent doctoral students in 25 years here and Carl is among the very top.”
By Phillip Ramati, UNC System communications
