All the Feels: NVIDIA Shares Expressive Speech Synthesis Research at Interspeech


AI has transformed synthesized speech from the monotone of robocalls and a long time-old GPS navigation methods to the polished tone of virtual assistants in smartphones and intelligent speakers.

But there is even now a gap amongst AI-synthesized speech and the human speech we listen to in each day dialogue and in the media. That is due to the fact people communicate with elaborate rhythm, intonation and timbre which is challenging for AI to emulate.

The hole is closing quickly: NVIDIA scientists are constructing designs and applications for higher-quality, controllable speech synthesis that capture the richness of human speech, devoid of audio artifacts. Their most recent tasks are now on screen in periods at the Interspeech 2021 convention, which runs by Sept. 3.

These designs can enable voice automated shopper services strains for banks and vendors, bring online video-video game or e book figures to daily life, and present actual-time speech synthesis for electronic avatars.

NVIDIA’s in-household inventive group even uses the engineering to generate expressive narration for a movie sequence on the electricity of AI.

Expressive speech synthesis is just 1 aspect of NVIDIA Research’s perform in conversational AI — a area that also encompasses purely natural language processing, automated speech recognition, search phrase detection, audio improvement and a lot more.

Optimized to operate competently on NVIDIA GPUs, some of this chopping-edge perform has been manufactured open resource by the NVIDIA NeMo toolkit, out there on our NGC hub of containers and other application.

Driving the Scenes of I AM AI

NVIDIA researchers and resourceful gurus don’t just talk the conversational AI talk. They stroll the stroll, placing groundbreaking speech synthesis versions to work in our I AM AI video clip series, which attributes world AI innovators reshaping just about every industry conceivable.

But right up until lately, these videos have been narrated by a human. Former speech synthesis models supplied restricted regulate over a synthesized voice’s pacing and pitch, so tries at AI narration didn’t evoke the emotional response in viewers that a proficient human speaker could.

That altered about the past yr when NVIDIA’s textual content-to-speech exploration group formulated far more strong, controllable speech synthesis designs like RAD-TTS, made use of in our profitable demo at the SIGGRAPH Serious-Time Are living competition. By coaching the textual content-to-speech model with audio of an individual’s speech, RAD-TTS can transform any textual content prompt into the speaker’s voice.

An additional of its features is voice conversion, in which one particular speaker’s words (or even singing) is delivered in yet another speaker’s voice. Influenced by the notion of the human voice as a musical instrument, the RAD-TTS interface gives customers high-quality-grained, body-stage control over the synthesized voice’s pitch, duration and power.

With this interface, our online video producer could record himself looking through the online video script, and then use the AI design to convert his speech into the woman narrator’s voice. Employing this baseline narration, the producer could then immediate the AI like a voice actor — tweaking the synthesized speech to emphasize certain words and phrases, and modifying the pacing of the narration to improved express the video’s tone.

The AI model’s capabilities go past voiceover perform: textual content-to-speech can be made use of in gaming, to aid men and women with vocal disabilities or to support buyers translate involving languages in their individual voice. It can even recreate the performances of iconic singers, matching not only the melody of a track, but also the emotional expression behind the vocals.

Providing Voice to AI Builders, Researchers

With NVIDIA NeMo — an open-supply Python toolkit for GPU-accelerated conversational AI — researchers, developers and creators attain a head start out in experimenting with, and fine-tuning, speech styles for their have applications.

Straightforward-to-use APIs and models pretrained in NeMo assist researchers create and personalize models for text-to-speech, all-natural language processing and real-time automated speech recognition. Numerous of the products are experienced with tens of thousands of hours of audio details on NVIDIA DGX techniques. Builders can fine tune any model for their use conditions, speeding up education making use of blended-precision computing on NVIDIA Tensor Core GPUs.

By means of NGC, NVIDIA NeMo also presents products skilled on Mozilla Common Voice, a dataset with virtually 14,000 several hours of group-sourced speech info in 76 languages. Supported by NVIDIA, the challenge aims to democratize voice technological know-how with the world’s premier open information voice dataset.

Voice Box: NVIDIA Researchers Unpack AI Speech

Interspeech delivers jointly extra than one,000 scientists to showcase groundbreaking perform in speech know-how. At this week’s conference, NVIDIA Study is presenting conversational AI design architectures as effectively as totally formatted speech datasets for developers.

Catch the adhering to sessions led by NVIDIA speakers:

Obtain NVIDIA NeMo versions in the NGC catalog, and tune into talks by NVIDIA scientists at Interspeech

Leave a comment

Your email address will not be published.