Pitch Shift AI Voices

Pitch Shifted AI Voice < 1.0 that sounds like Hulk

What is Pitch Shifting?

Pitch shifting changes the pitch of a sound without changing the playback speed. It’s useful for:

Creating “cartoon-like” higher voices (pitch factor > 1.0)
Deepening voices for characters like “Hulk” (pitch factor < 1.0)

Why Pitch Shifting?

When we spoke to doctors and parents about using Elato in hospitals, they were more excited about child-like voices to reduce children’s anxiety. One customer put it simply:

OpenAI Voices are not well suited for small toys. The ultimate goal is to build a toy for my 10-year-old daughter that can answer simple questions and tell fairy tales on request.

His Furby-like setup was an incredible idea. Here’s Roman’s current setup with an ESP32 XIAO:

How to create cartoon-like realtime AI voices with pitch shifting on ESP32 Arduino

Phil Schatzmann’s arduino-audio-tools lays the groundwork:

Repo: https://github.com/pschatzmann/arduino-audio-tools
Pitch shifting wiki: https://github.com/pschatzmann/arduino-audio-tools/wiki/Pitch-Shifting

Audio pipeline before pitch shift

OpusDecoder → BufferPrint → audioBuffer → QueueStream → VolumeStream → I2SStream

VolumeStream volume(i2s); // access from audioStreamTask only
QueueStream<uint8_t> queue(audioBuffer); // access from audioStreamTask only
StreamCopy copier(volume, queue);
AudioInfo info(SAMPLE_RATE, CHANNELS, BITS_PER_SAMPLE);

Audio pipeline with pitch shift

OpusDecoder → BufferPrint → audioBuffer → QueueStream → PITCH_SHIFT_OUTPUT → VolumeStream → I2SStream

PitchShiftFixedOutput pitchShift(i2s);
VolumeStream volumePitch(pitchShift); // access from audioStreamTask only
StreamCopy pitchCopier(volumePitch, queue);

Apply pitch factor dynamically

In our websocketEvent callback, we can configure pitch shift when the pitch factor is not 1.0:

// Only initialize pitch shift if needed
if (currentPitchFactor != 1.0f) {
  auto pcfg = pitchShift.defaultConfig();
  pcfg.copyFrom(info);
  pcfg.pitch_shift = currentPitchFactor;
  pcfg.buffer_size = 512;
  pitchShift.begin(pcfg);
}

This pitch shift implementation uses a granular synthesis approach. Implementation helpers:

In conclusion

Pitch shifting is a simple but powerful way to make realtime AI voices feel more playful and character-like on ESP32. With Elato you can set this in your NextJS app directly.

Full repo: https://github.com/akdeb/ElatoAI

Getting started

AI Models

Voice Cloning

Tutorials

Pitch Shift AI Voices

Pitch Shifted AI Voice < 1.0 that sounds like Hulk

What is Pitch Shifting?

Why Pitch Shifting?

How to create cartoon-like realtime AI voices with pitch shifting on ESP32 Arduino

Audio pipeline before pitch shift

Audio pipeline with pitch shift

Apply pitch factor dynamically

In conclusion

​Pitch Shifted AI Voice < 1.0 that sounds like Hulk

​What is Pitch Shifting?

​Why Pitch Shifting?

​How to create cartoon-like realtime AI voices with pitch shifting on ESP32 Arduino

​Audio pipeline before pitch shift

​Audio pipeline with pitch shift

​Apply pitch factor dynamically

​In conclusion

Pitch Shifted AI Voice < 1.0 that sounds like Hulk

What is Pitch Shifting?

Why Pitch Shifting?

How to create cartoon-like realtime AI voices with pitch shifting on ESP32 Arduino

Audio pipeline before pitch shift

Audio pipeline with pitch shift

Apply pitch factor dynamically

In conclusion