Find Elato on the OpenAI Cookbook as a resource for Realtime Voice AI Hardware.
OpenAI launched GPT-4o Realtime in 2024, which brought multimodal AI capabilities to the forefront. This model can understand and generate both text and audio in real-time, making it ideal for voice-based applications.
We are currently using the
gpt-realtime series models for Elato’s devices. More documentation here: https://platform.openai.com/docs/models/gpt-realtime.
To use OpenAI’s models on your ESP32, simply set your OPENAI_API_KEY in the .env file in your Deno server (and NextJS web-app if you want to interact with a WebRTC client on screen).