Introduction
Whisper is OpenAI's advanced automatic speech recognition (ASR) system that:
- Transcribes spoken language with high accuracy across multiple languages
- Trained on 680,000 hours of multilingual and multitask supervised data
- Performs robustly across various audio conditions and background noise
- Handles technical language and heavily accented speech effectively
Whisper's versatility comes from its Transformer-based encoder-decoder architecture that converts diverse audio inputs into accurate text outputs, making it valuable for applications from content transcription and accessibility tools to language learning platforms.
Instructions
1. Choose Interaction Method
- API: Use OpenAI's Audio API endpoint
- Python SDK: Official OpenAI package
- Open Source: Direct model implementation available
import OpenAI from 'openai';
const openai = new OpenAI({
apiKey: process.env.OPENAI_API_KEY,
});
async function transcribeAudio(filePath) {
const transcription = await openai.audio.transcriptions.create({
file: fs.createReadStream(filePath),
model: "whisper-1",
language: "en"
});
return transcription.text;
}
2. Configure Audio Parameters
- Prepare audio in supported formats (MP3, MP4, WAV, etc.)
- Consider file size limits (25MB for API)
- Specify language for better accuracy (optional)
- Choose transcription or translation mode
3. Key Parameters
- model: "whisper-1" (current API model)
- language: Language code (e.g., "en", "es", "ja")
- response_format: "json", "text", "srt", "verbose_json", "vtt"
- temperature: Controls randomness (0.0 to 1.0)
4. Post-Processing & Best Practices
- Review transcriptions for accuracy
- Use prompt engineering for domain-specific terminology
- Implement feedback loops for continuous improvement
- Consider using timestamps for longer audio
Capabilities
Speech Recognition
- • High-accuracy transcription
- • Multilingual support (100+ languages)
- • Robust to background noise
Translation
- • Direct audio-to-English translation
- • Preserves meaning across languages
- • Handles colloquialisms
Special Features
- • Language identification
- • Timestamp generation
- • Formatted subtitle creation
Applications
- • Content accessibility
- • Meeting transcription
- • Multimedia processing
Examples
// Basic Transcription
const transcription = await openai.audio.transcriptions.create({
file: fs.createReadStream("audio.mp3"),
model: "whisper-1",
});
console.log(transcription.text);
// Translation with Specific Parameters
const translation = await openai.audio.translations.create({
file: fs.createReadStream("spanish-audio.mp3"),
model: "whisper-1",
response_format: "srt",
temperature: 0.2
});
console.log(translation);
Table of Contents
Key Features
- • Multilingual: 100+ languages supported
- • Open-Source: Available for direct use
- • Versatile Output: Multiple formats (JSON, SRT, VTT)
- • Robust: Handles challenging audio conditions
- • Multi-task: Single system for recognition, translation, and identification