Help
The app isn't available yet. This page explains how it will work after launch.
Getting started
- When you launch the app, a setup window opens. Follow the steps.
- Choose the other person's language, your language, and what you're using it for (Games, Calls & meetings, Videos & classes). You can change these on the Home page later.
- When the Live Captions window appears for the first time, choose "Yes, continue".
- In that window, use the gear button → "Change language" to pick the language the other person speaks. A new language needs a download the first time.
- Use the gear button → "Position" and choose "Overlaid on screen".
- When setup is done, the Live Captions window hides and only translations appear. The app stays in the system tray at the right end of the taskbar.
Keyboard shortcuts
These work whichever app you're using.
- Ctrl+Alt+T … Pause or resume translation
- Ctrl+Alt+L … Move the captions (press again to lock them)
- Ctrl+Alt+M … Turn on or off the microphone that listens to your own words
- Ctrl+Alt+S … Open the box to type what you want spoken (Enter to speak, Esc to close)
- Ctrl+Alt+X … Stop the voice and any sentences waiting to be sent
- Ctrl+Alt+E … Correct what was heard and send it
- Ctrl+Alt+Y … Send the waiting sentence now
- Ctrl+Alt+R … Ask the other person to say it again
Making captions easier to read
On the app's "Captions" page, adjust things while watching a preview.
- Style … Choose "With background" or "Outline only", which has no background.
- Text size, lines shown at once, whether to show the original … Fewer lines cover less of the screen and are easier to follow.
- Position … Switch with "Move to top" and "Move to bottom". To put captions anywhere, press Ctrl+Alt+L, drag them, then press Ctrl+Alt+L again (they also lock by themselves after a short while).
Using it in games
- Set the game's display mode to "Borderless" or "Windowed". Captions can't appear over "Exclusive Fullscreen", a display mode that ordinary windows can't sit on top of.
- Clicks normally pass through the captions, so they won't interfere with your controls.
- Game audio gets captioned too. Turn the game volume down when translating voice chat for better accuracy.
- To speak, the Ctrl+Alt+M microphone lets you keep your hands on the game. Opening the typing box takes the keyboard away from the game.
Refining translations
These are under "More settings" on the app's "Translation" page.
- Glossary … Locks in translations of names, company names, game terms and so on. Write one per line as "original=translation" (for example, spike=スパイク).
- Tone of my words … Choose "Polite" or "Casual" for how your translated words sound.
Looking back at captions you missed
The app's "History" page collects what has been translated since the app started. It's never saved to a file and disappears when you close the app.
Signing up for a plan
- On the app's "Translation" page, use "Choose a plan (opens your browser)".
- Paste the license key you receive by email and press "Activate on this PC".
- To move to another PC, press "Deactivate this PC" on the old one. One key works on up to 2 PCs.
Having your own words spoken to the other person
To send a translated voice into a call, you need a microphone inside your PC (a virtual microphone). The app uses VB-CABLE by VB-Audio. VB-CABLE is a separate product from another developer, distributed as donationware. Business use requires buying a license.
- On the app's "Speak" page, follow the steps to install VB-CABLE (choose "Yes" on the Windows prompt). Once installed, the app finds and configures it automatically. You may need to restart your PC.
- Press "Test audio" to check that sound gets through inside your PC.
- Switch the microphone in your call app to "CABLE Output" (once per app). When you do, that app's name appears on the "Speak" page.
- The other person now hears only the translated voice. To send your own voice as well, turn on "Also send my own voice". You can then talk without switching the microphone back, and when you turn on the microphone with Ctrl+Alt+M, the translated voice follows your own. For the translated voice only, turn off "Keep sending while the interpreter mic is on". The microphone is opened only while a call app is listening. You can also choose which microphone to use.
- Type in the box and press Enter to have it translated and spoken. To speak by voice, turn on the microphone with Ctrl+Alt+M.
Speech recognition runs entirely on your PC, so your voice isn't sent anywhere. It keeps listening while you use other apps or games. The first time, download a speech model on the "Speak" page (Standard is about 190 MB; High accuracy, which is stronger on game terms and kanji, is about 574 MB; Best accuracy, slightly more accurate still, is about 1,081 MB. High and Best accuracy use your GPU if you have one). Your words turn into text shortly after you finish speaking. Mistakes in what was recognized, such as wrong kanji, are corrected from the flow of the conversation when translating. For games, choose "Games" on the Home page: game terms become easier to recognize, and Claude's translations sound more like voice chat. Speakers can pick up the other person's voice, so please use headphones. When there are several people on the other side, turn on "Separate speakers (experimental)" under "More settings" on the "Translation" page: the app listens only to the call app's audio and translates each speaker separately, labeled A, B and so on. If Windows doesn't have a voice for the other person's language, add it in Settings → Time & language → Language & region.
When it mishears you
Speech recognition can get things wrong. Under "More settings" on the app's "Speak" page, you can keep mistakes from reaching the other person.
- Wait before sending … Under "Before sending what it heard", choose to wait 2 or 4 seconds, or to wait until you press Ctrl+Alt+Y. While it waits, you can check the sentence in the captions. The longer it waits, the later the other person hears it. Typed sentences don't wait.
- Stop … Ctrl+Alt+X cancels waiting sentences and stops the voice partway through.
- Correct … Ctrl+Alt+E opens a box with the latest sentence. Press Enter to send your correction, or Esc to send nothing. If the original already went out, it first tells the other person you're saying it again.
- Add words it often mishears … Under "Words it often mishears", write wrong=right and the word is fixed before translating. You can also add one with "Fix automatically from now on" when you correct a sentence.
- When you can't make out the other person … Ctrl+Alt+R asks them to say it again. It skips translation, so it goes out right away and doesn't use your allowance.
When something isn't working
- No captions or translations appear
- Check that the Home page says "Running". Also check that you can hear the other person through your PC's speakers or headphones (Live Captions listens to the default output) and that translation isn't paused.
- Captions don't appear over a game
- Set the game's display mode to "Borderless" or "Windowed".
- Translations are wrong or captions make no sense
- Check that the Live Captions language matches the language being spoken. "Change caption language" on the Home page brings its window back so you can change it from the gear button.
- The translation changes after a sentence ends
- While someone is still speaking, the translation is provisional. To keep costs down, unfinished sentences are always translated by Haiku, whichever model you chose. When the sentence ends, it's replaced by your chosen model's translation.
- I want to translate videos or classes
- All sound you can hear on your PC gets captioned, so it just works. Choose "Videos & classes" on the Home page.
- I only want calls translated, not videos
- Choose "Calls & meetings" or "Games" on the Home page, or pause with Ctrl+Alt+T when you're not on a call.
- It says you've used up this month's translation allowance
- Your allowance resets on the 1st of each month. Switching the model to Haiku makes the same allowance last longer.
- The other person can't hear my voice
- Press "Test audio" on the "Speak" page. If sound gets through but still doesn't reach them, check that the microphone in your call app is set to CABLE Output. When it's set correctly, the app's name appears on that page.
- I can't hear my PC after installing VB-CABLE
- Your default speaker has changed to CABLE Input. Choose your original speaker again in Windows Settings → System → Sound.
- It says it can't open the microphone
- USB audio devices sometimes can't record when the playback and recording sample rates differ. Use "Open microphone settings", which appears on the "Speak" page, and set the microphone's "Format" to the same value as playback (usually 48000 Hz). It also may not open while music software (a DAW) is using the device. Once it can open, "Also send my own voice" starts sending by itself.
- The translated voice cuts out or is hard to hear
- Noise suppression in your call app may treat the synthesized voice as noise. In Discord, try turning off noise suppression in Settings → Voice & Video.
- Nothing gets translated when I speak into the microphone
- On the "Speak" page, check that "Microphone" is the one you normally use and that "Language to listen for" is your own language. While the tray icon shows a red mark, the app is listening to the microphone.
- I want to remove VB-CABLE
- You can remove it from "More settings" on the "Speak" page, or from Windows Settings → Apps.
Uninstalling the app
Remove it from Windows Settings → Apps. To also erase your settings and keys, first press "Erase and quit" on the "About" page.
Contact
Contact details will be published before sales begin.