My speech-to-text setup and opinions

hardware

Radeon 7800 XT (16 GB VRAM), 32 GB system RAM. Most laptop users won’t reproduce this.

Here’s what I currently use for converting speech to text:

• for just typing instead of writing i use Handy
  • Cohere transcribe is my transcription model
  • gemma4:latest served from ollama is my post-processing model
  • I use the default prompt for post processing:

Clean this transcript:
1. Fix spelling, capitalization, and punctuation errors
2. Convert number words to digits (twenty-five → 25, ten percent → 10%, five dollars → $5)
3. Replace spoken punctuation with symbols (period → ., comma → ,, question mark → ?)
4. Remove filler words (um, uh, like as filler)
5. Keep the language in the original version (if it was french, keep it in french for example)

Preserve exact meaning and word order. Do not paraphrase or reorder content.

Return only the cleaned transcript.

Transcript:
${output}

Overall I’m very satisfied with how it works for both polish and english. I found that having a few short sessions (2-3 sentences at a time) works better than a long rambling one. Also feels less awkward for me than making the model sit through a long “uhhhhh” while I gather my thoughts, but I guess that’s my high agreeableness manifesting.

• for meeting transcriptions I use openwhispr
  • parakeet TDT 0.6B is my transcription model
  • Qwen3.5 9B is my post-processing model
  • I’m using the default prompt everywhere (i dont see what it is, but i didn’t set anything custom). Overall it’s very hit or miss. Speaker identification sucks which leads to wrong meeting notes.

Caveats: I have a reasonably powerful GPU (Radeon 7800 XT with 16gb of vram) and 32gb of regular ram so it’s likely most laptop folks won’t be able to replicate it on their hardware.

I have mostly gone with the “Recommended” settings and havent experimented a ton. I might try to use the same models in openwhispr as I do in handy and see if I get better results (idk if feasible).

I did also try the clickup speech2text for writing comments but I need to do more editing there because the tone reads super american & excited

Overall speech-to-text still feels somewhat unnatural to me. I see it’s value when I use it, but my mind instinctively reaches for typing as the input method.

Languages

I alternate between Polish and English depending on context. Unfortunately cohere transcribe doesn’t support language detection. Having it set to 1 language and dictating in another led to inconsistent results so I hacked together a solution that lets me:

code authorship

By I, I mean Claude code

• define handy profiles
• switch between them
• display current profile

I’m currently using 3 profiles: English, Polish and Auto (it sets a model with auto detection, current parakeet). Handy doesn’t expose any cli/api thing for setting models and their options, so I rely on modifying the config file and restarting (no config reload either). This costs me about 500ms per language change which is acceptable.

Full script lives in my dotfiles; here’s the part that matters - the profile definitions and the switch itself:

DEFAULT_PROFILES = [
    {"id": "pl", "label": "Polish", "short": "Cohere", "model": COHERE,
     "language": "pl", "lang_detect": False, "langs": COHERE_LANGS},
    {"id": "en", "label": "English", "short": "Cohere", "model": COHERE,
     "language": "en", "lang_detect": False, "langs": COHERE_LANGS},
    {"id": "mixed", "label": "Mixed", "short": "Parakeet v3", "model": PARAKEET,
     "language": "auto", "lang_detect": True, "langs": PARAKEET_LANGS},
]

def validate(profile, sys_):
    """Refuse to apply a profile that would silently misbehave."""
    problems = []
    if model_path(profile["model"]) is None:
        problems.append(f"{profile['short']} is not downloaded.")

    # Read from Handy's own model catalog, not from the profile config —
    # a config file must not be able to claim a model detects language
    # when it doesn't.
    lang_detect, langs = model_facts(profile, sys_)
    if profile["language"] == "auto" and not lang_detect:
        problems.append(
            f"{profile['short']} cannot detect language, so 'auto' would "
            f"silently fall back to English.")
    elif langs and profile["language"] not in langs:
        problems.append(f"{profile['short']} does not support '{profile['language']}'.")
    return problems


def apply(profile, sys_):
    """Switch to a profile. Handy keeps settings in memory and only flushes
    them on exit, and there's no config-reload — so the edit has to happen
    while Handy is NOT running, then Handy gets relaunched."""
    problems = validate(profile, sys_)
    if problems:
        return problems

    pid = sys_.pgrep_handy()
    if pid is not None:
        sys_.terminate(pid)          # ...kill it...

    data = load_settings()
    data["settings"]["selected_model"] = profile["model"]
    data["settings"]["selected_language"] = profile["language"]
    save_settings(data)              # ...edit settings_store.json on disk...

    sys_.launch()                    # ...and restart. ~500ms round trip.
    # (a few more checks after this — e.g. that a denoising pipewire pin
    # survived the restart — elided here)

Launching the apps

I have a small 12 button keyboard that served different functions over time. I decided to dedicate part of it to launching speech to text programs and switching between handy modes. I’m using kmonad for it.

Relevant slice of the kmonad config (other bindings on the pad, e.g. time-tracking, elided):

(defalias
  ;; ...other aliases (time tracking, clipboard, editor launchers) elided...
  util    (layer-toggle util-layer)

  ;; --- speech-to-text row (top) ---
  ;; Chords are emitted directly so each app's push-to-talk hold works:
  ;; kmonad holds the modifiers down for as long as the physical key is held.
  ;; Cancel is the exception -- Handy binds it to bare `escape`, which would
  ;; also reach the focused app, so it goes through the Handy CLI instead.
  stt-x   (cmd-button "/usr/bin/handy --cancel")
  ow-meet A-S-w
  ow-dict A-w
  ow-tran A-t
  hy-ai   C-S-spc
  hy-dict A-q

  ;; --- Handy language profiles (util-layer top row) ---
  ;; Absolute path required: kmonad's PATH does not include ~/.scripts.
  hp-pl   (cmd-button "/home/user/.scripts/handy-profile pl")
  hp-en   (cmd-button "/home/user/.scripts/handy-profile en")
  hp-auto (cmd-button "/home/user/.scripts/handy-profile mixed")
)

(deflayer default
  @stt-x    @ow-meet  @ow-dict  @ow-tran  @hy-ai    @hy-dict
  ;; ...bottom row: other functions elided...
)

(deflayer util-layer
  @hp-pl    @hp-en    @hp-auto  _         _         _
  ;; ...bottom row: other functions elided...
)

In most cases (the push-to-talk apps: openwhispr and Handy’s own dictation/AI actions) I need to resort to emitting preregistered key combinations. In others (switching Handy’s language profile, or cancelling a recording) I call the handy-profile CLI or the Handy binary directly instead - cancel specifically has to bypass the bare escape key Handy normally binds it to, since that would also reach whatever app is focused.