Research Proposal · Edge Speech Synthesis Internal Technical Walkthrough · 15 Slides

Cache-Resident Adapter-MoE for Vocoder-Free Code-Switched Edge TTS

Helping small devices speak mixed-language sentences smoothly: plan ahead from the full sentence text, keep active language adapters ready in RAM, and blend switches gently without a heavy neural vocoder.

INPUT AT t = 0 Full Mixed Sentence Known [EN] Let's review [ZH] 下周计划 [EN] today ≤ 2 languages + 1 target speaker PILLAR 1 · LOOKAHEAD & CACHE Pin Active LoRA Experts Lock L1 + L2 in RAM · Async preload L3 Shared-bridge fallback → 0 I/O stall PILLAR 2 · ADAPTER-MoE + BLEND Shared Bridge + Routed LoRA Skip 1-word loanwords · 20–40 ms blend Steady speaker voice & smooth pitch (F₀) PILLAR 3 · VOCODER-FREE TTS Source-Filter Synthesis F₀ excitation source → spectral filter H(z) ~4.5M params · ~8–21 MB total active RAM
Memory Footprint Target
~8–21 MB RAM

~4.5M-parameter vocoder-free source-filter backbone + tiny ~0.3–0.6 MB language LoRA experts.

Switch Tail Latency Target
p99 Stall < 5 ms

Deterministic sentence lookahead pins needed adapters before frame 0 and preloads upcoming ones early.

Boundary Quality Target
≥ 30% Lower ΔF₀ Jump

Single-word loanword debouncing avoids rapid ping-pong; 20–40 ms cross-fade smooths real switches.

Press ← / → or Space to navigate · Press N for speaker notes · Press E to edit text Slide 01 / 15
01 · Problem Statement Why Code-Switched Speech Breaks on Small Devices

What Breaks When Languages Mix on Edge Devices?

Bilingual speakers switch languages mid-sentence—both for multi-word phrases and single borrowed words.

CASE A · MULTI-WORD PHRASE SWITCH (NEEDS SMOOTH ROUTED TRANSITION AT B₁, B₂) [EN] "Let's schedule our team's" ⚡ B₁ EN → ZH [ZH] "下周项目评审会议" ⚡ B₂ ZH → EN [EN] "right after lunch today." CASE B · SINGLE-WORD BORROWED LOANWORD (HARD SWITCHING CAUSES RAPID PING-PONG ARTIFACT) [ZH Matrix] "麻烦发我一下那个" [EN 1-Word Loanword] "link" [ZH Matrix] "我马上打开看一下。"
Failure Mode 1 · Acoustic Discontinuity

Voice & Pitch Jump at B₁ / B₂

Switching language modules abruptly resets pitch (F₀), speaking energy, and vocal tract timbre—making one speaker sound like two different people.

Failure Mode 2 · Runtime I/O Stalls

Mid-Sentence Adapter Wait Spikes

Fetching a missing language adapter at the exact moment token B₁ arrives stalls the synthesis thread and causes audible pauses or buffer underruns.

Failure Mode 3 · Memory Ceiling

Neural Vocoders Blow Edge RAM

Keeping a large multilingual model plus a heavy neural vocoder (e.g., HiFi-GAN) in RAM requires 100–300+ MB—far above tight embedded or mobile limits.

Problem Breakdown: Acoustic Smoothness × Latency × Memory Slide 02 / 15
02 · Literature Gap Table 1 · Comparison of Code-Switched Edge TTS Architectures

Why Current Approaches Force a Size vs. Smoothness Trade-Off

Existing systems either overload device memory or stall and jump when swapping language parts reactively.

Architecture Family Active RAM Footprint Switch-Boundary Latency (p99) Cross-Lingual Speaker Consistency 1-Word Loanword & Boundary Behavior
1. Monolithic Multilingual TTS + Neural Vocoder
Single big network + HiFi-GAN
High (120–300+ MB)
Too heavy for strict edge budgets
Low (< 5 ms)
All weights stay in memory
Moderate–High
Can still drift on language ID flip
Over-switches accent
No explicit loanword debounce
2. Reactive On-Demand Adapter Swapping
Loads adapter when token arrives
Low (~15–35 MB)
Only 1 active adapter kept
High Spikes (25–120 ms)
Blocks on cold flash/cache miss
Drops at Cold Swap
Hard swap resets prosody state
Ping-Pongs on 1 Word
Harsh step jump at boundaries
3. Ours: Lookahead Cache-Resident Adapter-MoE
Vocoder-Free Source-Filter + LoRA-MoE
Ultra-Low (~8–21 MB)
~4.5M core + tiny ~0.4 MB LoRAs
Near-Zero (< 5 ms p99)
Pinned L1/L2 + preload + fallback
High (Steady Voice)
Always-on Shared Bridge anchor
Smooth & Debounced
1-word filter + 20–40 ms blend
The Missing Link in Prior Work

Prior adapter-based TTS treats language changes as unexpected runtime interrupts, and still relies on a heavy neural vocoder for waveform generation.

Our Core Thesis

By pairing pre-synthesis sentence lookahead with a vocoder-free source-filter LoRA-MoE, we eliminate both the vocoder RAM bottleneck and mid-sentence swap stalls.

Table 1 · System Trade-Off Matrix Slide 03 / 15
02b · Framework Comparison Traditional / Existing Reactive Pipeline vs. Proposed Cache-Resident Adapter-MoE

Traditional / Existing Frameworks vs. Our Proposed Framework

Side-by-side runtime comparison of how each architecture handles a mixed-language sentence at the switch boundary.

Traditional / Existing Frameworks (Reactive or Monolithic) 1. Token-by-Token / No Lookahead Switch noticed late at boundary B₁ 2. On-Demand Swap OR Giant Model Blocks on flash load OR keeps all in RAM ⚠ High p99 Stall (25–120 ms) 3. Hard Boundary & 1-Word Flip Ping-pongs on 1 word · resets speaker state 4. Heavy Neural Vocoder (HiFi-GAN) Consumes 60–120+ MB RAM & high CPU Result: Audible Switch Pause + Harsh ΔF₀ Pitch Jump + Voice Drift Unfit for strict edge memory budgets (~120–300+ MB) or smooth real-time playback ❌ Reactive Loading · ❌ Heavy Vocoder RAM · ❌ Abrupt Boundary Step VS Proposed Framework: Lookahead Cache-Resident Adapter-MoE (Ours) 1. Sentence Lookahead at t = 0 Full text scanned before frame 0 ✓ 1-Word Loanword Debouncer 2. Sentence-Scoped Cache Pin L1/L2 · Async preload L3 · Bridge fallback ✓ p99 Boundary Stall < 5 ms 3. Shared Bridge + Routed LoRA ~4.5M core + ΔW_shared locks speaker voice 4. 20–40 ms Blend + Source-Filter Smooth F₀ excitation → filter H(z) (~8–21 MB) Result: Zero-Stall Playback + ≥ 30% Lower ΔF₀ Jump + Steady Speaker Voice 7× to 12× smaller active RAM (~8–21 MB) · Runs fast on a single mobile/edge CPU core ✓ Lookahead Pinning · ✓ Vocoder-Free Source-Filter · ✓ 20–40 ms Boundary Blend
Advantage 1 · Memory

~8–21 MB vs. 120+ MB

Vocoder-free source-filter + tiny ~0.4 MB LoRAs replace bulky neural vocoders.

Advantage 2 · Tail Latency

< 5 ms vs. 120 ms Stall

Sentence lookahead pins L1/L2 at t = 0 and overlaps L3 flash I/O early.

Advantage 3 · Smoothness

≥ 30% Lower ΔF0 Jump

1-word debouncer stops ping-pong; 20–40 ms blend smooths pitch & formants.

Advantage 4 · Voice Identity

Steady Single Speaker

Always-on Shared Bridge (ΔWshared) prevents speaker drift across switches.

Visual Framework Comparison · Traditional Reactive vs. Proposed Lookahead Adapter-MoE Slide 04 / 15
03 · Key Insight Deterministic Sentence Lookahead Before Frame 0

Why Wait to Be Surprised? The Full Sentence Is Known at t = 0

Unlike live speech recognition, sentence-level TTS sees every word and language change before making a single sound.

CONVENTIONAL REACTIVE STREAM (BLIND TO UPCOMING SWITCH) 1. Speak EN frames (0–900 ms) → 2. Hit ZH token at t = 900 ms! Cache Miss → Stall audio to load ZH → 3. Audible pause / buffer underrun + harsh voice jump OUR DETERMINISTIC SENTENCE LOOKAHEAD (FULL TEXT SCANNED BEFORE FRAME 0) At t = 0 ms: Scan Full Sentence Detect active languages {EN, ZH} + 1-word loanwords → Pin {EN, ZH} + Speaker in RAM Pre-compute frame gate schedule g_k(t) & blend windows → Synthesize Frame 0 → N Smoothly 100% cache hit at switch · 20–40 ms cross-fade
Observation 1 · Known Horizon

Full Sentence Text at t = 0

Text frontend runs in < 2 ms for a full sentence, revealing exact language spans and switch locations before synthesis starts.

Observation 2 · Bounded Working Set

≤ 2 Languages + 1 Speaker

>98% of code-switched sentences mix at most two languages for one speaker—requiring under 1.2 MB of pinned LoRA weights.

Observation 3 · Lookahead Preload

Early Load for Rare 3rd Language

If a third language appears later in a long sentence, the cache controller preloads it asynchronously while the first clause is synthesized.

Core Insight: Turn Unexpected Runtime Switches into a Pre-Planned Schedule Slide 05 / 15
04 · System Architecture Overview Figure 1 · Complete Cache-Resident Adapter-MoE + Source-Filter TTS Pipeline
Figure 1 · System Architecture: Cache-Resident Adapter-MoE + Vocoder-Free Source-Filter TTS ~4.5M Params · ~8–21 MB Active RAM · Vocoder-Free 1 · LOOKAHEAD FRONTEND Text & Router Prep Full Sentence + Speaker EN ZH 1 Speaker ID Sentence LID & G2P Phonemes + language tags 1-Word Debouncer • 1 borrowed word → keep main language expert • Phrase → route switch g(t) Phonemes Gate g(t) 2 · SHARED-PLUS-ROUTED ADAPTER-MoE BACKBONE (~4.5M PARAMS) Parallel Core + Shared Bridge + Routed LoRA Experts x Frozen Core Backbone (W₀) Shared encoder + duration/pitch predictor ~4.5M Shared Bridge Adapter (ΔW_shared) Always on · locks speaker identity & miss fallback MoE Router Gates g₁(t), g₂(t) Top-1 / Top-2 ROUTED LoRA LANGUAGE EXPERTS (B_k · A_k) LoRA Expert L1 (EN) Pinned · g₁(t) LoRA Expert L2 (ZH) Pinned · g₂(t) LoRA Expert L3 (MS / Others) Async Preload + Sum h(t) F₀, Env, Voicing 3 · BOUNDARY BLENDER & VOCODER-FREE SOURCE-FILTER TTS Smooth Switch + Direct Excitation-Filter Synthesis 20–40 ms Boundary Cross-Fade Blender Smooths pitch (F₀), energy & expert transition across language borders VOCODER-FREE SOURCE-FILTER SYNTHESIZER (NO HEAVY NEURAL VOCODER) Excitation Source Pitch (F₀) Harmonic Pulses + Aperiodic Noise e(t) Spectral Filter H(z) Shapes vocal tract formants from spectral envelope Speech Waveform Output Fast CPU · Low RAM 4 · SENTENCE-SCOPED ADAPTER CACHE & ASYNC PRELOADER Keeps Active LoRA Experts Resident in Fast RAM Pin Active L1 & L2 Locked for full sentence Async Preload L3 Background flash fetch On Miss → Bridge Zero-stall fallback 1. Sentence language schedule 2. Pinned LoRA weights 3. If miss → Shared Bridge
Figure 1 · Saved standalone as system-architecture-overview.svg and embedded above Slide 06 / 15
05 · Contribution 1A: Vocoder-Free Source-Filter TTS Figure 2 · Memory Breakdown & Excitation-Filter Signal Chain

Eliminating the Heavy Neural Vocoder with Source-Filter Synthesis

Instead of a massive neural vocoder, our ~4.5M backbone predicts interpretable pitch (F₀) and filter envelopes directly.

FIGURE 2A · ACTIVE RAM FOOTPRINT COMPARISON (MB) Conventional Multilingual Acoustic + Neural Vocoder (~145 MB) Multilingual Acoustic Model (~85 MB) HiFi-GAN Neural Vocoder (~60 MB) Ours: Vocoder-Free Source-Filter Core + Pinned LoRA-MoE (~8–21 MB Total) Core W₀ (~9 MB FP16) Bridge L1+L2 LoRA ← 7× to 12× smaller in RAM! FIGURE 2B · VOCODER-FREE SOURCE-FILTER SYNTHESIS CHAIN Pitch F₀(t) + Voicing Drives glottal pulse train Aperiodicity α(t) Shapes unvoiced noise Excitation e(t) Harmonics + shaped noise Time-Varying Filter H(z, t) Controlled by Spectral Envelope S(f, t) → Direct 24 kHz Speech Waveform
Module Params FP16 / INT8
Core Acoustic Trunk (W₀)
Encoder + Duration/Pitch + Decoder
~4.1M ~8.2 / 4.1 MB
Source-Filter Head
Harmonic-noise excitation + filter
~0.4M ~0.8 / 0.4 MB
Always-On Shared Bridge
Cross-lingual speaker anchor
~0.25M ~0.5 MB
Pinned Language LoRA (×2)
Rank r = 8–16 per active language
~0.2M each ~0.4 MB each
Why Source-Filter Helps Code-Switching

Because pitch (F₀) and spectral envelope are explicit physical signals, we can directly smooth and interpolate them across language switch boundaries!

Contribution 1A · ~4.5M-Parameter Vocoder-Free Source-Filter Backbone Slide 07 / 15
06 · Contribution 1B: Shared-Plus-Routed LoRA-MoE Figure 3 · Inside an Adapted Backbone Layer

How One Small Voice Core Speaks Multiple Languages as the Same Person

Three parallel weight paths keep speaker timbre fixed while injecting language-specific pronunciation and tone.

FIGURE 3 · SHARED-PLUS-ROUTED LoRA-MoE LAYER WIRING Input x(t) Frame hidden state Path 1 · Frozen Core Linear Weight W₀ Shared across all languages · never swapped Path 2 · Always-On Shared Bridge (ΔW_shared = B_s A_s) Active on 100% of frames · anchors speaker timbre & miss fallback Path 3 · Routed Language LoRA Experts (rank r = 8–16) Expert L1: B₁ · A₁ Scaled by gate g₁(t) Expert L2: B₂ · A₂ Scaled by gate g₂(t) + → h(t)
Why the Shared Bridge Matters

Prevents "Two-Speaker" Drift

Without a shared bridge, switching from English LoRA to Mandarin LoRA shifts vocal tract formants. ΔWshared carries cross-lingual prosody and speaker identity across the boundary.

Routing Modes

Top-1 Inside Spans · Top-2 at Borders

Inside a pure language span, only 1 language expert runs (g1 = 1, g2 = 0). During a 20–40 ms switch window, both pinned experts blend smoothly (g1(t) + g2(t) = 1).

Adapted Layer Formulation:   h(t) = W0 x(t)  +  ΔWshared x(t)  +  ∑k ∈ {L₁, L₂} gk(t) · (α/r) Bk Ak x(t) LoRA Rank r ∈ [8, 16] · < 0.5 MB per language
Contribution 1B · Shared-Plus-Routed Low-Rank Mixture of Experts Slide 08 / 15
07 · Contribution 2: Sentence-Scoped Cache & Preloader Figure 4 · Multi-Track Lookahead Cache & Preload Timeline

Zero-Stall Memory Management: Pin What’s Needed, Preload What’s Next

Sentence-scoped pinning locks active languages in RAM, overlaps flash I/O with early audio frames, and never blocks synthesis.

FIGURE 4 · SENTENCE-SCOPED CACHE PINNING, ASYNC PRELOAD & FALLBACK TIMELINE t = 0 ms (Text Scan) t = 950 ms (Switch B₁: EN → ZH) t = 1850 ms (Switch B₂: ZH → MS) 1. Audio Stream Speaking L1: English (EN) Speaking L2: Mandarin (ZH) · 0 ms stall Speaking L3: Malay (MS) 2. Pinned RAM Slots 🔒 Slot 1 (EN LoRA) + Slot 2 (ZH LoRA) + Speaker Vector Pinned in RAM for Full Sentence (No Eviction) 3. Async Preloader Background Flash I/O: Preloading L3 (MS)... ✓ L3 (MS) Ready in Slot 3 (1160 ms before Switch B₂ arrives!) 4. Miss Safety Net If flash storage is busy and L3 misses t = 1850 ms → Route frames via Always-On Shared Bridge (ΔW_shared) until ready
Policy 1 · Sentence-Scoped Pinning

Lock Active L1 + L2 in RAM

Unlike generic LRU caches, active adapters for the current sentence are pinned at t = 0 and cannot be evicted mid-utterance.

Policy 2 · Async Lookahead Preload

Hide Flash I/O Behind Speech

Loading a ~400 KB LoRA from flash takes ~2–8 ms; starting the fetch at t = 0 finishes hundreds of milliseconds before the switch boundary.

Policy 3 · Graceful Miss Fallback

Shared Bridge Prevents Dropouts

Even under extreme cold-start stress, synthesis never blocks on I/O—it falls back to ΔWshared to keep the speaker’s voice flowing.

Contribution 2 · Lookahead Cache Pinning, Async Preload & Zero-Stall Fallback Slide 09 / 15
08 · Contribution 3: Loanword Debouncing & Boundary Blending Figure 5 · 1-Word Debounce Filter & 20–40 ms Source-Filter Cross-Fade

Making Switches Sound Natural: Skip 1-Word Bounces, Blend Real Borders

Prevents rapid expert ping-pong on borrowed words and smooths pitch (F₀) and spectral transitions at real phrase switches.

FIGURE 5A · SINGLE-WORD LOANWORD DEBOUNCING Without Debounce: Rapid 1-Word Expert Ping-Pong (Causes Pitch Bounce) ZH Expert EN Expert "link" (1 word) With 1-Word Debouncer (Span < 2 Words or < 220 ms → Keep Matrix Expert) ZH Expert EN Expert EN phoneme G2P + Steady ZH acoustic expert
FIGURE 5B · 20–40 ms BOUNDARY CROSS-FADE (MULTI-WORD SWITCH) 20–40 ms Window (2–4 frames around B₁) 1. Cosine Expert Gates: g_L1(t) ↓ and g_L2(t) ↑ g_EN(t) = 1.0 g_ZH(t) = 1.0 2. Source-Filter Pitch (F₀) & Spectral Envelope Trajectory Smooth Blended F₀(t) Hard Switch Step (ΔF₀ jump)
Cosine Blend Gate (t ∈ [tB − W/2, tB + W/2]):   gL1(t) = ½ [1 + cos(π · (t − t0)/W)],    gL2(t) = 1 − gL1(t),    W ∈ [20, 40] ms Eliminates phase/pitch clicks at boundaries
Contribution 3 · 1-Word Loanword Debouncing + 20–40 ms Source-Filter Boundary Blending Slide 10 / 15
09 · Runtime Execution Flowchart Figure 6 · End-to-End Deterministic Decision Flow per Utterance

How a Mixed-Language Sentence Flows Through the Runtime

Every branch is bounded in memory and latency—with zero blocking storage calls on the audio thread.

STEP 1 · AT t = 0 Sentence Lookahead LID + G2P + Pin L1/L2 in RAM + Start async preload if L3 STEP 2 · DEBOUNCE CHECK Foreign Span == 1 Word? (< 2 words or < 220 ms) YES (1-Word Loanword) Keep Matrix Language Expert Active (No Switch) Pronounce via bilingual phoneme G2P · avoids rapid adapter ping-pong NO (Phrase) STEP 3 · CACHE CHECK Target LoRA in RAM? (Pinned L1/L2 or preloaded L3) HIT (>99%) Routed LoRA + Bridge W₀ + ΔW_shared + g_k(t)ΔW_k + 20–40 ms Cosine Blend RARE MISS (Cold L3) Shared-Bridge Fallback Run W₀ + ΔW_shared (0 stall) Blend in L3 once async load completes STEP 4 · WAVEFORM Source-Filter TTS Excitation e(t) → Filter H(z) Direct 24 kHz Speech
Figure 6 · Deterministic Runtime Execution & Fallback Flowchart Slide 11 / 15
10 · Research Questions & Hypotheses Table 2 · Alignment of RQ1–RQ3, Testable Hypotheses H1–H3 & Ablations

Three Research Questions & Falsifiable Hypotheses

Each architectural pillar maps directly to a measurable acoustic or systems hypothesis.

ID Research Question Testable Hypothesis (Falsifiable Target) Key Ablation Experiment
RQ1 / H1
Backbone & Voice
Can a ~4.5M vocoder-free source-filter backbone with low-rank language adapters preserve one speaker’s voice across languages? H1: Shared-Bridge + Routed LoRA keeps cross-lingual speaker cosine similarity within ≤ 0.03 of a monolithic multilingual model while using < 20% active RAM (~8–21 MB). With vs. without Always-On Shared Bridge (ΔWshared); LoRA rank r ∈ {4, 8, 16} sweep.
RQ2 / H2
Lookahead Cache
Does sentence-lookahead pinning + async preload eliminate mid-sentence adapter loading stalls on edge hardware? H2: Deterministic sentence pinning + async preload keeps p99 switch-boundary stall latency < 5 ms and yields 0 audio buffer underruns under cold-cache stress. Reactive LRU on-demand loading vs. Sentence-Pinned + Async Preload + Bridge Fallback.
RQ3 / H3
Switch Control
Do single-word loanword debouncing and 20–40 ms boundary blending reduce audible switch jumps? H3: Reduces switch-boundary pitch step error (ΔF0) by ≥ 30% and improves code-switched listener preference (CMOS ≥ +0.25) over hard token-level switching. Hard frame switch vs. 1-word Debounce only vs. Debounce + 20–40 ms Cosine Blend.
Why H1 Is Novel

Proves that an explicit source-filter backbone + shared low-rank bridge decouples speaker vocal identity from language routing without a heavy vocoder.

Why H2 Is Novel

Exploits the deterministic lookahead property of TTS sentences to turn unpredictable MoE expert paging into a pre-scheduled cache pipeline.

Why H3 Is Novel

Addresses both linguistic granularity (1-word loanwords vs. phrase switches) and physical signal continuity (F0 & envelope blending).

Table 2 · Research Questions RQ1–RQ3 & Hypotheses H1–H3 Slide 12 / 15
11 · Evaluation Protocol Table 3 · Acoustic, Perceptual & Edge Systems Evaluation Matrix

How We Measure Success: Ear Quality + Boundary Smoothness + Edge Speed

We evaluate both global utterance quality and local +/-100 ms switch-boundary windows on edge CPU targets.

Evaluation Bucket Metrics What It Measures & How Target Threshold
1. Intelligibility & Naturalness Mixed CER / WER (ASR),
5-scale MOS & side-by-side CMOS
Pronunciation accuracy across both languages and listener preference on phrase & loanword sentences. CMOS ≥ +0.25 vs. hard switch; no CER degradation
2. Switch-Boundary Smoothness (±100 ms) Boundary ΔF0 step (Hz),
Energy step (dB),
ECAPA Speaker Cosine Sim
Local discontinuity across switch borders B1, B2 and voice consistency between L1 and L2 segments. ≥ 30% lower ΔF₀ jump;
Speaker sim drop ≤ 0.03
3. Edge Runtime & Tail Latency CPU RTF, TTFA (ms),
p99 Boundary Stall (ms),
Adapter Cache Hit Rate (%)
Single-core ARM/x86 synthesis speed and worst-case frame wait at language switches under cold/warm cache. RTF < 0.25 on mobile CPU;
p99 stall < 5 ms; Hit > 99%
4. Hardware Resource Footprint Peak Resident RAM (MB),
Per-Language Adapter Size (MB)
Total working memory (backbone + bridge + 2 pinned slots) and storage cost to add a new language. ~8–21 MB Peak RAM;
< 0.6 MB per language LoRA
Test Corpus Composition

Evaluated on three stratified code-switched sets: (A) Multi-word intra-sentential phrase switches, (B) Single-word borrowed loanwords, and (C) Rapid 3-language stress utterances.

Why Local Boundary Metrics Matter

Utterance-wide averages hide brief clicks or pitch jumps at switch points. Measuring ΔF0 and energy inside a ±100 ms boundary window directly captures switch artifacts.

Table 3 · 4-Bucket Evaluation Protocol & Target Thresholds Slide 13 / 15
12 · Execution Plan & Scope Framing Figure 7 · 4-Phase Gated Roadmap & Table 4 · Workshop vs. Thesis Scope

4-Phase Gated Research Roadmap & Publication Framing

Structured with clear Go/No-Go verification gates so a focused workshop paper expands cleanly into a full thesis.

PHASE 1 · FOUNDATION Frontend & ~4.5M Backbone Lookahead LID/G2P + Source-Filter core Gate 1: RTF < 0.25 & clean baseline audio PHASE 2 · ACOUSTIC MoE LoRA-MoE & Boundary Blend Shared bridge + 1-word debounce + blend Gate 2: ΔF₀ jump ↓ ≥30% & steady voice PHASE 3 · SYSTEMS RUNTIME Sentence Cache & Preloader Pin L1/L2 + async preload + miss fallback Gate 3: p99 stall < 5 ms & 0 underruns PHASE 4 · VALIDATION Listening Tests & Release MOS/CMOS + ablations + edge profiling Gate 4: Paper submission + demo deck
Scope Dimension Short Conference / Workshop Paper (ICASSP / Interspeech) Full Thesis Proposal Scope
Language Pair & Data 1 primary bilingual code-switched pair (e.g., Mandarin–English) + single-word loanword test set. 3+ languages (e.g., ZH–EN–MS), multi-speaker adaptation, and rapid 3-language intra-sentential switches.
Core Technical Focus Shared+Routed LoRA-MoE on ~4.5M source-filter core + 1-word debounce + 20–40 ms blend + lookahead cache trace. Full on-device C++/ONNX ARM runtime, FP16/INT8 quantization, hardware flash-to-RAM DMA profiling.
Key Ablations Shared Bridge on/off; Hard vs. Blended switch; Reactive vs. Lookahead cache hit/stall rate. Full LoRA rank & blend-window sweeps, perceptual threshold of bridge fallback under synthetic flash throttling.
Figure 7 & Table 4 · Phased Execution Plan & Publication Framing Slide 14 / 15
13 · Summary & Next Steps Cache-Resident Adapter-MoE for Vocoder-Free Code-Switched Edge TTS

Plan Early. Keep Ready in RAM. Switch Gently.

A practical, low-memory architecture for natural code-switched speech on everyday edge devices.

1 · Modular Vocoder-Free Voice

~4.5M Core + Shared-Bridge LoRA-MoE

Direct source-filter synthesis (F0 excitation → spectral filter H(z)) + always-on Shared Bridge + ~0.4 MB language LoRAs fits in ~8–21 MB RAM.

2 · Zero-Stall Lookahead Memory

Sentence Pinning + Async Preload

Pre-synthesis lookahead pins active L1, L2 adapters before frame 0, preloads L3 in the background, and uses ΔWshared on misses (p99 stall < 5 ms).

3 · Smooth Acoustic Transitions

1-Word Debounce + 20–40 ms Blend

Skips unnecessary expert flips on single borrowed words and cosine cross-fades expert gates, pitch (F0), and spectral envelope across phrase switches.

Immediate Verification Decisions
  • Primary Phase 1–2 language pair: Mandarin–English (SEAME / ASRU benchmark) vs. Malay–English?
  • Target edge hardware baseline: Mobile ARM Cortex-A core vs. Raspberry Pi / desktop single-thread CPU?
Interactive Deck Shortcuts

Press N (or click Notes below) to toggle speaker notes on any slide · Press E to edit any text inline · Press Ctrl+S to save your edits as a standalone HTML file.

Thank you · Questions & Technical Feedback Welcome Slide 15 / 15
01 / 15
Speaker Notes · Slide 1