Open-Source Arabic Dialect TTS Models and Reusable Training Pipeline

Editor’s note: In this article, Vadim Belski presents ScienceSoft’s R&D project on training open-source text-to-speech (TTS) models for Emirati and Saudi Arabic dialects. He describes the training data strategy, initial experiments, and the final reusable training pipeline for Qwen3-TTS — an open-source LLM-based model. The article also links to the resulting model files, browser demos, and sample audio outputs. If you are planning to implement a voice AI solution, you can consult Vadim or other AI software development experts at ScienceSoft.

Market Gap Behind This R&D Project

In enterprise voice AI projects, I often meet clients who hesitate to send customer speech to OpenAI or similar cloud services. That concern is especially important in key GCC markets, such as Saudi Arabia, the UAE, and Qatar, where data protection rules make cross-border movement of personal data a controlled process and often push companies toward on-premises deployments.

Another constraint for voice AI in the GCC is language. Modern Standard Arabic can sound out of place in everyday customer interactions, where people expect local rhythm, vocabulary, and pronunciation.

Mixed Arabic-English speech adds another issue. Arabic conversations often include English company and product names, acronyms, and technical terms. In my tests, even large voice models often distort English names by pronouncing them with an Arabic accent.

So, for GCC-based voice AI implementations, we ideally want a TTS model that can adapt to local dialects, accurately distinguish English words from Arabic, and be deployable on proprietary infrastructure. High-quality commercial TTS models that fit all these requirements are not always available for a specific enterprise use case. This gap pushed ScienceSoft to start its own open-source TTS R&D project.

Research on Available Open-Source TTS Models

From the beginning, my aim was production-grade quality. For this project, that meant stable inference, a deployable architecture, and preprocessing capable of handling numbers, dates, currencies, abbreviations, and Arabic-English text. I started with Emirati Arabic — one of the least represented Arabic dialects in speech AI.

Assembling the Training Dataset

At the start of the project, I did not find a ready-made dataset with usable licensing terms that combined natural Emirati Arabic, Arabic-English code-switching, and the transcript control I needed. So, instead of trying to build a full-scale production corpus from scratch, we created a controlled synthetic dataset to test whether an open-source model could be adapted under realistic low-resource conditions. For that, we applied a distillation-style approach, using a strong commercial model as the teacher. The team prepared Arabic texts with realistic English insertions across everyday dialogue scenarios. Then we used a commercial TTS provider to turn those texts into speech. Paid generation increased the experiment cost, but the provider’s Arabic speech quality gave us a reliable starting point for training.

We ended up with about 70 hours of synthetic bilingual audio in Emirati Arabic and English. For each training sample, we kept the generated audio together with the original text as a transcript, so the model could learn how written input maps to speech.

First Experiments With Phoneme-Based TTS Models

Before training the first model, I needed a text-normalization front end that could convert written input into pronounceable text. I used NVIDIA NeMo for this layer because its text normalization tools already covered common cases such as numbers, dates, and currencies. For the bilingual dataset, I adapted the front end to handle abbreviations and English words inside Arabic sentences.

I first tested two phoneme-based neural TTS models that generate audio based on grapheme-to-phoneme (letter-to-sound) mappings. For the experiment, I used the open-source FastPitch and VITS implementations available in NVIDIA NeMo.

The results quickly exposed the biggest constraint. Each training cycle took about a week, so every mistake pushed the next correction back by another long run and increased compute costs. Even after those runs, the models still made pronunciation errors, which suggested that phoneme-based models probably require substantially more dialect data.

As a practical workaround, I added a correction dictionary used to replace recurring mispronunciations with correct phonetic forms. The approach worked reasonably well with VITS.

How Qwen3-TTS Changed the Research Direction

When Qwen3-TTS came out in January 2026, I changed course. It is an open-source LLM-based TTS model that supports streaming and offers compelling capabilities, despite not officially supporting Arabic. I decided to test Qwen3-TTS with the same 70-hour Emirati-English training dataset. The result surprised me. After only two days of training, the adapted model could generate intelligible Arabic audio for short test phrases from the same dataset.

That shorter training cycle mattered, but speed alone was not enough to justify the project switch to Qwen3-TTS. The stronger reason was that Qwen3-TTS also raised the quality ceiling. Two capabilities made the model especially interesting for us:

• Natural speech and adjustable emotional tone. The two previous models sounded flat, even when they pronounced individual sounds correctly. Qwen3-TTS uses text context to adjust tone, rhythm, and pace, which made the output sound closer to live speech. I could also set an emotion, such as confidence or curiosity, through a separate generation instruction.
• Voice cloning. This capability uses a reference voice recording to carry individual speaker traits, such as timbre, pitch, and speaking manner, into newly generated speech. I tested this using my own English recording as a reference, and the adapted model generated Arabic that matched my voice characteristics.

Reusable Training Pipeline

After the first Qwen3-TTS experiments began to work, I packaged the repeated model adaptation and training steps into a reusable pipeline. I designed it to support training for other Arabic dialects once the team had dialect audio, matching transcripts, and a reference voice sample. Another goal was to produce models that could support real-world voice AI use, not only short demo clips. For that, the pipeline needed consistent data preparation, controlled training runs, checkpoint comparison, and a documented way to generate audio from a trained model.

The diagram below shows the process.

 

Model adaptation. Before training, I made Qwen3-TTS recognize Arabic as a separate language and connect it to the Arabic voice sample. Technically, I added an Arabic language identifier, initialized its embedding from existing languages, and registered the reference voice through the speaker encoder.

Training data preprocessing turns raw audio and text into a training format Qwen3-TTS can read. The pipeline stores each training sample as audio, transcript text, and a reference voice in a one-record-per-line JSONL file. The text normalization front end spells out numbers, currency amounts, abbreviations, and English insertions for accurate speech generation. Then the pipeline uses the Qwen3-TTS tokenizer to convert audio into speech codes and align them with the transcript.

The training loop teaches the adapted base model to reproduce the dialect while keeping training stable on available hardware. The training script fine-tunes all trainable model parameters in mixed precision. Gradient accumulation and gradient clipping help stabilize the run on available GPU resources. The voice encoder stays frozen, so the model learns the dialect without overwriting the reference voice signal. Each training run produces checkpoint models for output quality evaluation.

Evaluation loop. After each run, I needed a measurable way to choose the best checkpoint. For that, the pipeline tests audio from the corresponding checkpoint by transcribing it with automatic speech recognition and comparing the transcript with the source text. The word error rate (WER) indicates how well the checkpoint preserves the intended content. The evaluation results guide the next fine-tuning decision and the final checkpoint selection.

I first validated the pipeline on Emirati Arabic. To test reuse, I repeated the same workflow for Saudi Arabic, using a new dialect dataset built through the same distillation-like approach. The base model adaptation, dataset preprocessing, model training, and checkpoint evaluation workflow processes stayed the same. This second experiment proved the workflow as a reusable pipeline rather than a one-off Emirati experiment.

Open Artifacts Readers Can Inspect

Usage note: The artifacts are provided to help readers inspect the R&D results. They are not presented as ready-to-deploy production models.

Current Model Limitations and Next R&D Steps

During testing, native reviewers found that, on longer passages, the model sometimes drifted into Arabic that sounded foreign. My current hypothesis is that I pushed Arabic-English code-switching into the training too early. Because I trained Arabic and English together, English pronunciation patterns may have started to leak into the Arabic output. I plan to fix this issue first, then use the same pipeline for other Gulf Arabic dialects, such as Qatari, Kuwaiti, and Omani Arabic.

What This Project Means for Voice AI

I see strong potential in Qwen3-TTS as a base for dialect-specific voice AI, far beyond Arabic. The same pipeline can fine-tune the model to any other underrepresented dialects.

• First, this approach addresses requirements I keep seeing in enterprise voice AI projects. Companies often need on-premises deployment, natural-sounding speech, and reliable code-switching, but available TTS systems often force trade-offs in these areas.
• Second, voice cloning adds another practical advantage. A team can build a library of approved voice profiles for support agents, virtual assistants, training scenarios, branded voice experiences, or any other business cases.
• Moreover, Qwen3-TTS is already supported by vLLM-Omni, an open-source framework for serving multimodal AI models. Over time, this could reduce the custom work needed for fast streaming, reliable serving, and production integration of Qwen3-TTS-based voice AI systems.

Together, these factors make open-source Qwen3-TTS a strong alternative to commercial TTS products.

Check our AI consulting services