Introduction
Language is more than just words, it's culture, identity, and connection. For millions of Urdu speakers worldwide, especially those in the diaspora, Roman Urdu has become the bridge between traditional script and digital communication. But what if we could automate the translation between Urdu script and its Romanized form using deep learning?
In this project, I built a sequence-to-sequence Neural Machine Translation (NMT) system using Bidirectional LSTM networks to translate Urdu text into Roman Urdu. Working with poetic text from the beautiful world of Ghazals, I explored the limits of what BiLSTM-based models can achieve for low-resource, culturally rich datasets.
The Challenge
Translating Urdu to Roman Urdu isn't just about character mapping, it's about understanding context, phonetics, and linguistic nuances. Urdu's right-to-left script, combined with the ambiguity in vowel representation, makes this a fascinating challenge. Add poetry into the mix, and you're dealing with archaic words, metaphors, and stylistic variations that test any machine learning model.
Dataset: Poetry as Training Ground
I used the Urdu_ghazals_rekhta dataset, a collection of classical Ghazals(a form of lyric poetry) that includes Urdu script and its transliterations. Ghazals are known for their emotional depth and complex vocabulary, making them an excellent testbed for translation models.
The dataset presented unique challenges:
- Poetic language: Words and phrases rarely used in everyday Urdu
- Inconsistent transliteration: Variations in how Roman Urdu is written
- Limited data: A relatively small corpus compared to major language pairs
Building the Model
Architecture Design
I implemented a classic encoder-decoder architecture with a modern twist:
Encoder:
- Bidirectional LSTM with 2 layers
- Processes Urdu text from both directions to capture contextual information
- Hidden size: 512 dimensions
Decoder:
- 4-layer LSTM that generates Roman Urdu character by character
- Attention mechanism to focus on relevant parts of the input
- Embedding dimension: 256
The bidirectional nature of the encoder was crucial—Urdu's grammatical structure often requires understanding the full context before determining the correct transliteration.
Preprocessing Pipeline
Data preparation was critical:
- Normalization: Cleaned Urdu text, standardized Unicode characters
- Tokenization: Character-level tokenization for both Urdu and Roman scripts
- Vocabulary Building: Created separate vocabularies for source and target languages
- Data Splitting: 50% training, 25% validation, 25% testing
Experimentation Journey
I conducted extensive experiments to find the optimal configuration:
Experiment 1: Baseline Model
- Embedding dim: 128
- Hidden size: 256
- Learning rate: 1e-3
- Result: BLEU score of 32.4, high perplexity
Experiment 2: Increased Capacity
- Embedding dim: 256
- Hidden size: 512
- Learning rate: 5e-4
- Result: BLEU score improved to 45.7, significant perplexity reduction
Experiment 3: Regularization Focus
- Added dropout (0.3)
- Batch size: 64
- Learning rate: 1e-4
- Result: Best performance - BLEU: 51.2, lowest CER
The sweet spot was finding the balance between model capacity and overfitting. The poetic nature of the data meant the model needed enough complexity to capture nuance but not so much that it memorized training examples.
Results & Insights
Quantitative Metrics
- BLEU Score: 51.2 (final model)
- Perplexity: 8.7
- Character Error Rate: 12.3%
Qualitative Examples
Example 1:
- Input (Urdu): محبت کا سفر
- Ground Truth: mohabbat ka safar
- Model Output: mohabbat ka safar
- ✓ Perfect match!
Example 2:
- Input (Urdu): دل کی دھڑکن
- Ground Truth: dil ki dharkan
- Model Output: dil ki dhadkan
- ~ Close, minor vowel variation
The model performed impressively on common words and phrases but struggled with rare poetic terms and proper nouns, which is expected given the limited training data.
Key Learnings
-
BiLSTM Still Relevant: Despite the Transformer hype, BiLSTMs can achieve respectable results for specialized tasks with limited data.
-
Character-Level Works: For transliteration tasks, character-level models capture phonetic patterns better than word-level approaches.
-
Poetry is Hard: The model's performance on everyday Urdu was better than on archaic poetic vocabulary, highlighting the importance of domain-specific training data.
-
Hyperparameter Sensitivity: Small changes in learning rate and dropout had significant impacts on convergence and generalization.
Deployment & Accessibility
To make this research accessible, I deployed the model using Streamlit, creating an interactive web application where users can input Urdu text and get Roman Urdu translations in real-time. The deployment process taught me valuable lessons about model optimization for production environments.
Future Directions
While proud of what I achieved, there's room for improvement:
- Transformer Architecture: Experimenting with attention-only models
- Data Augmentation: Expanding the dataset with modern Urdu text
- Multi-task Learning: Joint training on translation and transliteration
- Contextual Embeddings: Incorporating pre-trained Urdu language models
Conclusion
This project was a deep dive into the intersection of language, culture, and artificial intelligence. Building an NMT system for Urdu to Roman Urdu taught me not just about neural networks, but about the beautiful complexity of human language and the challenges of capturing it in code.
Tech Stack: PyTorch, Python, Streamlit, Kaggle GPU
https://colab.research.google.com/drive/17yxUqoi6LcNHqq3IMq7sud1ies7CJyuP?usp=sharing
