Towards Scalable Sign Production: Leveraging Co-Articulated Gloss Dictionary for Fluid Sign Synthesis

IIIT Hyderabad, University of BATH
ICVGIP 2025

Supplementary Teaser Video

Abstract

Sign Language Production (SLP) systems can significantly improve accessibility for Deaf communities by translating spoken or written language into sign language videos. For millions of Indian Sign Language (ISL) users, such systems could bridge persistent communication gaps in education, healthcare, and public services. However, SLP in ISL is hindered by the scarcity of annotated datasets and the expressive complexity of the language. Existing ISL datasets, such as iSign and ISL-CSLRT , are designed for recognition and lack gloss-level\footnote{Gloss is word representation of sign.} annotations, forcing full-sentence modeling that struggles to generalize to new constructions. We propose an interpolation-based approach that stitches together individual gloss-level signs using modern text-to-motion frameworks like FramePack \cite{zhang2025framepack}. This produces contextually accurate and grammatically consistent sign sequences while retaining fine-grained articulation. To support this pipeline, we curate a 1,000-sentence ISL dataset, translated by a native interpreter and annotated with precise gloss boundaries. Results show that with high-quality gloss-level supervision, interpolation-based synthesis offers a practical, scalable path for inclusive SLP in low-resource sign languages like ISL.

Method

Method

For Indian Sign Language Production, we exploit fine-grained gloss-boundary and sign-type annotations with SOTA interpolation method; we show it enables smooth, context-aware visual synthesis for sign-language generation, and to support this, we collected a carefully curated 1,000-sentence interpreted ISL corpus - SanketVaani-1K.

Dataset: "SanketVaani-1K"

SanketVaani-1K SanketVaani-1K SanketVaani-1K

SanketVaani-1K: 1,000 ISL sentences with precise, manually annotated gloss boundaries. Annotated by expert signers for precision and linguistic consistency. spans 40 practical domains such as healthcare, education, banking, and civic services, cooking etc supporting real-world deployment.

Comparision

BibTeX


This work is licensed under a Creative Commons Attribution-NonCommercial-
NoDerivatives 4.0 International License.
ICVGIP 2025, Mandi, India
© 2025 Copyright held by the owner/author(s).
ACM ISBN 979-8-4007-1930-1/2025/12
https://doi.org/10.1145/3774521.3774576