Sony Patents In-Game Sign Language Translator

Sony has taken a significant step towards enhancing gaming accessibility for deaf players by filing a patent for an in-game sign language translator. This innovative technology aims to bridge communication gaps by translating different sign languages within video games.

Sony Patents ASL to JSL Translator for Video Games

Proposed to Use VR Devices and Work Over Cloud Gaming

Sony's latest patent, titled "TRANSLATION OF SIGN LANGUAGE IN A VIRTUAL ENVIRONMENT," introduces a groundbreaking system designed to facilitate real-time sign language translation in video games. This system would enable American Sign Language (ASL) to be translated into Japanese Sign Language (JSL) for Japanese-speaking users, enhancing the gaming experience for deaf players.

The technology outlined in the patent involves translating sign gestures into text, converting that text into another language, and then transforming it back into the corresponding sign gestures of the target language. This process allows virtual indicators or avatars to communicate sign language in real-time during in-game interactions.

Sony explains in the patent, "Implementations of the present disclosure relate to methods and systems for capturing sign language of one user (e.g., Japanese), and translating the sign language to another user (e.g., English). Because sign languages vary depending on geographical origins, sign language is not universal. This provides a need for appropriately capturing the sign language of one user, understanding the native language, and generating new sign language as output for another user in their native sign language."

To implement this system, Sony suggests using VR-type devices or head-mounted displays (HMDs). "In some implementations, the HMD connects via a wired or wireless connection to a user device, such as a personal computer, game console, or other computing device," Sony detailed. "In some implementations, the user device renders graphics for display through the HMD that provide immersive viewing of the virtual environment for the user."

Furthermore, Sony proposes that user devices can communicate seamlessly over a network with a game server. "In some implementations, the game server executes a shared session of a video game, maintaining the canonical state of the video game and its virtual environment," Sony said, "and to which the user devices are synchronized regarding the state of the virtual environment."

This setup allows users to interact within the same virtual environment over a shared network or server. Sony also mentioned that the game server could be integrated into a cloud gaming system, which "renders and streams video" between each user device, ensuring smooth and efficient communication.

By developing this technology, Sony aims to create a more inclusive gaming environment, where deaf players can fully participate and enjoy the gaming experience regardless of the sign language they use.