Those Oh-So-Cool Signing Gloves

You know a video’s going viral when at least three people ask or tell me about it in as many days. 

On one hand*, COOL. And surely a good starting foundation for more advanced technology. On the other hand*, you know there’s a but coming…

  1. It doesn’t seem to address facial expression or body language, which are two essential components of sign language. Those two don’t just add flavor; they add meaning. Not sure how you can track these things just yet.
  2. “Pure” sign languages (i.e., ones that haven’t been adapted for speech) also tend to be more spatial than linear, plus the grammar is typically wildly different from spoken language. Even sign systems designed to transliterate speech generally don’t catch all components of spoken language, so I’d expect the voicing to be piecemeal at best.
  3. There’s no reverse translation; it’s sign-to-voice only, so it doesn’t make spoken language visually accessible for d/hh people.
  4.  The translation would probably be akin to running something through Google Translate– if not worse.

More than that, not all d/hh people know, use, or even prefer sign language. Even among signers, quite a few prefer to use transliteration rather than interpretation. Anecdotally, I’ve heard of captioning gaining far more popularity in colleges than sign language interpreting. Personally, while I have no qualms about using sign language to converse directly with other signers, I’ve had way too many interpreting mishaps to trust it for anything beyond basic conversation with English speakers. It’s far too much reliance on a third party’s understanding and expertise for my liking. I’m not that much more optimistic about a machine.

On the other other hand*, I could see these gloves working better for straight fingerspelling or Cued Speech, especially if they were combined with an automated transcription software. Unlike sign languages, cued speech has a finite set of eight handshapes that can be matched with a similarly finite selection of phonemes to produce words. I expect it’d sound incredibly robotic– which would certainly add an extra twist to the blog name, A Croaking Dalek— but there would likely be less potential for word jumble like what you’d get with ASL or Signed English. 

 * I promise these puns are completely unintentional.

TED Talk and Captioning

It’s finally been released: a TED talk on Cued Speech by Cathy Rasmussen.

Now, a fellow cuer, Benjamin Lachman, posted the video to our Facebook page and asked for some crowdsourcing on adding accurate captions. Another cuer, Aaron Rose, took him up on that request and that link up there on the website now has accurate captions– although for some reason, the direct Youtube link still transcribes Cued Speech as “cute speech” among other things (which admittedly makes me giggle).

For me, just seeing that request made me think of possibilities for captioning Cued Speech videos. See, I’ve captioned for sign language videos before, both my own and others. Captioning is not extraordinarily difficult, but it can be very time-consuming. Essentially, you’ve got to break up the caption lines and align them with the correct timestamps, and this entails a lot of right-clicking the frame and watching mouth movements to make sure you end on the right word. It’s even trickier when you have to translate the content into a different language, and a phrase in the original language doesn’t match up with the timing for the captioned language. This applies even when you’re the one who produced the content.

But with Cued Speech, I think seeing the handshapes with the mouth would help facilitate that process, especially when combined with speech recognition software that will automatically sync a pre-uploaded transcript with the correct timestamps. It would also enable other cuers to contribute captions to the video (as Aaron did) without any discrepancies in interpretation, because it’s straight-up transliteration. Not to mention, it would be excellent cue-reading practice for budding cuers.

It’s kind of exciting to think how accessible Cued Speech videos can be with the captioning process. In that kind of work, every little bit to make it easier helps.