Introduction

Captioning is reinforcing audio experiences with text.  This offers an alternative experience for those that are unable to hear the audio or understand the language being spoken.  We see this concept used all the time on TVs in noisy bars or busy airports.  Perhaps you’ve even kicked back on your couch to watch a foreign film with subtitles in your native language.  This concept works well if you have a video screen, a specific language, and a stationary audio source.

Unfortunately, what works for bars, airports, and living rooms doesn’t always do the trick in themed attractions.  If you consider a dark ride, each guest travels through a carefully crafted experience.  They go from queue line to pre-show, to load platform, and then to the ride experience itself.  Throughout their journey, audio may come from many sources besides video screens.  The guest might encounter a talking animatronic character, safety announcements in corridors, or background sounds to enhance the themed environment.  We must also consider that each guest has unique needs.  Forcing captions upon people that don’t want them may distract from their experience.  Guests with hearing disabilities will want to see captions that describe the audio experience.  International visitors will prefer to see dialog translated into their native language.

How can we enhance the experience of each guest with special needs without affecting the experience of others?  Simple!  We offer them a portable device that is aware of their location in the attraction and can deliver captioning uniquely suited for them.

This guide demonstrates how to create a system that delivers Augmented Reality (AR) captions to guests within a dark ride attraction.  Since Alcorn show control devices like the V16X and RidePlayer are aware of show timing and audio throughout the entire experience, they are the ideal systems to stream the captioning data.  Captions are then delivered to the guests based on their needs and location using a seamless combination of iBeacon technology, Epson Moverio smart glasses, and the Birket Engineering InThrall app.

Integrating the System

As you can see from the application diagram, we’re going to implement captioning for a dark ride attraction.  Dark rides involve separate systems for managing areas like queue lines, pre-shows, and individual ride vehicles.   The idea is that each system is responsible for streaming its own captioning data via WIFI Multicast.  For example, a ride system with a pre-show and 20 ride vehicles would involve at least 21 simultaneous multicast streams of captioning data.  When the guest steps into the preshow area, the smart glasses will sense their location and “tune in” to the Preshow captioning stream.  When the guest boards a vehicle, the glasses will tune in to the captioning stream for their vehicle.

System Components

Let’s look at the gear we have designed into this ride system and learn about the role it plays in the captioning experience.

RidePlayer – Onboard Synchronous Audio Player and Show Controller

This product is designed to endure the high-vibration environments of ride vehicles and to provide many features to reduce the need for auxiliary equipment aboard the vehicle. Some of the key features include 16 channels of polyphonic audio playback as well as onboard show control.  Other useful features like DSP, amplification, network audio, voltage monitoring, and GPS integration are all rolled into a nice compact, rugged, and energy-efficient package to make this product the ultimate onboard audio and control solution.

For this dark ride application, a RidePlayer is installed on each ride vehicle.  This device links with the ride control system and coordinates with the offboard show control systems.  This ensures an onboard audio experience that synchronizes perfectly with all the show elements going on around the vehicle.  Captioning data is added alongside the audio tracks in a show control timeline so that they are streamed in perfect time with the audio experience.

V16X – Show Controller

The V16X is responsible for show control in the preshow area.  It is fully programmable using our free WinScript Live software and remotely controls all the equipment in the pre-show.  This includes triggering AV playback in the AV Binloop Uncompressed unit.  It also includes streaming captioning data that is synchronous with the preshow AV presentation.

A/V Binloop Uncompressed – Multi-channel Synchronous Video Player

The purpose of this unit is to provide 3 channels of 1080p60 uncompressed playback for a 3-screen experience in the pre-show area.  Content is stored on solid-state media drives as uncompressed Targa sequences.  This product physically connects to each display device using a 3G-SDI connection to provide video without the need for extension devices.  This device also connects to the V16X via Ethernet so that video clips can be played at the appropriate time and synchronized with other devices in the system. This device synchronizes with the V16X so captions streamed from show control will be in perfect time with the AV presentation.

Epson Moverio BT-300 – Smart Glasses

This is a smart device that runs the Android OS and functions in a similar manner as tablets and smartphones.  The key difference is that the display output of the device is not a touch screen, but rather a projection that appears within the wearer’s field of view.  This concept is often referred to as Augmented Reality (AR).  When the display is black, the wearer can see right through the glasses.  In this application, we use this technology to display captions within the guest’s line of sight while allowing them to see the environment around them.

iBeacons

These devices are intelligent transponders based on Bluetooth LE technology.  Smart devices like tablets, phones, and smart glasses can sense when iBeacon devices are near and read unique information from them.  For this application, this unique information contains a multicast IP address associated with captioning data for a specific area.  iBeacons installed in the pre-show area will share the multicast address for preshow captions (224.0.0.1).  IBeacons installed on the ride vehicles will share the address for that vehicle (224.0.0.11, 224.0.0.12, etc.).

Birket Engineering InThrall – Captioning App

This is an application developed by Birket Engineering that can run on any Android-based device; including the Epson Moverio smart glasses.  This app is responsible for combining the functionality of the smart glasses, iBeacons, and captioning streams from Alcorn McBride devices into a seamless captioning experience for the guest.

InThrall runs automatically when the glasses power up.  Most of the time, a black screen is displayed so that the guest can see right through the glasses and enjoy the attraction.  When an iBeacon is sensed, the app tunes into the multicast address specified by the iBeacon and begins displaying captions for that area.  The location, size, and color of the captions are flexible and are determined by the programming of the captioning data.

WIFI Network Infrastructure

The Epson Moverio glasses depend on a WIFI network to receive captioning data.  This requires an infrastructure of network switches, routers, and wireless bridges to ensure WIFI distribution throughout the attraction.  Keep in mind that WIFI is only required for glasses to receive captioning data.  Other types of wireless or wired networks may be more suitable for integrating devices like ride control systems, show control systems, and AV systems.

Implementation

Show Control Programming

The captioning for this attraction is implemented with two show control scripts; one for the V16X running the Preshow and another for a RidePlayer onboard one of the ride vehicles.  The V16X is responsible for streaming captions intended for guests in the Preshow area, and the RidePlayer will stream captions for guests onboard the same ride vehicle (RV).

These two scripts have been included with this application note and are appropriately named:

AR Captioning – Preshow.WSL

AR Captioning – RV.WSL

Although Alcorn McBride goes through great effort to make this programming significantly easier than many other control systems, there is a learning curve with using WinScript Live, the V16X, and RidePlayer. To learn more about this interface, we offer free training in the form of interactive in-person classes and online courses.  We also provide many application notes like this one that demonstrate how to use our products in various applications.

Devices

A great first step when writing any script is to configure the list of devices that will be connected to the show controllers.  This involves browsing a comprehensive list of devices in the WinScript library by manufacturer and model number, choosing the device, and then configuring the physical connection to the device (i.e. Ethernet, Serial, etc.).

The Preshow V16X must control the AV Binloop Uncompressed and stream captioning data to the InThrall app via a unique multicast address (224.0.0.1):

The RV RidePlayer must stream captioning data to the InThrall app on a different multicast address (224.0.0.11):

Sequences – Preshow

Sequences are the heart of the show control script and contain all the functional events that are programmed.  Most of the sequences of the Preshow V16X are devoted to tasks like starting or stopping the show, controlling elements of the show such as the doors and monitoring show status.  The actual show itself is handled within the Timeline_Preshow sequence.  This is also the only sequence that is important for captioning purposes.

When we look at the events within Timeline_Preshow, we see a video track representing the playback of the preshow presentation.  Below, there is a group called Captions that contains several Send Caption events.  These are used throughout the timeline so that the captions are transmitted at the appropriate time during the preshow.  The timing of these captions can easily be adjusted by dragging the events or modifying the execution time on the left.

Sequences – Ride Vehicle

Most of the sequences in the RV script are devoted to triggering timelines for the various scenes within the ride in sync with the offboard systems.  To learn more about these concepts, be sure to check out our application notes devoted to Dark Rides.  This application will only focus on the sequences that pertain to captioning.

Here’s what the RV sequences look like:

All captioning within this script is contained within the Timeline_Scene sequences.  The Timeline_Scene sequences are timelines that trigger audio playback onboard and transmit captioning data in sync with the audio track.  The audio track appears in the timeline as a ribbon that represents the length of the clip.  Just like the Preshow timeline, you’ll find a Captions group containing Send Caption events that transmit captions at the appropriate times throughout the scene.

Configuring iBeacons

Although iBeacons are a standardized technology, the configuration interface of these devices is typically unique to their manufacturer.  This means that the method in which the iBeacon is configured may look a little different than this example, but the same key concepts will still apply.

iBeacons have configurable values called Major and Minor that are shared with nearby smart devices.  The InThrall app is designed to associate these fields with a Multicast IP address.  The idea is simple.  If the smart glasses get close to an iBeacon, the Major/Minor values of the iBeacon tell InThrall which multicast address to listen to for captioning data.  All you need to do is make sure that these fields are configured for the same Multicast IP address that the show control device is using to transmit captioning data.

Here’s how you configure the Major ID and Minor ID fields to configure the multicast address:
Multicast address = AAA.BBB.CCC.DDD
Major ID = (AAA*256) + BBB
Minor ID = (CCC*256) + DDD

Here’s how you would configure the ID’s to use the default multicast address 224.0.0.1:
Major ID = (224*256) + 0 = 57344
Minor ID = (0*256) + 1 = 1

Configuring Smart Glasses & Birket Engineering InThrall App

The Epson Moverio smart glasses are Android-based devices.  Linking the glasses to the WIFI network of the attraction is just a matter of launching the WIFI settings in the Android OS.  From here, you can scan the available WIFI networks and connect the device.  Once configured, the device will connect to the WIFI network automatically whenever it powers up.

Next, you need to install the Birket Engineering InThrall app.  InThrall is a licensed app that must be side-loaded onto your Android device.  Please contact Birket Engineering for more details on obtaining the app and licensing:

http://www.birket.com

Once you reach this point, all you need to do is launch the InThrall app.  The screen of the device will go completely black, allowing you to easily see through the glasses.  It will tune into Multicast address 224.0.0.1 by default and begin scanning for iBeacon devices.  If an iBeacon device is detected, it will read the Major/Minor values from the iBeacon and tune in to the associated Multicast address.

Conclusion

This application note can serve as a starting point in implementing captioning with your own application.  Keep in mind that it’s easy to scale the system to include as many or as few captioning sources as you need.  In fact, you don’t even require the iBeacons if you only have a single captioning source.  You can simply program your Alcorn McBride show controller to Multicast captions to InThrall’s default address.

Now it’s time for you to implement your own project with the V16X, RidePlayer, and the AV Binloop Uncompressed.  Please don’t forget that we are here to help you so feel free to contact us with questions.