Talking to Video Games

Craig Lupienski

“EAT MY SHORTS” the Game Boy hissed at me, as if through a tin can telephone. It was the first time I heard a video game talk to me. Bart Simpson was not happy that he was stung by bees, and he verbally made me aware of his displeasure. In the years following Bart’s cries, I heard a severely compressed “GHOSTBUSTERS!” shout on my NES and a melodic “SAAAAYYY-GAAAAHHH” sing out from my Genesis. I liked hearing my video games talk. Partly because of the sheer novelty in the 80s and early 90s, but also because speech felt like it fostered a deeper connection.

sinistarThe words and phrases were often exclamations. Communicative in both function and feeling, but rarely conversational. Gauntlet alerting frantic players that the warrior is near death is wonderfully useful and Sinistar’s proclamation “BEWARE, I LIVE!”  can still invoke a sense of anxiety. Early video game speech was novel, though not necessarily without purpose. The “Sega chant” may have consumed an eighth of Sonic the Hedgehog’s cartridge for no real apparent reason other than to impress, but I find it hard to argue that Gauntlet and Sinistar would be the same games without their little bits of speech. It’s not conversational or relevant to plot development, but it’s absolutely vital to their designs in different ways.

In the mid to late 90s, I didn’t have the luxury of dividing my time and discretionary income across too many systems, so I threw my hat in with Nintendo, the Nintendo 64 providing me with the multiplayer games that were more important to me at that time. As such, I didn’t experience many of the cinematic games touting fully voiced dialog during that time period. I remember, though, playing Metal Gear Solid in 1998 for the first time and finding myself nothing less than engrossed. I loved every minute of the soap opera. I loved every line of dialog delivered by bobbling heads. I loved Metal Gear Solid, and I still do, if for nothing else than that feeling it first gave me.

Now, I suppose I feel fully voiced soap operas are the novelty. I usually find myself bored and reaching for my phone to check Twitter. Of course, this is contrary to popular thought and the current market. Soap operas are not a novelty. Cinematic games are less a subset than they are a driving force. Every AAA game talks now, and at great length, but I usually struggle to remember what anyone says. It’s not as interesting as Sinistar taunting me from afar or as useful as Gauntlet warning me of impending death. It’s just babbling. We let video games talk, and they haven’t shut up since.

heyyoupikachu

My time spent with underdog systems in the 90s did, however, expose me to experiences built on the inverse: games that invited me to them. Hey You, Pikachu! on the Nintendo 64 and Seaman on the Dreamcast both came bundled with microphones and recognized specific words and phrases. Sometimes, anyway. The technology was nascent and I’m not convinced it worked a good chunk of the time, but the idea that you could wish Pikachu a good morning was there. It’s a good idea. A valuable idea.

Sega’s Binary Domain included an optional voice recognition component in 2012. I’ve read widespread reports that the feature didn’t work very well for many people, and I had some difficulty myself, but it seemed to work for the most part in my experience. I could verbally tell my teammates to wait, to provide cover fire and I could even engage in very basic and very short yes-or-no conversations. In one instance, I accidentally fired at a teammate who quickly chastised me. Reflexively, I said “sorry” into my microphone. The teammate forgave me. I was stunned. This wasn’t exactly an emergent situation; after all, the AI was designed to recognize the word “sorry,” but I didn’t actually intend on apologizing. It just happened. And the game reacted.

fable3Video games don’t often give us much room for interaction. Death is usually how we communicate with the world. Games like Ico and Fable III incorporated a handholding mechanic (one to great acclaim, the other to some derision), and it’s still an aspect I point to when I talk about letting players interact with their world in ways other than killing. Fable III was still built on a foundation of murdering bad guys, but holding hands with my wife or child served as a welcome foil.

As much as I would like to see more of that sort of thing, it’s still an abstract representation of a real physical act. Pressing a button or pinning a shoulder trigger to hold hands isn’t quite the same as talking to a game to hold a conversation. The latter is not abstract at all. Talking is something most people do naturally, and given games can talk back, it only makes sense that verbal communication with a game could serve to further draw the player in. If only used as an ancillary component like handholding in Fable III, communicating with games can be extraordinarily valuable.

There are hurdles, of course. Voice recognition technology is still far off from feeling natural, and games can only recognize so many words or phrases and offer only so many reactions. This is not different than how a game interprets button presses, but button presses are not analogous to real life. We understand the limitations. We expect more from speech because of how it relates to the real world, and video games just aren’t up to snuff yet. Technology in general is not up to snuff. Siri on the iPhone not only doesn’t understand me from time to time, she will correctly display the words I said on the screen but “autocorrect” my speech and decide I actually meant something else. Still, when Siri accurately sets a timer I requested, I have to stop myself from thanking her. That’s the power of verbal communication done well.

I am a big fan of games toying with how we interact and how we experience attachment. Games, in general, are still not very good at experimenting with these things. You will be tasked with indiscriminately mowing down waves of bad guys for 20 minutes only to be rewarded with five minutes of incongruent dialog designed to engineer unearned emotion. You will be killing again in no time. I don’t believe we have universally bad writers in the industry, nor do I feel we have reached the limits of what video games can say to us. But we’re doing something wrong. Video games have veered too far off course from the interesting, valuable speech of Gauntlet and Sinistar into mealy-mouthed, lengthy exposition for no real apparent reason other than to impress. It’s minutes, hours, of the “Sega chant” over and over again.

Let’s scale it back and let’s open the lines of communication.

lifeline

3 Comments

  1. I agree 100%! I want new ways to interact. I want to look my party members in the eye when I send them off on the suicide mission.

    Personally, I think there are two other forms at work here. Conversation is an intimate, human thing that is more than words, it’s body language, cadence and eye contact. On a AAA title, each one of those things is managed by a different team of people. (writer, animator, actor, facial animator) It’s incredibly hard to keep them working in tandem.

    Secondly, is that we haven’t really found a way to let the player impart the same emotions into the game. I’m not going to go into another Kinect eulogy here, but I bemoan that we have again, decided that this generation of consoles is going to continue to constrain player input through the tight, lo fidelity conduit that is the twin-stick controller. It’s an input device that is designed around the handgun and is effectively the rate limiting step in a lot of your grander designs.

    These experiences you seek are around if you look for them, but only as experiments. Facade still surprises me to this day. David Cage, for all the derision his work gets is actively trying to solve these problems.

  2. John

    I remember playing Hey You, Pikachu! when I was younger and not thinking that the technology didn’t work accurately (which it didn’t), rather that Pikachu just didn’t listen to me a lot of the time. That was still back when most bad games didn’t come across as bad to me, just hard. Hell, I remember really gritting my teeth and trying my best at Superman 64, genuinely wanting to get further in it.

    But with Hey You, Pikachu!, I was just really invested in the idea that I could talk to Pikachu. Some of it not listening to me was actually part of the game, I think. Something about it not really being my friend just yet and actively disobeying me. And it’s not that the line between that and the game straight-up not working was blurred, it was that that second idea didn’t exist. I was super invested in talking to Pikachu. Like…I was literally spending time with Pikachu. Hanging out.

Leave a Reply

Your email address will not be published.