The shape of emerging language learning and absorption

Can there be a software solution that answers the present problems of unguided language absorption?

The Problem

Though I’m rusty, I am fluent in a second language (Spanish). I am also an ongoing learner of a third language (Japanese). In my academic past, I’ve taken classes for both – Spanish for the relatively easy ‘A’ and Japanese for the challenge. Working through Japanese academic materials a few years later, it’s clear that things are a mess. Genki is the premier learning package for mastering the fundamentals of the Japanese language. But the materials are scattered across a textbook, a workbook, two sets of audio files, at least three apps, and additional online-only help.

Even brand new packages for Japanese, such as Tobira Beginning Japanese aren’t much better – on top of the obligatory textbook and forthcoming workbook, the Tobira site has a gated set of instructional videos, in-depth grammar guides (not in the textbook) and other errata. For pretty much any of these options, Anki is the unspoken requirement for SRS vocabulary acquisition alongside a smattering of kanji guides, tadoku story books, and YouTube channels. Trolling through reddit, you’ll find pretty much any language learning clique is obsessed with Anki, so it’s unavoidable. Keeping track of this mess across disparate multimedia is a depressing exercise for those who aren’t learning as a full-time job.

Keep in mind that these are the best language acquisition materials for Japanese — Marugoto is substantially less guided and several other series are outdated.

Frankly, I’m surprised that a company like Ubisoft or Nintendo hasn’t come along and revolutionized language learning. At least, they could radically change those painful early months of picking up a language.

The Idea

So you have Nintendo, which opened up the concept of gaming during both the Wii and Switch era, creating more levels of interaction between player and console. In the early 2000s, Nintendo released a motion control console alongside immersion software for music and exercise. After the momentary failure of their Wii U console in the late 2000s, Nintendo merged their portable and home console markets by releasing a relatively- powerful, fully-portable console, the Nintendo Switch, in 2017.

You also have Ubisoft, that created a real guitar and bass learning tool (Rocksmith) using a custom guitar jack that converted instrument sounds into signals recognizable by the software. At minimum, tens of thousands have succeeded in drastically improving their bass and guitar skills.

Now imagine combining those concepts and using them to tackle the language learning problem. Imagine you turn on a language learning game on your handheld or celly and you have this:

• Voice recognition with avatars representing teachers, an otaku friend if you’re learning Japanese, a fútbol-obsessed friend if you’re learning Spanish, random people, boss, significant other, bartenders, relatives, etc. Avatars pick up and dynamically respond to your questions, comments, and responses.
• These virtual friends have their own in-game drama, so you have to actually engage with them verbally to understand what’s going on.
• A system that corrects you, checks your intonation and tone, offers alternative ways to say things, etc. Basically, an in-game PDA that unobtrusively jumps in when it senses you need help.
• Some sort of writing element, through games, that gets you learning how to write.
• Media (movie clips, fake commercials, magazines, newspaper articles) that dynamically scales to your level. You would decide to go to the movies with your friend, for instance, and the “movie” would be just a little bit out of your comfort zone, but 95% intelligible
• The game periodically drops new vocabulary and grammar each day, instead of books that drop 50 at a time. Periodic quizzes and smaller SRS options would keep you picking up vocab every day.
• Dynamic conversation challenges like “Mention three food words in your conversation with Sandy”
• Ending each game at defined levels (whether it’s European A1-A2, JLPT, whatever).
• Some sort of overarching story involving your avatar friends that takes you from beginning to end in your language journey. Maybe you get married at the end of the game, leave the country, find a job, graduate school, etc.

I’ve been thinking about this since Rocksmith, and I think the technology is totally ready for this type of game. Yes, there is a vocal population that will complain that “nothing replaces the classroom environment.” There are some valid responses to this critique – such as that not everyone has the time or resources to commit to guided classes. I’ll go one further: I entirely and outright reject the argument that the classroom environment is always best.

1. MOOCs have drastically opened up entirely new fields of employment and changed lives. Firms hire data analysts and data engineers with nothing but data bootcamp experience and Coursera certificates.
2. Covid has also permanently constrained the in-class learning experience. The dream of the late 90s and 2000s of the interactive, remote classroom is here.
3. The introductory steps to anything are substantially more achievable through technology.
4. Finally, the classroom environment is itself a poor substitute for immersion. And yes, you did learn your first language in a classroom, but it doesn’t follow that this is most efficient for picking up a second language. Your first language also included a metric ton of immersion in your home life.

Let me add that every person is different and every person learns in a unique way. With that said, professional and personal fulfillment should not be gated behind university walls. Yes, I have a bachelor’s degree and a master’s from traditional universities, but I also have several other MOOC certifications that I include on my resume.

Walkthrough of the Concept

So here’s my concept, in classic product innovation-style:

• Imagine you have a console with a headset to pick up your speech.
• Now imagine that the software has fairly strong speech recognition.
• Create an open immersion game loaded with introductory terminology.
• Center the learning experience around virtual avatars, where your real conversations approximate an immersion environment.

Imagine the benefits:

• Software that never tires of you turning it on for practice, and doesn’t burn out on correcting you.
• Integrating media content that scales to your current level of learning. Feeling particularly slow today? The software backs up and doesn’t push you beyond your patience level. Mastered everything today? The game’s conversation throws in a small curveball and supports you when you ask about it quizzically.
• Did you stop doing the practice for a couple months, but want to get on the horse again? Just turn it on and let the conversation adjust itself to your current level.