¡Ajipirijou! josé albornoz's blog

How I built my own Twitch-Plays-Pokémon: Part 1

I gave this talk at Baruco 2014. I'm publishing it now as a series of blogposts.
You can find the slides in my speakerdeck. You can also see the recording in YouTube.

Introduction: How I discovered TPP

When was the last time you worked on a completely non productive project, just for the fun of it?
Today I'm going to tell you about the last time I did.

Let me take you back to February 14th of this year.
It was Valentine's day and a cold Friday night.
While most people that have a partner were probably on a romantic dinner, if you were single, you were more likely doing less-exciting stuff, like spending the night on the internet.

So there I was, on reddit. After browsing for a while,
I stumbled upon this thing called Twitch Plays Pokémon.
It took me a couple minutes to realize what was going on.

Some dude hooked up a GameBoy emulator to a streaming website and allowed everyone to control it through a chat system.
Hundreds of people were playing the same Pokémon session at the same time.

My first reaction was: "OMG this is awesome!"
My second reaction was: "I need to make my own".

How can I make my own: The First Approach

Saturday morning.
I woke up early and went to my bookshelf.
Opened my cookbook, not an ops cookbook, but more of a programming cookbook, and headed to the "How to make a Twitch Plays Pokémon" recipe.

What do we need to make a Twitch Plays Pokémon?
The ingredients list reads:

For the GameBoy emulator, I've always liked "Visual Boy Advance", just a typical emulator written in C.
I had no idea how to stream my screen, but that was probably trivial, so I just skipped it.
The real challenge was controlling the GameBoy emulator with Ruby.

The first thing that popped into my mind was:
How the heck do I control an emulator with Ruby?
Can I even control my Mac with Ruby?
Most of the time I write Rails applications, and other than Selenium, I couldn't think of a Ruby project that actually "controls" the computer.

After googling for a while I ended up in a stackoverflow question titled something along the lines of "How do I send keystrokes from a Ruby app?".
The solution said: "You could install the ffi gem and use it to load user32.dll"
That's "just what i needed!", wrote "Lucas d. Prim".
That does the job! the only problem is that I have no Windows Machine and I want to stay like that for the next 30 years.
Plus, my idea is to run this in a server with no desktop environment, so the Windows solution was definitely a no-go.

Scrolled a bit more and somebody suggested to use the "Java Robot" class through JRuby.
It also requires a desktop environment, but at least I can use it in my Mac.

I had some experience porting Java code to JRuby, so by the end of the day I had a very simple sinatra application that could control an emulator in my Mac.

Thank god somebody from Oracle actually took the time to write a Java class that simulates keypresses.

Read Part 2 ->