Side Project: Conversation Player

Here’s a side project I’ve been working on. It’s a widget that plays back conversations in real time. Here’s a video demo of it:

The idea is to share text message snippets in a more digestible manner. The usual way to distribute a chat log to a friend would to send them a transcript of the messages, along with timestamps and all. The information is there, but it’s mentally taxing to read through it, and it’s especially difficult to make sense of the timestamps.

We figured it would be much easier to watch the conversation unfold in real time. This way, you see the conversation one message at a time, instead of all of it at once.

2

In the future, this will be a widget that you can embed in a blog or website. The reader can toggle the playback speed and use the slider to jump anywhere in the conversation.

We also built a set of utilities to import conversations from Facebook Messenger and Skype into a JSON format that our app can understand. Right now, these utilities are a bit clumsy to work with, so we’re holding back on releasing the project.

Conversation Player is built using React. It’s my first time using React, so it was good coding practice. I also had to learn the whole modern Javascript setup with babel, browserify, webpack, and so on. Everything is buzzword compliant now. Yay!

Roboroast: upload your photo to get an algorithmically generated insult!

I’d like to share a side project I’ve been working on for the past few weeks. Roboroast is an app that automatically generates humorous insults for you or a friend based on how you look. It was written in collaboration with my friend Andrei Danciulescu.

The basic operation is as follows. There’s a subreddit called /r/RoastMe where random people post a picture of themselves, and other people proceed to “roast” the person with funny comments making fun of his appearance.

Our app takes your photo and uses a face recognition algorithm to find a poster in /r/RoastMe who looks like you. Then we display the comments for your closest matches.

You can try it at roboroast.tk.

Sample Results

Here’s some roasts for myself:

Here’s some for Andrei:

High Level Overview

The project comprises of roughly 3 parts:

Part 1 is the Reddit scraper. We use the PRAW API to go through all posts on the /r/RoastMe subreddit, saving comments to MongoDB and saving images to the filesystem.

Part 2 is the Face++ uploader. Face++ is a cloud service with a REST API that handles our face matching. To use it, we upload all the images from part 1 into a “faceset” which we can query later.

The first two components only need to be run periodically, maybe once a month to update the faceset with new posts from Reddit. Part 3 is the webapp, which is the use facing component. It accepts user uploads, searches for matches using the Face++ API, and renders a list of insults to the user.

Technology Stack

As mentioned before, we used a number of third party APIs; PRAW for scraping Reddit posts, and Face++ for face recognition.

All the backend code is written in Python. The web app uses the Flask web framework, and is wrapped with NGINX and Gunicorn to handle connections and serve static files. We use MongoDB for the database.

The frontend is built with Bootstrap. We also use javascript libraries jQuery and handlebars.js.

The whole thing is hosted on a single AWS EC2 instance.

How good is the face matching?

The face matching is actually decent. Face++ produces reasonable matches most of the time.

To see the matching results for yourself, you can append ?r=1 to the end of the URL (on the results page). This is hidden by default.

Do the insults make sense?

Although the face matching does a decent job, we found that the quality of results were somewhat hit-or-miss.

When we envisioned the concept for this app, we assumed that most insults were going to make fun of the subject’s face. However, many insults refer to their non-facial appearance, or clothing, or objects in the background. Since we only do face matching, these comments will make no sense.

Other times, comments will refer to the title of the post — in other words, an insult depends on both the submission title and the picture. Again, these make no sense with only the picture.

We attempt to mitigate this with heuristics that analyze the comment, in order to exclude roasts which refer to the title or articles of clothing. This approach had limited success because natural language processing is hard.

Conclusion

When Andrei initially proposed this idea for an app, I thought the concept was pretty cool and unique. In a month or so we had a prototype, and I spent a few more weeks polishing the project for release. The quality of results you get is still highly variable, but we’re working on improving our algorithms.

In any case, it’s my first time with a lot of these technologies, and I had fun and learned a lot building it.

Teaching Myself Electronics: Zero to Arduino in 5 Weeks

I’m about to graduate with a degree in computer science, but I can’t describe how a computer works. Okay, maybe that’s an exaggeration. I can tell you all about assembly language and operating system kernels, and I have a good idea of how to build a CPU out of basic logic gates.

That’s where my knowledge ends. I have no idea how to build an AND gate, or how to coerce my 120V power supply to gently power these gates without frying them.

Learning is good, and this is a pretty big knowledge gap. I’m going to teach myself electronics. My plan is to learn by building things. There’s a lot of mathematical theory to learn, much of it is not that useful, and it’s easy to get bogged down in random details. Much better is to just experiment and go back and learn the theory when needed.

Week 1: Electronic Playground

The first problem was getting components. Unlike computer programming, where everything you need is on the internet, for hardware I’ll actually need to buy things. This is difficult when you don’t know exactly what you need. I also didn’t want a million different parts littering my bedroom haphazardly.

Eventually I settled on this all-in-one kit (cost $30).

It has a lot of components: LEDs, resistors, capacitors, even antenna and speakers. All the components are fixed to a board, and to connect them together, you use wires that clip to springs protruding from the board.

The kit comes with an instruction booklet that describes all kinds of things you can wire with it. For example, here’s a “harp” — it makes different tones when you hover your hand over the photoresistor:

This schematic is a bit too advanced for me at this stage — unfortunately the booklet doesn’t attempt to explain how it works.

That’s fine, the following books do an excellent job of starting from the basics:

After playing with this for a while, I learned a lot of basic things like how current / voltage / resistance works, how to read common schematic symbols, and how to decode a resistor.

Week 2: Multimeter

Electricity is invisible, and debugging circuits is difficult without being able to see what’s going on. I went ahead and got a multimeter (cost $20):

It was easy enough to measure resistance and voltage (both AC and DC). The current measurement was not very sensitive though and I could barely register any reading.

Around this time I attended a workshop in Manhattan that taught how to read schematics and build it on a breadboard. We made a 555 timer circuit which made a LED blink on and off:

I can’t understand how it works right now, but breadboards are pretty neat. Much easier than sticking wires into springs on my electronic kit at home.

Week 3: Baby steps with Arduino

By now I was reaching the limits of what my electronic kit could offer, and I needed to graduate to something more serious.

So I went to the nearest electronics shop and got an Arduino Uno kit (cost $90). The Arduino is a microcontroller and lets you prototype circuits easily with a breadboard. The Arduino Uno is only $25, but my kit comes with an assortment of components and sensors.

Before long I had the Arduino up and running. It runs a dialect of C, so I felt at home in the programming environment.

Here’s a program that blinks the onboard LED on and off in a loop (kind of equivalent of hello world):

// the setup function runs once when you press reset or power the board
void setup() {
  // initialize digital pin 13 as an output.
  pinMode(13, OUTPUT);
}

// the loop function runs over and over again forever
void loop() {
  digitalWrite(13, HIGH);   // turn the LED on (HIGH is the voltage level)
  delay(1000);              // wait for a second
  digitalWrite(13, LOW);    // turn the LED off by making the voltage LOW
  delay(1000);              // wait for a second
}

Week 4: Transistor Switching

I didn’t really know what a transistor did, but it’s what logic gates are made of and the backbone of all computers, so it can’t hurt to learn about them, right?

I started off building logic gates from transistors, but couldn’t get it work. It turned out that I misunderstood how a transistor operates (it’s not the most intuitive at first glance). Luckily, I had a friend in electrical engineering and she patiently cleared up my misconceptions.

Transistors are used for logic gates, but I didn’t know that transistors can also amplify a current. I also learned about the many different types of transistors.

Here’s a circuit that I built (from the Arduino kit manual). It uses a transistor as a switch to control a motor:

By the way, here’s how you make an AND gate with two transistors:

While wiring things up, I accidentally burned a LED and a BJT transistor. Apparently 5 volts without a resistor is fatal to many components. In software, if you mess up, you get a segmentation fault in your console or something — never the smell of burnt plastic in your room.

Week 5: Arduino Controlled Desk Lamp

Here’s an idea. Wouldn’t it be nice if your lamp can turn itself on when it gets dark? Useful or not, let’s build it!

This is actually a major milestone for me. Up until now, I’ve been mostly following existing schematics, using parts carefully selected by people who wrote the schematics. But for this project, we improvise everything from scratch. Oh yea, also it’s the first time I’m working with 120V alternating current.

First I got a $15 desk lamp. It’s the kind that plugs into the wall AC socket. I start by using a wire stripper to expose copper wires that I can plug into the breadboard:

I’m a bit nervous working with 120V current, obviously it’s a lot more powerful than the 5V of Arduino. Generally, 120V won’t kill or seriously injure you, but it does deliver an unpleasant shock.

After stripping the cord, the lamp can be plugged into the breadboard. But the voltage is too high for the Arduino to handle directly; instead, I need a relay which acts as a buffer and a switch. A transistor can act as a switch too, but relays can handle more current.

To detect light, I have a separate circuit that uses a photoresistor (changes resistance depending on amount of light shining on it). The Arduino can read the photoresistor by measuring analog voltage.

Here’s the schematic of my design:

Now we do software. It does a loop every 500ms, detects the amount of light, and decides whether the lamp should be on or off. It’s slightly complicated by the fact that when the lamp is on, it produces light and that affects the photoresistor readings. I need to compensate for that but it’s not too bad.

Here’s the code I came up with:

int photoPin = 5;
int lampPin = 3;

bool isLampOn = false;

void setup(){
  pinMode(lampPin, OUTPUT);
  Serial.begin(9600);
}

bool shouldTurnLampOn(){
  int lightLevel = analogRead(photoPin);
  if(isLampOn){
    return lightLevel < 70;
  }
  else{
    return lightLevel < 50;
  }
}

void loop(){
  if(shouldTurnLampOn()){
    digitalWrite(lampPin, true);
    isLampOn = true;
  }
  else{
    digitalWrite(lampPin, false);
    isLampOn = false;
  }
  delay(500);
}

Here’s a video demo:

Onwards

Actually my lamp idea probably isn’t that useful. Nevertheless, I made a lot of progress in just a few weeks and I’m proud of myself for that.

I’ve barely scratched the surface of all the cool things you can do with electronics, but it’s a good start. Now I have all kinds of ideas on what to build next. I’d better get to it!