Ephemerals

This is a term project developed during 15-112 Fundamentals of Programming at Carnegie Mellon University. My project was selected as the top 10 best projects in my class (among 200+ submissions). 

Inspiration

Nowadays more and more people are becoming control freaks, Ephemeral gives you an opportunity to sit down and appreciate randomness. Except for the two original images you uploaded, you have no control to what your post will be. Every time you do an operation such as commenting, refreshing…the poem will randomly change into a completely new and unique poem that never exists before. In addition, since the generating initiative is based on user’s operations, different users on different computer will see different poem for the same post at the same time! Also, for advanced users, try upload the same images and you’ll find interestingly different stylized results. Things on this website is ephemeral, the only way you can capture it is to paste the poem and leave a comment.

What it does

Ephemeral is an Avant-garde image sharing platform. It allows and only allows users to upload two images, one determines the content, the other determines the style.

The website will automatically recognize what is in the content picture and lists out three-most-probable word. For instance, if you put in a content picture of Des glaneuses, a famous painting of Jean-François Millet depicting several ladies working on a farm, Ephemeral will return “thresher”, “plow” and “Arabian camel”. The first two are accurate, though the third one isn’t, when you check the console you’ll find that the probability of “Arabian camel” is only 0.03%.

 

The next thing Ephemeral can do is to stylize your content image based on your content image, using a Convolutional Neural Network(with TensorFlow). If you put in a content image of Baker Hall and a stylize image of Van Gogh’s Starry Night, you will get a combined image with the content’s content and style’s style, in other words a “Starry Baker Hall”. This is achieved by randomly generating a noise picture based on the content picture, and then use gradient descent to minimize the difference between the newly generated picture and target style picture. You can still view the original images by clicking the navigation bar below. The last function of this web app is that, it can automatically generate a Shakespearean style poem based on the three key words, which it automatically recognized from the input image. This algorithm is based on a probability theory model called Markov Chain, basically finding the most possible trend of words.

Other features are somewhat mediocre, but still took me a lot of time and code, to make this platform fully functioning. You can login, logout, register. If you are logged in, you can create, delete, edit your posts, you can like, share to Facebook, leave a comment under every post, you can hide your post if you don’t want others to see it. You have full security, A cannot edit or delete B’s post in any way (it will raise a 404 not found page if you force it). If you don’t want to log in or register for any reason, you can still view the posts as a guest, but that’s really the only thing you can do.

How I built it

1. Image Recognition and Style Transfer

The image recognition function is based on Google’s deep learning Inception V3 model. And style transfer is based on this article by Gatys et al. 2016: Image Style Transfer Using Convolutional Neural Networks.

2. Website:

Implemented with Django.

Accomplishments 

selected as the top 10 best projects in my class (among 200+ submissions)