AIception faces + asteroids

Intro

We, human beings, take a lot of pictures of our surroundings. A lot them contain human faces (strange, right?).

An interesting task (and a prerequisite for others) is to be able to find the human faces in images. This is called face detection.

AIception has a public endpoint https://aiception.com/api/v2.1/face to help us find faces in images and we will use it.

Parts:

  1. Call the AIception face endpoint
  2. Draw bounding boxes for the detected faces
  3. Crop a face
  4. Image of friends + asteroids game = faceteroids

All github code is here:

https://github.com/Razvy000/blog_aiception_faces

1. Call the AIception face endpoint

Create the following folder structure.

/aiception
   /js
       aiception.js
   index1.html

Download aiception.js from [here].

Add the following to index1.html:

<!DOCTYPE html>
<html>

<head>
    <title>HTML5 + AIception faces API = faceteroids</title>
</head>

<body>
    <script src="https://ajax.googleapis.com/ajax/libs/jquery/3.1.1/jquery.min.js"></script>
    <script src="js/aiception.js"></script>

    <script type="text/javascript">
    
    // call AIception API here

    </script>
</body>

</html>

We are now ready to call the AIception API and see what it returns! All we need is the token and an image_url.

<script type="text/javascript">
    // call AIception API here
    var token = "YOUR_TOKEN_HERE"

    var image_url = "https://upload.wikimedia.org/wikipedia/en/8/86/Einstein_tongue.jpg"
    faces(token, image_url, function(error, success) {
        if (error) {
            console.log(error);
        } else {
            console.log(JSON.stringify(success, null, 2));
        }
    });
</script>

Now, open index1.html and check your chrome console (CTRL+SHIFT+i).

Ta da:

{
  "answer": {
    "faces": [
      [         // heey we detected a face!
        12,
        64,
        167,
        219
      ]
    ]
  },
  "created_at": "Tue, 14 Feb 2017 09:12:10 GMT",
  "image_url": "https://upload.wikimedia.org/wikipedia/en/8/86/Einstein_tongue.jpg",
  "state": "done",
  "this_url": "https://aiception.com/api/v2.1/face/328",
  "type": "face"
}

If everythings looks similar to above congratulations! You made your first API call to AIception.

The answer is what we are looking for. The JSON response has a “faces” key with a list of faces. Each face is a list of coordinates.

If we use another image_url with 2 faces:

var image_url = "https://upload.wikimedia.org/wikipedia/commons/thumb/e/e2/Albert_Einstein_and_Charlie_Chaplin_-_1931.jpg/800px-Albert_Einstein_and_Charlie_Chaplin_-_1931.jpg"

We will get:

{
  "answer": {
    "faces": [
      [          // this is the first face detected
        531,
        201,
        716,
        387
      ],
      [          // this is the second face detected
        142,
        93,
        365,
        316
      ]
    ]
  },
  "created_at": "Tue, 14 Feb 2017 09:17:53 GMT",
  "image_url": "https://upload.wikimedia.org/wikipedia/commons/thumb/e/e2/Albert_Einstein_a…ie_Chaplin_-_1931.jpg/800px-Albert_Einstein_and_Charlie_Chaplin_-_1931.jpg",
  "state": "done",
  "this_url": "https://aiception.com/api/v2.1/face/329",
  "type": "face"
}

Just the coordinates makes little sense so let’s actually see what they mean.

2. Draw bounding boxes for the detected faces

What do those coordinates mean? Each list of 4 coordinates is actually a bounding box x1, y1, x2, y2 of the detected face.

Let’s add a canvas element “myCanvas” and begin our processing when the document is ready:

<!DOCTYPE html>
<html>
<head>
    <title>HTML5 + AIception faces API = faceteroids</title>
</head>
<body>

<canvas id="myCanvas" width="100" height="100" ></canvas>


<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.1.1/jquery.min.js"></script>
<script src="js/aiception.js"></script>

<script type="text/javascript">

    $(document).ready(function() {
        // run when ready
    }); // end ready    
</script>

</body>
</html>

Create a function that can draw green rectangles.

        var canvas = document.getElementById('myCanvas');
        var context = canvas.getContext('2d');

        var drawRect = function(x, y, width, height){
            context.beginPath();
            context.rect(x, y, width, height);
            context.lineWidth = 8;
            context.strokeStyle = 'green';
            context.stroke();
        }

Create a function that will call the API and draw each rectangle from the result.

        var drawFaces = function(img){
            faces(token, image_url, function(error, success) {
                if (error) {
                    console.log(error);
                } else {
                    console.log(JSON.stringify(success, null, 2));

                    for(faceid in success.answer.faces){
                            face = success.answer.faces[faceid] 
                            drawRect(face[0], face[1], face[2]-face[0], face[3]-face[1])
                    }             
                }
            });
        }

Finally load the image and call drawFaces.

        var loadImage = function(){
            var img = new Image;
            img.onload = function() {
                canvas.height = img.height;
                canvas.width = img.width;
                context.clearRect(0, 0, canvas.width, canvas.height);
                context.drawImage(this, 0, 0);
            };
            img.src = image_url;

            drawFaces(img);   
        }
        
        loadImage();

 

Putting everything together this is how index2.html looks.

<!DOCTYPE html>
<html>
<head>
    <title>HTML5 + AIception faces API = faceteroids</title>
</head>
<body>

<canvas id="myCanvas" width="100" height="100" ></canvas>

<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.1.1/jquery.min.js"></script>
<script src="js/aiception.js"></script>

<script type="text/javascript">

    $(document).ready(function() {

        var token = "eyJhbGciOiJIUzI1NiJ9.eyJpZCI6MywidGltZSI6MTQ4MDk2Nzk5MS42NjIzMTR9.jFJhqMT4969s-_V8TArPFHi5b0da6u4p_RcQA9W0OGI"
        
        var image_url = "https://upload.wikimedia.org/wikipedia/commons/thumb/e/e2/Albert_Einstein_and_Charlie_Chaplin_-_1931.jpg/800px-Albert_Einstein_and_Charlie_Chaplin_-_1931.jpg"

        var canvas = document.getElementById('myCanvas');
        var context = canvas.getContext('2d');

        var drawRect = function(x, y, width, height){
            context.beginPath();
            context.rect(x, y, width, height);
            context.lineWidth = 8;
            context.strokeStyle = 'green';
            context.stroke();
        }

        var drawFaces = function(img){
            faces(token, image_url, function(error, success) {
                if (error) {
                    console.log(error);
                } else {
                    console.log(JSON.stringify(success, null, 2));

                    for(faceid in success.answer.faces){
                            face = success.answer.faces[faceid] 
                            drawRect(face[0], face[1], face[2]-face[0], face[3]-face[1])
                    }             
                }
            });
        }

        var loadImage = function(){
            var img = new Image;
            img.onload = function() {
                canvas.height = img.height;
                canvas.width = img.width;
                context.clearRect(0, 0, canvas.width, canvas.height);
                context.drawImage(this, 0, 0);
            };
            img.src = image_url;

            drawFaces(img);   
        }
        
        loadImage();       

    }); // end ready    
</script>

</body>
</html>

Cool! Try a few different images of you or your friends.

Now, let’s say you want to do something more advanced like cropping Brad’s Pitt/Angelina Jolie face and putting it into your own image. Can you do it automatically?

3. Crop a face

Create index3.html. And use the code from index2.html.

We want to create a new canvas and draw only the extracted face.

Create another canvas.

<canvas id="myCanvas" width="100" height="100" ></canvas>
<canvas id="myCanvas2" width="100" height="100" ></canvas>

Reference it.

        var canvas = document.getElementById('myCanvas');
        var context = canvas.getContext('2d');

        var canvas2 = document.getElementById('myCanvas2');
        var context2 = canvas2.getContext('2d');

Let’s extra the first face we find.

                        if(faceid == 0){
                            src_width =  face[2]-face[0]
                            src_height =face[3]-face[1]
                            canvas2.height = img.height;
                            canvas2.width = img.width;
                            context2.drawImage(img, face[0], face[1], src_width, src_height, 0, 0, src_width, src_height);
                        }

context.drawImage seems daunting at first but here is a great reference https://developer.mozilla.org/en/docs/Web/API/CanvasRenderingContext2D/drawImage

Putting everything together.

<!DOCTYPE html>
<html>

<head>
    <title>HTML5 + AIception faces API = faceteroids</title>
</head>

<body>

    <canvas height="100" id="myCanvas" width="100"></canvas>
    <canvas height="100" id="myCanvas2" width="100"></canvas>

    <script src="https://ajax.googleapis.com/ajax/libs/jquery/3.1.1/jquery.min.js"></script>
    <script src="js/aiception.js"></script>
    
    <script type="text/javascript">
    $(document).ready(function() {

        var token = "YOUR_TOKEN_HERE"

        var image_url = "https://upload.wikimedia.org/wikipedia/commons/thumb/e/e2/Albert_Einstein_and_Charlie_Chaplin_-_1931.jpg/800px-Albert_Einstein_and_Charlie_Chaplin_-_1931.jpg"

        var canvas = document.getElementById('myCanvas');
        var context = canvas.getContext('2d');

        var canvas2 = document.getElementById('myCanvas2');
        var context2 = canvas2.getContext('2d');

        var drawRect = function(x, y, width, height) {
            context.beginPath();
            context.rect(x, y, width, height);
            context.lineWidth = 8;
            context.strokeStyle = 'green';
            context.stroke();
        }

        var drawFaces = function(img) {
            faces(token, image_url, function(error, success) {
                if (error) {
                    console.log(error);
                } else {
                    console.log(JSON.stringify(success, null, 2));

                    for (faceid in success.answer.faces) {
                        face = success.answer.faces[faceid]
                        drawRect(face[0], face[1], face[2] - face[0], face[3] - face[1])
                        if (faceid == 0) {
                            src_width = face[2] - face[0]
                            src_height = face[3] - face[1]
                            canvas2.height = img.height;
                            canvas2.width = img.width;
                            context2.drawImage(img, face[0], face[1], src_width, src_height, 0, 0, src_width, src_height);
                        }
                    }
                }
            });
        }

        var loadImage = function() {
            var img = new Image;
            img.onload = function() {
                canvas.height = img.height;
                canvas.width = img.width;
                context.clearRect(0, 0, canvas.width, canvas.height);
                context.drawImage(this, 0, 0);
            };
            img.src = image_url;

            drawFaces(img);
        }

        loadImage();

    }); // end ready
    </script>
</body>

</html>

Nice! Now you know how to extract the head. Want can you do with it?

Well we had a crazy idea that combines all the tricks we learned so far and we will share it with you.

 

4. Image of friends + asteroids game = faceteroids maybe?

Play it <here> (arrow keys, spacebar)

How we did it?

We took a very cleanly written html5 javascript canvas asteroids game and we added the AIception face detection to extract the faces and make them into asteroids!

Here is the AIception fork and the commit to make it all possible.

  • we hijacked the main loop
  • we added code for the extraction and drawing of the face

 

What’s next?

Here are some ideas to get you started with your own with AIception API:

  • extend 2. draw bounding boxes to allow the user to select a custom image_url
  • make an android / ios app that swaps faces
  • jigsaw puzzle using faces

Publish your own crazy idea that uses AIception API, if we love it we will share it on this blog as well!

 

Leave a Reply

Your email address will not be published. Required fields are marked *