# Supercharging Android Apps With TensorFlow (Google's Open Source Machine Learning Library)

Discussion:

in november 2015, google and open sourced , its latest and greatest machine learning library. this is a big deal for three reasons:

1. Machine Learning expertise: Google is a dominant force in machine learning. Its prominence in search owes a lot to the strides it achieved in machine learning.
2. Scalability: the announcement noted that TensorFlow was initially designed for internal use and that it’s already in production for some live product features.
3. Ability to run on Mobile.

this last reason is the operating reason for this post since we’ll be focusing on android. if you examine the , you’ll find a little directory. i’ll try to shed some light on the android tensorflow example and some of the things going on under the hood.

## A Look of Recognition

the app glances out through your camera and tries to identify the objects it sees. sometimes it does a good job, other times it can’t quite pin down the object, and at times it leads to thought provoking guesses! overall, it feels pretty magical.

The Android example page gives you an idea on how to build the app, and ultimately culminates in producing this APK (I built and uploaded the APK to save you some time since the building process requires installing the Android NDK and Bazel, Google’s build tool).

NOTE: Android 5.0 or later required since the example uses the Camera2开心时时彩计划软件下载 package introduced in Android 5.0.

	adb install -r -g /path/to/apk.apk


## App Structure Walkthrough

the core tensorflow engine is built with c++, but programmers can write their tensorflow software in either c++ or python. the android tensorflow example uses the c++ interface in the following manner:

1. On startup, the app launches an Android activity () which then starts a fragment ()
2. The fragment does some setup to basically start the camera and feed the incoming stream of images to an object it instantiates ()
3. The listener consults the classifier () about each image it gets, and receives the classification and confidence score for each image.

	public native int initializeTensorflow( );

private native String classifyImageBmp(Bitmap bitmap);


The native开心时时彩计划软件下载 keywords in these method signatures indicate that these methods are implemented in native C++ code. Look for them under the “android/jni” directory and true enough, you’ll find

	JNIEXPORT jint JNICALL
TENSORFLOW_METHOD(initializeTensorflow)(...) {
...
}

JNIEXPORT jstring JNICALL
TENSORFLOW_METHOD(classifyImageBmp)(...) {
...
}


(short for Java Native Interface) is a way in which the Java parts of an Android app can communicate with the native C++ parts. So when we call classifyImageBmp(bitmap)开心时时彩计划软件下载 in our Java code, it will actually invoke the C++ function exported in tensorflow_jni.cc and return the value it returns.

a bitmap file cannot directly be sent to tensorflow as input. it has be transformed into an input tensor that we’d send in step #2 in the flow above. a tensor is an n-dimensional array of values, and is the motif tensorflow uses to send data between all of its different parts/operations. this model expect a 3-dimensional array that supplies the red/green/blue value of each pixel in the image. the dimensions are:

1. X-index of the pixel
2. Y-index of the pixel
3. indication of which value this cell holds (0 for red, 1 for green, 2 for blue)

and the value of the cell would be the actual value of r or g or b channel for that pixel.

## The Model

As you read the example’s , you’ll notice that it instructs you to download a zip file containing the TensorFlow model and add it to the assets开心时时彩计划软件下载 directory. This zip file contains two files that are important for us:

1. tensorflow_inception_graph.pb开心时时彩计划软件下载- At 54 MBs unzipped, this file constitutes the majority of the APK size (58 MBs). This is our trained machine learning model and where the magic comes from. It’s a pre-built TensorFlow describing the exact operations needed to compute a classification from input image data. This Graph is serialized and encoded into binary with Google’s so it can be deserialized across different platforms (think of it as a binary-encoded JSON file).

2. imagenet_comp_graph_label_strings.txt- this contains the 1000 classifications that the output of the model corresponds to (e.g. “vending machine”, “water bottle”, “coffee mug”). These classifications are by the ImageNet Large Scale Visual Recognition Challenge which the model was built to compete in.

## The Possibilities

want to learn more about machine learning? consider checking out the . there’s also a good discussion in here: .

want to comment? , .

Written on January 6, 2016