FaceAI Android SDK is a fast and lightweight SDK that allows you to add Facial Expression and Facial Emotion Recognition AI to Android app. It works on the App’s UI itself and does not send or store any of the biometric data to a server.
Released: October 10, 2022
Table of Contents
- How to use EnxWebAIAndroid SDK?
- Initialize FaceAI
- Face Detector
- Face Pose
- Face Age
- Face Gender
- Face Emotion
- Face Features
- Face Arousal Valence
- Face Attention
- Face Wish
- Stop FaceAI
How to use EnxFaceAIAndroid SDK?
Maven
A Maven version is available to integrate EnxFaceAIAndroid
SDK into your application. Modify your App gradle to download the EnableX Android SDK.
- Edit the
build.gradle
for your project and add the following code snippet to theallprojects.repositories
section:
allprojects { repositories { maven { url "https://dl.bintray.com/enablex/maven" } } }
- Modify
build.gradle
for your module and add the following code snippet to thedependencies
section:
implementation 'XYZ'
Permissions
- Define Permissions in
AndroidManifest.xm
l
<uses-permission android:name="android.permission.CAMERA"/> <uses-permission android:name="android.permission.CHANGE_NETWORK_STATE"/> <uses-permission android:name="android.permission.RECORD_AUDIO"/> <uses-permission android:name="android.permission.INTERNET" /> <uses-permission android:name="android.permission.ACCESS_WIFI_STATE"/> <uses-permission android:name="android.permission.ACCESS_NETWORK_STATE"/> <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"/> <uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE"/> <uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS"/> <uses-permission android:name="android.permission.BLUETOOTH"/> <uses-permission android:name="android.permission.BLUETOOTH_ADMIN"/>
Features
- Define Features in
AndroidManifest.xm
l
<uses-feature android:name="android.hardware.camera"/> <uses-feature android:name="android.hardware.camera.autofocus"/> <uses-feature android:glEsVersion="0x00020000" android:required="false"/>
Initialise FaceAI
FaceAI analyses Video Streams within an ongoing EnableX video session. To get started with analysis, you must bind the FaceAI Object with the Room in which Video Session is on.
Clas: EnxFaceAI
Method: public void initFaceAI(Activity activity, JSONObject roomMeta, EnxRoom room, EnxStream stream, EnxFaceAIObserver enxFaceAIObserver)
– To start analysis on given Stream
Parameters:
activity
– Current class instance or activity instanceroomMeta
– NSDictionary Object. It contains information of the connected Room.room
– Instance of the connected Room usingEnxRTCAndroid
native SDKstream
– The Local Stream Object which will be analyzed.enxFaceAIObserver
–EnxFaceAIObserver
Observer is used to return callback
/* After successful room connecction, initiate EnxFaceAI on onRoomConnected Callback method */ EnxFaceAI enxFaceAI = new EnxFaceAI(); enxFaceAI.initFaceAI(this, roomMetaJsonObject, room, localStream, this);
Face Detector
This is to detect how many faces are there in a Video Stream. The Event Listener continuously receives data in JSON as the detector tries to detect faces in the changing video frame.
Class: EnxFaceAI
Method: public void enableFaceDetector(boolean enable)
– To start/stop analysis
Parameters:
enable
: Boolean. Set totrue
to enable/start Face Detector analysis. Set tofalse
to disable/stop analysis
Callback Method:
onFaceDetectorData:
Gets called repeatedly with Face Detection analysis report with JSON Object. JSON Object Reference appended below:
{ faces: Array(n), rects: Array(n), status: string, fullFrameDetection: Boolean, totalFaces: Number, totalFacesChangedFrom: Number | undefined }
faces
: Array. The detected faces in form of ImageData objects (zero or one; or multiple faces, if fullFrameDetection is true)rects
: Array of objects. Describes the bounding boxes (zero or one; or multiple rects, if fullFrameDetection is true)x
: Upper left point x coordinatey
: Upper left point y coordinatewidth
: Width of the bounding boxheight
: Height of the bounding box
status
: String. Its status of the face trackerINIT
: Detector initializing; zero or many faces could be returnedTRACK_OK
: Detector is correctly tracking one face; one face is returnedRECOVERING
: Detector lost a face and attempting to recover and continue tracking; zero faces are returned
faces
: Array. The detected faces in form of ImageData objects (zero or one; or multiple faces, if fullFrameDetection is true)rects
: Array of objects. Describes the bounding boxes (zero or one; or multiple rects, if fullFrameDetection is true)x
: Upper left point x coordinatey
: Upper left point y coordinatewidth
: Width of the bounding boxheight
: Height of the bounding box
status
: String. Its status of the face trackerINIT
: Detector initializing; zero or many faces could be returnedTRACK_OK
: Detector is correctly tracking one face; one face is returnedRECOVERING
: Detector lost a face and attempting to recover and continue tracking; zero faces are returned
fullFrameDetection
: Boolean. It is true when detection was full-frame and multiple faces can be returned, false otherwise.totalFaces
: Number. It represents the total number filtered of faces detected, smoothened over an interval of time. By default, one face is the maximum number. If multi-face is enabled, the maximum is 6. This output is not synchronized with faces and rects arrays, do not use it to count their lengths!totalFacesChangedFrom
: Number. Optional. When there is a significant change in the number of faces, it is defined and represents the previous number of faces. In case no change occurred, it is undefined. This output is not synchronized with faces and rects arrays.
Note: if you ever notice some false positives in the events, i.e. the face is detected as present even if there is no one, you can further filter the results by the confidence property of the elements contained in the rects
array (e.g. rects[0].confidence > 10
)
enxFaceAI.enableFaceDetector(true); // To start analysis enxFaceAI.enableFaceDetector(false); // To stop analysis // callback @Override public void onFaceDetectorData(String type, String value) { }
Face Pose
This is to analyse face rotation and position in a Video Stream. The Event Listener continuously gets data in JSON as FaceAI detects face rotation in the video stream. Face Rotation angle data is represented in terms of radiants as Pitch, Roll and Yaw.
Class: EnxFaceAI
Method: public void onFacePoseData(String type, String value)
– To start/stop analysis
Parameters:
enable
: Boolean. Set totrue
to enable/start Face Pose analysis. Set tofalse
to disable/stop analysis
Callback Method:
onFacePoseData
:
Called repeatedly with Face Rotation & Position analysis report with JSON Object. JSON Object Reference appended below:
{ output: { pose: { pitch: Number, roll: Number, yaw: Number } } }
output
: Face Rotation & Position Reportpose
: Filtered (smoothened) pose rotation angles expressed in radiants aspitch
,roll
andyaw
.

enxFaceAI.enableFacePose(true); // To start analysis enxFaceAI.enableFacePose(false); // To stop analysis // Callback @Override public void onFacePoseData(String type, String value) { }
Notes:
- max and min ranges for rotation angles are currently limited to +- (Pi/2) in radians, corresponding to +- (90°) in degrees, for each of the 3 axes
- the ZERO point is when a face looks straight at the camera
Face Age
This is to analyse and predict face age in a Video Stream. Face Age predicts within an age range. The Event Listener continuously gets data in JSON as FaceAI analyses face age. If the prediction quality is poor, the event is not fired.
Class: EnxFaceAI
Method: public void enableFaceAge(boolean enable)
– To start/stop analysis
Parameters:
enable
: Boolean. Set totrue
to enable/start Face Age analysis. Set tofalse
to disable/stop analysis
Callback Method:
onFaceAgeData
– Gets called repeatedly with Face Age analysis report with JSON Object. JSON Object Reference appended below:
{ output: { age: { -18: Number, 18-35: Number, 35-51: Number, 51-: Number }, numericAge: Number } }
output
: Face Age Analysis Reportage
: Filtered (smoothened) age prediction:-18
: Probability Weightage suggesting less than 18 years old.18-35
: Probability Weightage suggesting between 18 to 35 years old.35-51:
Probability Weightage suggesting between 18 35 years old.51-
: Probability Weightage suggesting equal or greater than 51 years old.
numericAge
: Numeric. Estimated Age
enxFaceAI.enableFaceAge(true); // To start analysis enxFaceAI.enableFaceAge(false); // To stop analysis // Callback @Override public void onFaceAgeData(String type, String value) { }
Note: in case of poor quality of the prediction, by default, the event is not fired (i.e. skipped for that frame).
Face Gender
This is to analyse face gender in a Video Stream. The Event Listener continuously gets data in JSON as FaceAI analyses face gender.
Class: EnxFaceAI
Method: public void enableFaceGender(boolean enable)
– To start/stop analysis
Parameters:
enable
: Boolean. Set totrue
to enable/start Face Gender analysis. Set tofalse
to disable/stop analysis
Callback Method:
onFaceGenderData
– Gets called repeatedly with Face Gender analysis report with JSON Object. JSON Object Reference appended below:
{ output: { gender: { Female: Number, Male: Number }, mostConfident: String } }
output
: Face Gender Reportgender
: Filtered (smoothened) probabilities of the gender prediction:Female
: Probability weightage for gender is femaleMale
: Probability weightage for gender is male
mostConfident
: Gender name of the most likely result if its smoothened probability is above the threshold, otherwise it is undefined.
enxFaceAI.enableFaceGender(true); // To start analysis enxFaceAI.enableFaceGender(false); // To stop analysis // Callback @Override public void onFaceGenderData(String type, String value) { }
Face Emotion
This is to analyse face emotions in Video Stream. It analyses basic 8 emotions in a human face, viz. Angry, Disgust, Fear, Happy, Sad, Surprise, Neutral. It also returns most dominate emotion on face. The Event Listener continuously getss data in JSON as FaceAI analyses face emotion.
Class:EnxFaceAI
Method: public void enableFaceEmotion(boolean enable)
– To start/stop analysis
Parameters:
enable
: Boolean. Set totrue
to enable/start Face Emotion analysis. Set tofalse
to disable/stop analysis
Callback Method:
onFaceEmotionData
– Called repeatedly with Face Gender analysis report with JSON Object. JSON Object Reference appended below:
{ output: { dominantEmotion: String, emotion: { Angry: Number, Disgust: Number, Fear: Number, Happy: Number, Neutral: Number, Sad: Number, Surprise: Number } } }
output
: Face Emotion ReportdominantEmotion
: Name of Dominant Emotion if present, otherwise it is undefined.emotion
: Filtered (smoothened) values of the probability distribution of emotions. The sum of all the probabilities is always 1, each probability in the distribution has a value between 0 and 1.Angry
: Probability for Angry.Disgust
: Probability for Disgust.Fear
: Probability for Fear.Happy
: Probability for Happy.Sad
: Probability for Sad.Surprise
: Probability for Surprise.Neutral
: Probability for Neutral.
enxFaceAI.enableFaceEmotion(true); // To start analysis enxFaceAI.enableFaceEmotion(false); // To stop analysis // Callback @Override public void onFaceEmotionData(String type, String value) { }
Face Features
This is to analyse face features in a Video Stream. The Event Listener continuously gets data in JSON as FaceAI analyses face features.
Class:EnxFaceAI
Method: public void enableFaceFeatures(boolean enable)
– To start/stop analysis
Parameters:
enable
: Boolean. Set totrue
to enable/start Face Features analysis. Set tofalse
to disable/stop analysis
Callback Method:
onFaceFeatureData
– Gets called repeatedly with Face Features analysis report with JSON Object. JSON Object Reference appended below:
{ output: { features: { ArchedEyebrows: Number, Attractive: Number, .... .... } } }
output
: Face Features Reportfeatures
: Filtered (smoothened) probabilities of each face independent feature in range 0.0 – 1.0. The following features are evaluated:- Arched Eyebrows
- Attractive
- Bags Under Eyes
- Bald
- Bangs
- Beard 5 O’Clock Shadow
- Big Lips
- Big Nose
- Black Hair
- Blond Hair
- Brown Hair
- Chubby
- Double Chin
- Earrings
- Eyebrows Bushy
- Eyeglasses
- Goatee
- Gray Hair
- Hat
- Heavy Makeup
- High Cheekbones
- Lipstick
- Mouth Slightly Open
- Mustache
- Narrow Eyes
- Necklace
- Necktie
- No Beard
- Oval Face
- Pale Skin
- Pointy Nose
- Receding Hairline
- Rosy Cheeks
- Sideburns
- Straight Hair
- Wavy Hair
enxFaceAI.enableFaceFeatures(true); // To start analysis enxFaceAI.enableFaceFeatures(false); // To stop analysis // Callback @Override public void onFaceFeatureData(String type, String value) { }
Face Arousal Valence
This is to analyse face arousal valence in a Video Stream. The Event Listener continuously gets data in JSON as FaceAI analyses Face Arousal Valence.
Class:EnxFaceAI
Method: public void enableFaceArousalValence(boolean enable)
– To start/stop analysis
Parameters:
enable
: Boolean. Set totrue
to enable/start Face Arousal Valence analysis. Set tofalse
to disable/stop analysis
Callback Method:
onFaceArousalValenceData
– Gets called repeatedly with Face Arousal Valence analysis report with JSON Object. JSON Object Reference appended below:
{ output: { arousalvalence: { arousal: Number, valence: Number, affects38 : { "Afraid": Number, "Amused": Number, .. }, affects98 : { "Adventurous": Number, "Afraid": Number, .. }, quadrant : String } } }
output
: Face Arousal Valence Reportarousalvalence
: Filtered (smoothened) values.arousal
: Range 1.0 to 1.0. It represents the degree of engagement (positive arousal), or disengagement (negative arousal).valence
: Range -1.0 to 1.0. It represents the degree of pleasantness (positive valence), or unpleasantness (negative valence).
output
: Face Arousal Valence Reportarousalvalence
: Filtered (smoothened) values.arousal
: Range 1.0 to 1.0. It represents the degree of engagement (positive arousal), or disengagement (negative arousal).valence
: Range -1.0 to 1.0. It represents the degree of pleasantness (positive valence), or unpleasantness (negative valence).affects38
: An object containing the smoothened probabilities of the 38 affects in range [0.00, 1.00]: Afraid, Amused, Angry, Annoyed, Uncomfortable, Anxious, Apathetic, Astonished, Bored, Worried, Calm, Conceited, Contemplative, Content, Convinced, Delighted, Depressed, Determined, Disappointed, Discontented, Distressed, Embarrassed, Enraged, Excited, Feel Well, Frustrated, Happy, Hopeful, Impressed, Melancholic, Peaceful, Pensive, Pleased, Relaxed, Sad, Satisfied, Sleepy, Tired

affects98
: An object containing the smoothened probabilities of the 98 affects in range [0.00, 1.00]: Adventurous, Afraid, Alarmed, Ambitious, Amorous, Amused, Wavering, Angry, Annoyed, Anxious, Apathetic, Aroused, Ashamed, Worried, Astonished, At Ease, Attentive, Bellicose, Bitter, Bored, Calm, Compassionate, Conceited, Confident, Conscientious, Contemplative, Contemptuous, Content, Convinced, Courageous, Defient, Dejected, Delighted, Depressed, Desperate, Despondent, Determined, Disappointed, Discontented, Disgusted, Dissatisfied, Distressed, Distrustful, Doubtful, Droopy, Embarrassed, Enraged, Enthusiastic, Envious, Excited, Expectant, Feel Guilt, Feel Well, Feeling Superior, Friendly, Frustrated, Glad, Gloomy, Happy, Hateful, Hesitant, Hopeful, Hostile, Impatient, Impressed, Indignant, Insulted, Interested, Jealous, Joyous, Languid, Light Hearted, Loathing, Longing, Lusting, Melancholic, Miserable, Passionate, Peaceful, Pensive, Pleased, Polite, Relaxed, Reverent, Sad, Satisfied, Selfconfident, Serene, Serious, Sleepy, Solemn, Startled, Suspicious, Taken Aback, Tense, Tired, Triumphant, Uncomfortable

quadrant
: A string representing one of the four quadrants in the cirumplex model of affect (“High Control”, “Obstructive”, “Low Control”, “Conductive”, or “Neutral”)

enxFaceAI.enableFaceArousalValence(true); // To start analysis enxFaceAI.enableFaceArousalValence(false); // To stop analysis // Callback @Override public void onFaceArousalValenceData(String type, String value) { }
Face Attention
This is to analyse face attention in a Video Stream. The Event Listener continuously get data in JSON as FaceAI analyses face attention.
Class:EnxFaceAI
Method: public void enableFaceAttention(boolean enable)
– To start/stop analysis
Parameters:
enable
: Boolean. Set totrue
to enable/start Face Attention analysis. Set tofalse
to disable/stop analysis
Callback Method:
onFaceAttentionData
– Gets called repeatedly with Face Attention analysis report with JSON Object. JSON Object Reference appended below:
{ output: { attention: Number } }
output
: Face Attention Reportattention
: Filtered value (smoothened) in range [0.0, 1.0]. A value close to 1.0 represents attention, a value close to 0.0 represents distraction.
enxFaceAI.enableFaceAttention(true); // To start analysis enxFaceAI.enableFaceAttention(false); // To stop analysis // Callback @Override public void onFaceAttentionData(String type, String value) { }
Face Wish
This is to analyze face wish in a Video Stream. The Event Listener keeps getting data in JSON as FaceAI keeps analyzing face wish.
Class: EnxFaceAI
Method: public void enableFaceWish(boolean enable)
– To start/stop analysis
Parameters:
enable
: Boolean. Set totrue
to enable/start Face Wish analysis. Set tofalse
to disable/stop analysis
Callback Method:
onFaceWishData
– Get called notifies repeatedly with Face Wish analysis report with JSON Object. JSON Object Reference appended below:
{ output: { wish: Number } }
output
: Face Wish Reportwish
: Filtered value (smoothened) in range [0, 1.0]. A value closer to 0 represents a lower wish, a value closer to 1.0 represents a higher wish.
enxFaceAI.enableFaceWish(true); // To start analysis enxFaceAI.enableFaceWish(false);// To stop analysis // Callback @Override public void onFaceWishData(String type, String value) { }