Building MANTAX: An Ethical Facial Recognition System for NGOs

#Introduction

This is a technical deep-dive into MANTAX, an ethical facial recognition system designed for NGO use cases. In this blogpost, we'll explore the complete architecture, data flows, and implementation details—everything a developer needs to understand how this system works.

---

#System Architecture Overview

MANTAX follows a three-tier architecture with clear separation between presentation, business logic, and machine learning components.

#Technology Stack

| Layer | Technology | Purpose | |-------|------------|---------| | Frontend | Electron + Vanilla JS | Desktop app with custom titlebar | | Styling | SCSS → CSS | macOS Tahoe Liquid Glass design | | Backend | Flask (Python) | REST API with 31 endpoints | | ML Runtime | ONNX Runtime (ArcFace) + PyTorch (FaceNet) | Dual-model embedding extraction | | Face Detection | OpenCV DNN + MediaPipe | Face localization + 468-point landmarks |

---

#The Data Pipeline

When a user uploads an image, it flows through a well-defined pipeline. Let's trace this journey:

---

#Face Detection Module

The detection module (src/detection/__init__.py) handles the first critical step: finding where faces are in an image.

#Detection Flow

#Primary Detection Method: OpenCV DNN

The system uses a pre-trained Caffe model for deep learning-based face detection:

Model files required:

• deploy.prototxt.txt - Caffe architecture definition
• res10_300x300_ssd_iter_140000.caffemodel - Pre-trained weights

#Fallback: Haar Cascade

If DNN fails to load, the system gracefully falls back to Haar Cascade:

#Face ROI Extraction

Once faces are detected, the API extracts the region of interest (ROI):

---

#Facial Landmark Detection

After detecting faces, the system extracts 468 facial landmarks using MediaPipe Face Mesh:

#Landmark Extraction Code

#Geometric Feature Extraction

For comparison purposes, we extract scale-invariant geometric features:

---

#Embedding Extraction (Dual-Model)

This is the core of the system—converting face images into mathematical embeddings that can be compared.

#ArcFace Implementation

ArcFace provides superior discrimination between different faces:

#FaceNet Implementation

FaceNet provides secondary signals and neural activation visualizations:

#Neural Network Activations

FaceNet also extracts intermediate layer activations for visualization:

Activation layers extracted: | Layer | Output Shape | Purpose | |-------|-------------|---------| | conv1 | (64, 112, 112) | First convolutions | | bn1 | (64, 112, 112) | First batch norm | | layer1 | (64, 56, 56) | Low-level features | | layer2 | (128, 28, 28) | Mid-level features | | layer3 | (256, 14, 14) | High-level features | | layer4 | (512, 7, 7) | Final features | | embedding | (128,) | Final embedding |

---

#The Compare Endpoint (Complete Flow)

Here's the full comparison flow from the /api/compare endpoint:

#Dual-Model Scoring

The scoring combines multiple signals with learned weights:

#Confidence Bands

Rather than binary decisions, the system outputs confidence bands:

---

#API Endpoints Reference

The Flask API exposes 31 endpoints for all operations:

| Endpoint | Method | Purpose | |----------|--------|---------| | /api/health | GET | System health check | | /api/embedding-info | GET | Current model info | | /api/diagnostics | GET | System diagnostics | | /api/detect | POST | Face detection | | /api/extract | POST | Embedding extraction | | /api/add-reference | POST | Add reference image | | /api/references | GET | List all references | | /api/references/<id> | DELETE | Remove reference | | /api/compare | POST | Compare embeddings | | /api/visualizations/<type> | GET | Get visualization | | /api/clear | POST | Clear session |

---

#Visualizations (14 Types)

The system provides 14 different AI visualizations to help investigators understand why scores were computed:

#Visualization Implementation Example

---

#Session State Management

The API maintains in-memory session state:

#Persistence

References are saved to JSON for persistence across restarts:

---

#Testing Infrastructure

The system includes comprehensive tests:

#Running Tests

---

#File Structure

---

#Key Design Decisions

#1. Dual-Model Architecture

Using both ArcFace (512-dim) and FaceNet (128-dim) together provides better discrimination than either alone. ArcFace handles the primary matching while FaceNet provides secondary signals and activation visualizations.

#2. Confidence Bands, Not Binary Decisions

The system outputs confidence bands (Very High/High/Moderate/Insufficient) instead of "match/no-match". This ensures human investigators always make the final decision.

#3. Local-Only Processing

No images are sent to external servers. All computation happens on the user's machine, addressing NGO privacy concerns.

#4. Non-Reversible Embeddings

Facial embeddings cannot be used to reconstruct the original face—providing an additional layer of privacy protection.

#5. Consent Tracking

Every reference image includes metadata about consent status, source, and purpose—essential for NGO documentation requirements.

---

#Summary

MANTAX is a fully functional ethical facial recognition system built with:

• Flask API (2,131 lines) with 31 endpoints
• Dual-model embedding (ArcFace 512-dim + FaceNet 128-dim)
• OpenCV DNN face detection with MediaPipe landmarks
• Electron desktop app with macOS Tahoe Liquid Glass UI
• Comprehensive testing (E2E, edge cases, frontend)

The system is designed for NGO use cases with:

• Local-only processing (no cloud)
• Human-in-the-loop verification
• Consent tracking
• Confidence bands instead of binary decisions

In the next blogpost, we'll explore the JavaScript refactoring journey—how we tackled a 3,429-line monolithic app.js and broke it into 7 modular files following best practices.

---

#Demo Video

Here's a demo showing a no-match scenario:

Next: The JavaScript Refactoring Story