Embedding Metadata

In plain terms: most models give you back just a vector. The face models give you a little extra for each face they find — where it is in the photo (a bounding box), how confident they are, and optionally an age/gender guess. You get this automatically, no second call needed. Again, skip this unless you're working with faces.

Starting from version 0.2.2, face detection models (Buffalo_L and SFace+YuNet) return bounding box metadata alongside embeddings. This allows you to access face location and confidence information without re-running detection.

Metadata Fields (Face Detection Models)

For each detected face, the following metadata is automatically included:

Field	Type	Range	Description
bbox_x1	float	0.0–1.0	Normalized x-coordinate of top-left corner
bbox_y1	float	0.0–1.0	Normalized y-coordinate of top-left corner
bbox_x2	float	0.0–1.0	Normalized x-coordinate of bottom-right corner
bbox_y2	float	0.0–1.0	Normalized y-coordinate of bottom-right corner
confidence	float	0.0–1.0	Detection confidence score

Buffalo_L only — the following fields are included when attributes=genderage is specified:

Field	Type	Range	Description
gender_female_prob	float	0.0–1.0	Probability of female gender
gender_male_prob	float	0.0–1.0	Probability of male gender
age	float	0.0–100.0	Predicted age in years

Coordinates are normalized to the 0-1 range, making them independent of the original image resolution. To convert to pixel coordinates, multiply by the image width/height:

pixel_x1 = bbox_x1 * image_width
pixel_y1 = bbox_y1 * image_height

Metadata Storage

When you insert images using face detection models:

• Embeddings are stored in Ahnlich DB as usual
• Metadata (bounding boxes, confidence) is merged into the StoreValue for each face
• Metadata is returned in GetSimN, GetPred, and ConvertStoreInputToEmbeddings responses

API Response Structure

The ConvertStoreInputToEmbeddings API returns EmbeddingWithMetadata for face models:

Proto

message EmbeddingWithMetadata {
  keyval.StoreKey embedding = 1;           // The face embedding vector
  optional keyval.StoreValue metadata = 2; // Bounding box + confidence
}

For OneToMany models (face detection), multiple EmbeddingWithMetadata objects are returned—one per detected face.

Usage Examples

Rust — accessing bounding box metadata:

Rust

use ahnlich_client_rs::prelude::*;

let response = client.convert_to_embeddings(
    store_name,
    vec![StoreInput::Image(image_bytes)],
    PreprocessAction::ModelPreprocessing,
    None,
    HashMap::new(),
).await?;

// For face detection models, variant is OneToMany
if let Some(Variant::Multiple(multi)) = &response.values[0].variant {
    for face in &multi.embeddings {
        if let Some(embedding) = &face.embedding {
            println!("Embedding dimensions: {}", embedding.key.len());
        }
        
        if let Some(metadata) = &face.metadata {
            let bbox_x1 = metadata.value.get("bbox_x1").unwrap();
            let bbox_y1 = metadata.value.get("bbox_y1").unwrap();
            let confidence = metadata.value.get("confidence").unwrap();
            
            println!("Face at ({}, {}) with confidence {}", 
                     bbox_x1, bbox_y1, confidence);
        }
    }
}

Python — accessing bounding box metadata:

Python

from ahnlich_client_py import AhnlichAIClient

response = await client.convert_store_input_to_embeddings(
    store="faces_store",
    inputs=[image_bytes],
    preprocess_action=PreprocessAction.ModelPreprocessing,
)

# Each face has embedding + metadata
for face_data in response.values[0].multiple.embeddings:
    embedding = face_data.embedding.key  # 512-dim vector for Buffalo_L
    metadata = face_data.metadata.value
    
    bbox_x1 = float(metadata["bbox_x1"].value)
    bbox_y1 = float(metadata["bbox_y1"].value)
    confidence = float(metadata["confidence"].value)
    
    print(f"Face at ({bbox_x1}, {bbox_y1}) with confidence {confidence}")

TypeScript — accessing bounding box metadata:

TypeScript

import { AhnlichAIClient } from '@deven96/ahnlich-client-node';

const response = await client.convertStoreInputToEmbeddings({
  store: "faces_store",
  inputs: [{ image: imageBytes }],
  preprocessAction: PreprocessAction.MODEL_PREPROCESSING,
});

// Each detected face has embedding + metadata
for (const faceData of response.values[0].multiple.embeddings) {
  const embedding = faceData.embedding.key; // Float32Array
  const metadata = faceData.metadata.value;
  
  const bboxX1 = parseFloat(metadata.bbox_x1.value);
  const bboxY1 = parseFloat(metadata.bbox_y1.value);
  const confidence = parseFloat(metadata.confidence.value);
  
  console.log(`Face at (${bboxX1}, ${bboxY1}) with confidence ${confidence}`);
}

Gender and Age Predictions (Buffalo_L)

Buffalo_L can compute age and gender predictions for each detected face by setting attributes=genderage in model_params. This adds three additional metadata fields per face: gender_female_prob, gender_male_prob, and age.

Rust — enabling gender and age predictions:

Rust

use std::collections::HashMap;
use ahnlich_client_rs::prelude::*;

let mut model_params = HashMap::new();
model_params.insert("attributes".to_string(), "genderage".to_string());

let response = client.convert_to_embeddings(
    store_name,
    vec![StoreInput::Image(image_bytes)],
    PreprocessAction::ModelPreprocessing,
    None,
    model_params,
).await?;

// Access gender/age metadata
if let Some(Variant::Multiple(multi)) = &response.values[0].variant {
    for face in &multi.embeddings {
        if let Some(metadata) = &face.metadata {
            let female_prob = metadata.value.get("gender_female_prob").unwrap();
            let male_prob = metadata.value.get("gender_male_prob").unwrap();
            let age = metadata.value.get("age").unwrap();
            
            println!("Age: {}, Female: {}, Male: {}", age, female_prob, male_prob);
        }
    }
}

Python — enabling gender and age predictions:

Python

from ahnlich_client_py import AhnlichAIClient

response = await client.convert_store_input_to_embeddings(
    store="faces_store",
    inputs=[image_bytes],
    preprocess_action=PreprocessAction.ModelPreprocessing,
    model_params={"attributes": "genderage"}
)

# Access gender/age metadata
for face_data in response.values[0].multiple.embeddings:
    metadata = face_data.metadata.value
    
    female_prob = float(metadata["gender_female_prob"].value)
    male_prob = float(metadata["gender_male_prob"].value)
    age = float(metadata["age"].value)
    
    print(f"Age: {age}, Female: {female_prob}, Male: {male_prob}")

TypeScript — enabling gender and age predictions:

TypeScript

import { AhnlichAIClient } from '@deven96/ahnlich-client-node';

const response = await client.convertStoreInputToEmbeddings({
  store: "faces_store",
  inputs: [{ image: imageBytes }],
  preprocessAction: PreprocessAction.MODEL_PREPROCESSING,
  modelParams: { attributes: "genderage" }
});

// Access gender/age metadata
for (const faceData of response.values[0].multiple.embeddings) {
  const metadata = faceData.metadata.value;
  
  const femaleProb = parseFloat(metadata.gender_female_prob.value);
  const maleProb = parseFloat(metadata.gender_male_prob.value);
  const age = parseFloat(metadata.age.value);
  
  console.log(`Age: ${age}, Female: ${femaleProb}, Male: ${maleProb}`);
}

Use Cases for Metadata

• Face cropping: Use bounding boxes to extract face regions from original images
• Visualization: Draw bounding boxes on images to show detected faces
• Quality filtering: Filter results by confidence score (e.g., only faces with confidence > 0.8)
• Spatial queries: Find faces in specific image regions (e.g., "faces in the top-left quadrant")
• Deduplication: Identify overlapping detections using bounding box coordinates
• Demographic analysis (Buffalo_L with attributes=genderage):
  • Age-based filtering (e.g., "find faces that appear under 18")
  • Gender distribution analysis in group photos
  • Age group clustering (children, adults, elderly)
  • Demographic insights for audience analysis

Models Without Metadata