Importing Workflow

✦ Introduction

Aegis Photo Voyager features a high-performance, non-blocking Smart Photo Importer. Unlike cloud-based tools that stream your data to third-party servers, Photo Voyager copies and indexes your photos completely offline, utilizing advanced multi-threaded background workers to maintain a fast, responsive user interface.

This tutorial walks through the technical details of how the import workflow operates under the hood, how to connect physical or digital sources, and how the importer initiates the offline AI enrichment pipeline.

1 Import Sources

The Smart Importer supports three main categories of ingestion sources, accessible as cards in the user interface:

1. External SSDs, HDDs & Local Folders

You can ingest media from external hard drives, SD cards, or any local folder directory. When a folder path is selected:

• The application queries filesystem metadata using get_drive_info(path).
• A unique Drive ID and Drive Name are recorded in your local SQLite database.
• This enables Photo Voyager to remember that specific physical drive and skip scanning files that have already been imported from it in the past.

2. Google Photos & Apple iCloud ZIP Archives

Ideal for migrating large archives out of cloud services (e.g., using Google Takeout or Apple Data Export):

• Select the .zip archive in the importer UI.
• The application spawns a secure, temporary directory prefixed with aegis_import_.
• The ZIP file is extracted, scanned, and media files are safely imported.
• Once the import completes or aborts, the temporary directory is automatically deleted to free up local disk space.

3. iPhone / Mobile Devices (USB)

Direct mobile imports leverage platform-specific connection layers to bridge mobile storage with your PC:

2 Ingest Modes & Filtering

Ingestion in Aegis Photo Voyager is highly customizable. You can configure filters and organizational rules in the dialog before starting an import:

Date Filtering Modes

To avoid reprocessing files that are already indexed in your master library, choose one of these three ingest modes:

Attribution & Curation Options

• Contributor Profiles: Attribute imported photos to a specific profile (e.g. "Clara's Camera", "Dave's iPhone"). This adds search-friendly owner metadata to the photo catalog.
• Curation Staging Queue: Check "Stage in Curation Queue before import" to hold new photos in a temporary database table. You can open the Curation Workspace later to approve, rename, edit details, or discard items before committing them to the main library.

Target File Organization

Photos are copied into the Photos Master Directory (configured in Settings → Sources) and organized in a clean, chronological folder hierarchy:

Photos Master Directory/
└── YEAR/
    └── YEAR_MONTH_DAY/
        └── original_filename.ext

If a file with an identical name already exists in the destination folder, the importer automatically resolves the conflict by appending a counter (e.g., photo_1.jpg, photo_2.jpg) instead of overwriting the file.

3 Background Ingestion Architecture

To ensure that the user interface never blocks or freezes during heavy file operations, imports run inside a separate, fully detached background process:

Execution Flow

1. When you click Start Import, the main GUI serializes all parameters (source path, date filters, profile attributes) into a file named importer_job.json within ~/.AegisPhotoVoyager/.
2. Any old status tracking files (e.g. import_status.json) are deleted.
3. The GUI spawns a detached subprocess:
   # Command executed internally by Pyside6 QProcess python main.py --mode importer
4. The background process begins reading importer_job.json, scanning files, and running copy operations. Because it is completely detached, you can safely close the main application and the import will continue running in the background until completion.

4 Status Monitoring & Liveness

During the import, the detached process periodically writes stats to import_status.json. The main GUI reads this file to update the progress bar, activity label, and ETA calculations.

Status File Payload Structure

The JSON status file follows a strict schema, reporting overall progress as well as statistics on duplicates and items copied:

{
    "name": "importer",
    "pid": 24890,
    "current_status": "running",
    "current_activity": "Importing DSC_0024.JPG",
    "progress_pct": 34,
    "start_time": 1784382900.5,
    "details": {
        "scanned": 120,
        "duplicates": 15,
        "imported": 40,
        "total": 350
    },
    "updated_at": "2026-07-18T14:08:30.123456"
}

Liveness & ETA Algorithms

• Liveness Verification: The GUI uses a polling loop (every 500ms in progress dialogs; 1000ms on the main window) that queries the pid reported in the status JSON. It verifies the process is active using psutil.pid_exists(pid), with a fallback to native signal checking (os.kill(pid, 0)) to avoid reporting "stuck" progress if a background worker unexpectedly crashes.
• Dynamic ETA: Remaining time is computed on-the-fly:
  Rate = Processed Files / Elapsed Time
  ETA (seconds) = (Total Files - Processed Files) / Rate
• Job Cancellation: If you click Cancel, the GUI pulls the background PID from the status file and terminates the task immediately by sending a termination signal (SIGTERM / SIGKILL).
• Metrics Logging: Upon completion or error, the worker updates the status file to finished or error, saves metadata stats into the local SQLite table activity_metrics, and logs details in background_history.jsonl.

5 Starting AI Processing

After your photos have been successfully copied into the master directory, the local AI processing pipeline is triggered to index and enrich the new assets.

Automated Post-Import Sequence

If you did not use the Curation Staging Queue (photos were copied directly into the master library):

1. The importer immediately launches the File Discovery (discoverer) background process to register the files in the catalog.
2. Once registered, the GUI's sequence timer kicks off the sequential processing queue.

The Sequential AI State Machine

To avoid resource contention (e.g. running out of VRAM/RAM or overheating), the pipeline executes stages one by one. If you close the app during this sequence, background workers continue running, and the GUI restores polling hooks next time it starts.

Pre-Flight Validation & Smart Start

Before launching the pipeline manually (via the ▶ play button on the toolbar or the Smart Start onboarding wizard), the application runs several checks to make sure your AI engines are ready:

• LLM Connection Check: Sends a test image request to Ollama / LM Studio to verify it yields valid structured JSON.
• Face Recognition Diagnostic: Validates that required dependencies like face_recognition and model files are present on disk.
• Embedding Model Test: Assures the text embedding model (e.g. nomic-embed-text) is loaded and responding with numerical vector arrays.

✦ Troubleshooting