Skip to content

How libphash works

Every hash in libphash comes out of the same short path: an image is loaded into a context, the context prepares it, and a hash function reads it. This page is a map of that path: what runs at each step, which settings reach it, which error codes it can return, and what the context keeps between calls. Each step has a page of its own, linked from here; this one says how the steps fit together.

The path of an image

flowchart TD
    subgraph load ["ph_load_from_…()"]
        direction LR
        S[File, buffer<br/>or pixels] --> D[Decoder] --> T[Transparency] --> O[Orientation]
    end
    subgraph prep ["in ph_compute_…(), before seven of the nine hashes"]
        direction LR
        G[Grayscale] --> R[Reduction]
    end
    load --> C[(Context)]
    C --> prep
    prep --> H[Hash]
    C -->|ColorHash,<br/>ColorMoments| H

The load steps run once, inside a ph_load_from_*() call, and leave the image in the context. The hash steps run inside each ph_compute_*() call, on that image. Seven of the nine algorithms go through grayscale and reduction; ColorHash and ColorMoments read the color image directly, the arrow from the context straight to the hash. Comparing two hashes takes no context at all.

Step Runs in Settings that reach it Codes it can return
Reading the source the load — PH_ERR_INVALID_ARGUMENT, PH_ERR_IO
Decoding the load max_pixels, decode_scale, load_grayscale PH_ERR_UNSUPPORTED_FORMAT, PH_ERR_DECODER_UNAVAILABLE, PH_ERR_IMAGE_TOO_LARGE, PH_ERR_CORRUPT_DATA, PH_ERR_ALLOCATION_FAILED
Transparency the load alpha_mode —
Orientation the load, from a file or a buffer auto_orient PH_ERR_ALLOCATION_FAILED
Grayscale the first hash that needs it, then kept gray weights PH_ERR_ALLOCATION_FAILED
Reduction each hash the algorithm's parameters PH_ERR_ALLOCATION_FAILED
The hash ph_compute_*() the algorithm's parameters PH_ERR_INVALID_ARGUMENT, PH_ERR_EMPTY_IMAGE, PH_ERR_REQUIRES_COLOR
Comparing ph_hamming_distance() and the others — a digest of the wrong kind is refused

Configuring a context lists every setting with its range and default, and says when each is read: a load setting by the next load, a hash setting by the next hash. Handling errors says what each code means and what to do about it, and which step of a load returns it.

ph_load_from_pixels() enters the path at transparency: it takes pixels that are already decoded, so there is no source to read, no decoder and no orientation tag (Loading images).

What a context holds

A ph_context_t is created by ph_create(), freed by ph_free(), and holds four things between the two:

  • The loaded image, at most one: its pixels after the load steps, and what the hashes derive from it — the grayscale image and the area sums several hashes reduce from. The first hash that needs them computes them, and the next hash of the same image reuses them. The next load replaces all of it (One context, many images).
  • The settings, thirteen of them, each starting at its default. A load or a hash reads them; nothing else changes them but the setters.
  • Scratch memory for the hashes' working buffers, grown on demand and kept for the next hash, so hashing a stream of images of similar size does not allocate it again. It is released by ph_free().
  • The detail of the last failed load, the text ph_get_last_error_message() returns (Handling errors).

Because of the first three, a context is the unit of reuse: loading the next image into the same context keeps its settings and its memory. Because of all four, it is also the unit of threading. One context is used by one thread at a time, and two threads with a context each need no lock between them. ph_hash_files() follows the same rule inside: each of its workers creates a context of its own, configured as a copy of a template (Hashing many files). One image is always loaded and hashed on one thread (One image, one thread).

Which decoder reads a file

The first bytes of the data choose the decoder, never the file name. The decoders a build has are fixed when it is compiled, and are tried in order: the native decoders first, libjpeg-turbo, libpng and libwebp, each for its own format, then stb_image, which is in every build and takes whatever none of them claimed. That is why the two builds read different sets of formats:

  • The Full build decodes JPEG, PNG and WebP with the native decoders, and BMP, GIF, TGA, PSD, HDR, PIC and PNM with stb_image.
  • The Minimal build has only stb_image, which also reads JPEG and PNG, but not WebP: a WebP file is refused with PH_ERR_DECODER_UNAVAILABLE rather than as an unknown format.

TIFF is read by neither. The table of formats per build, and what happens to an animated GIF or WebP, are on Loading images; which build to install is on Installing. A program asks the build it runs on with ph_can_use_jpeg(), ph_can_use_png() and ph_can_use_webp(), or logs every decoder at once with ph_get_build_info().

Where to read on

To Read
load images, untrusted ones included Loading images
give contexts the same settings Configuring a context
act on a failure Handling errors
hash many files on several threads Hashing many files
keep hashes and find near duplicates Storing and searching hashes
make hashing cheaper Performance
know what a step does to a hash Image preparation
see where each part lives in the source Development: source map