Fingerprint

A fingerprint (hash) is 64 characters calculated from the content of a file. If even one character of the file changes, the fingerprint becomes completely different. The content cannot be worked out from the fingerprint alone.

---

1. How it is taken

• SHA-256 (US standard FIPS 180-4). Written as 64 lowercase hexadecimal characters.
• It is taken over the bytes of the original file exactly as they are. The file name, timestamps and folder are not part of the fingerprint.
• Even after the original is locked with a key, the fingerprint is that of the original before locking. The fingerprint of the locked file is kept separately as the “stored fingerprint” (section 4).

2. It is taken twice

When	Where	manifest field
The moment the recording ends	The device (phone, scanner, camera)	sha256_device
On arrival at the store	The store (immediately on receipt)	sha256_store

• If the two differ, the file is not stored and the depositor is told. Something changed on the way.
• If the two match, this fingerprint becomes the record’s original fingerprint.
• If the device locks the file before sending it (09-Keys section 3), the store cannot see the original. The device also sends the fingerprint of the locked file, and the store checks it as the stored fingerprint. The original fingerprint (sha256_store) is the value the device took, copied as it is.
• When it cannot be taken on a device (for example, when receiving a bundle of files already scanned at the Korean Studies Advancement Center), sha256_device is left empty, and the fingerprint of the file exactly as received is used as the first fingerprint. The fingerprint of the bundle file (zip) is also kept, so it can be traced back to the bundle it came from. This is how arkive does it now.

secretary v0 calls these fields sha256_phone and sha256_server. They are the same values (03-manifest section 6).

3. Regular checks

• The store takes the fingerprints of the files it holds again once a day and compares them.
• The results are kept as records that are never deleted. If they match, “checked”; if they differ or the file is missing, “changed” or “missing”, with the time.
• If a change is found, the depositor is told, and a file with the same fingerprint is found in the backups and restored.
• At present arkive checks only on request (/api/fixity), and secretary has no regular check yet.

4. Original fingerprint and stored fingerprint

Once the original is locked with a key, the administrator cannot see it, so it also cannot take the original fingerprint again. So the fingerprint is split in two.

Name	Fingerprint of what	Who checks it
original fingerprint (sha256_store)	The original before locking	The depositor (unlocking it on their own device)
stored fingerprint (sha256_stored)	The locked file	The administrator (regular checks)

While nothing is locked (secretary v0, arkive today), the two are the same, so only the original fingerprint is kept.

5. Face → volume Merkle tree

An old document or a book has many faces. A fingerprint is taken for each face, and those fingerprints are paired and combined upward until one volume fingerprint remains at the top. This is called a Merkle tree.

           volume fingerprint
               /        \
           a+b            c+d
         /     \         /     \
     face 1  face 2   face 3  face 4
     fprint  fprint   fprint  fprint

• If even one face changes, the volume fingerprint changes, and it is possible to point out which face changed.
• It is possible to prove that a face is in that volume without showing the other faces. Only a few fingerprints on the path upward need to be handed over (inclusion proof).

What is set:

• How to combine: use the method of section 2.1 of RFC 6962, the Certificate Transparency standard, as it is (RFC 9162 has the same definition). A leaf is SHA-256(0x00 ‖ original fingerprint 32 bytes), a branch is SHA-256(0x01 ‖ left ‖ right). When the count is odd, the last item is not duplicated to make a pair. Duplicating would open a hole where different lists have the same root.
• A leaf is an original file. In most cases one file is one face. When one scan contains two faces, as with one of arkive’s old documents, that scan file is one leaf, and the two faces are referred to as regions inside that file (03-manifest section 4).
• The order is the order of faces in the volume (IIIF manifest order). If the same file is used for several faces, it is included only once, at the place where it first appears.
• If faces are added or their order changes, a new volume fingerprint is made and the old one is kept. The version number goes up.

For recordings, one file is currently one leaf. The September HGP specification proposes splitting a recording into pieces each time the speaker changes and taking separate fingerprints (so that each passage can be sealed). This will be decided again when recordings are stored in pieces.

6. Daily bundle fingerprint

A single root that combines, by the same method (section 5), the original fingerprints of the records made during one day.

• Combined separately for each depositor. Not mixed with other people’s records.
• A day is the local date when the recording started, with midnight as the boundary (same as secretary). cheroki uses 4 a.m. as the boundary, so it differs.
• Order is by start time, and if equal, by record number (id).
• Closing time is 48 hours after the day ends. If a record arrives late because a phone was offline for a long time, that day’s daily fingerprint is made again as a new version and the old version is kept.
• For Nodeone as a whole, the daily fingerprints of each depositor closed that day are combined again by the same method into one node daily fingerprint.

Anchoring: following the September HGP specification, only one line, the node daily fingerprint, is anchored outside. There are two places: one that lasts a long time (Bitcoin timestamp proof, OpenTimestamps) and one that can be checked immediately (a public ledger). Only the root and the time are anchored; no content, names or individual fingerprints go out. This is not done now. For now, the work goes as far as making the daily fingerprint, and anchoring will be done in secretary v2.

Public records one by one (registry entries, decision 0016): a record whose scope is “public” may be put on a public ledger one by one. This is not anchoring but the registry of HGP-0005 section 10 in the September specification. Only three things go up: the version number, the fingerprint of the whole record (the SHA-256 of bytes written by RFC 8785), and the registry entry number of the previous record. All other fields stay inside the record. A record that is not public must not even reveal that it exists, so it goes only into the daily root. When a record holds fingerprints of private values, they are mixed with 32 random bytes (salt). Values with few possibilities (temperature, dates) can be guessed without salt. For now, provenance (physical product provenance) uses this.

7. What a fingerprint proves and what it cannot

It proves	It cannot prove
The file has not changed since the fingerprint was first taken	That the content is true
If anchored outside, that it already existed at that time	Who made it (a signature has to do that, section 8)
That a face is in that volume, and a record is in that day	That the person consented (a pledge has to do that)

What HGP proves is acts, not truth.

8. Signatures

A fingerprint alone cannot prove “who”. The fingerprint has to be signed with the depositor’s key. Depositor signatures use Ed25519 (RFC 8032) (decision 0018). The signing key is derived from the depositor’s owner key and is used only inside the device. What is signed: the original fingerprint, the record number and the time. When the key changes (recovery, succession), the new public key and the reason are kept as a line that is never deleted (09-Keys section 5). The C2PA approach (X.509 certificates) will be looked at again when an institution is the depositor. From secretary v0.2 (11/16).

Signatures on registry entries (decision 0016): a registry entry put on a public ledger is signed with the store operator’s wallet key (secp256k1). This signature says “the store put this record up at this time”; it is not the signature of the depositor or the maker. A person looks at a preview and approves each one, so it is not a machine signature, but the key is on the server, so we state that this is a promise, not a lock (01-Permissions section 1, items 3 and 4).

9. Mapping to IIIF

IIIF (Presentation 3.0), the international digital archive standard, models a face as a Canvas and a volume as a Manifest. An HGP face fingerprint corresponds to one canvas, and a volume fingerprint to one manifest. So if the IIIF manifests that arkive makes now carry the original fingerprint of each canvas and the volume fingerprint of the manifest, any viewer in the world that reads IIIF can check whether “this face is that original face of that volume”. IIIF has no separate fingerprint field. Attaching our own fields directly to the manifest would fail the official IIIF validator, so the values are carried in two places IIIF 3.0 provided from the start (Decision 0009).

• Values for people to read go in metadata as text. The original fingerprint on each canvas, the volume fingerprint on the manifest.
• Values for machines to read are published separately as a “fixity bundle” file, pointed to by the manifest’s seeAlso. type: "Dataset", format: "application/json", profile: "https://hgp.hahoe.io/ns/#fixity-v1". The bundle records manifest, hgp:root (volume fingerprint), hgp:root_version, method (how the volume fingerprint was made), and for each canvas, canvas, hgp:sha256 (original fingerprint) and derivation (crop region, rotation, reduced size).
• The profile name is https://hgp.hahoe.io/ns/#fixity-v1 (08-Name file section 4, Decision 0014). It replaces the earlier provisional name urn:x-hgp:fixity.
• Now that the public name file address exists, hgp:sha256 may be attached directly to canvases. Put https://hgp.hahoe.io/ns/v1.jsonld in @context before the IIIF address. Attaching hgp:root to the manifest is still blocked by the validator (08-Name file section 3).

There is one thing to watch. What an arkive canvas draws on screen is not the original scan but a reduced copy (1600 pixels), so the fingerprint attached to the canvas is the fingerprint of the original scan, not of that copy, and the copy carries “which original it came from and how it was cropped and reduced”. IIIF 3.0 also allows canvases with a length (duration), so a single recording can be carried as one canvas, and one day as one manifest, in the same way.