Citation anchors
Point at the sentence in the original file that an answer came from — in the document itself, with its formatting, not in a text dump of it.
Why
An assistant that answers from documents has to be able to show its source, and showing it means the original file with the cited passage highlighted. For PDF that is solved. For Office formats it is not, and the usual workaround makes it impossible: converting to PDF on a server destroys the mapping between what you indexed and what you display.
Apertura is the parser and the renderer, so it can hand out an identifier during extraction and resolve the same identifier during rendering. That is the whole idea, and it is only available to something that owns both ends.
The expected shape
Sketched to show the direction. Names and signatures will change before any of it is released.
import { toBlocks } from '@apertura/extract';
for (const block of toBlocks(document)) {
await store.add({
text: block.text,
// Store it next to the chunk. It is stable across reopenings of the file.
anchor: block.anchor,
});
}Why searching for the text does not work
The obvious alternative is to take the chunk of text and search the rendered document for it. It fails in ordinary cases: the same sentence appears twice, the extractor normalised whitespace the renderer did not, a field expanded differently, a soft hyphen sits in the middle of a word. An identifier issued by the thing that produced both sides has none of those failure modes.
What an anchor promises
- Stable across reopening the same bytes — index once, resolve later.
- Resolvable even when the page it lives on has been virtualised away.
- Not stable across editing the document in Word. A changed file is a new file and needs reindexing; pretending otherwise would silently cite the wrong paragraph.