Skip to main content

metawarc

Bounded indexing, extraction, and analysis for WARC collections

Index WARC collections into a versioned DuckDB catalog with Parquet sidecars, then query, extract, analyze, and replay without copying payloads.

pip install metawarc
metawarc index 'archives/**/*.warc*' --dbfile collection.db --resume

Documentation contents

Start with a section below, or use the sidebar from any page. The CLI entry point is metawarc. Source WARC files stay immutable; the DuckDB catalog and Parquet sidecars are the operational index.