Waybackpy Guide: Check, List and Save Wayback Captures in Python

Waybackpy is a free Python library and command-line tool for working with the Wayback Machine’s public APIs. With it you can find the oldest, newest or closest archived capture of a URL, list every capture through the CDX index, list the known URLs of a domain, and ask the archive to save a fresh capture. It is a convenient wrapper around the archive’s own endpoints. What it does not do is download and rebuild a whole website for you, so for a working copy of an archived site you combine it with a downloader or a cleanup step.

This guide covers installation, the most useful waybackpy features with short examples, sensible limits for larger jobs, and how waybackpy fits next to other tools. If you need a finished copy of an archived site rather than code, our Wayback website download service delivers clean HTML files.

What you can do with waybackpy

Goal	Waybackpy feature
Check whether a URL was archived and when	Availability API: oldest, newest, near a date
List all captures of a URL with dates and status codes	CDX server API
Find every archived address on a domain	Known URLs listing
Ask the archive to capture a page now	Save API
Download and clean a full website	Not included; use a downloader

How to install waybackpy

You need Python 3 and pip. In a terminal, run:

pip install waybackpy

Then confirm the command-line tool is available and see the options for your installed version:

waybackpy --help

The library has changed over its major versions, and older tutorials online may use class and method names that no longer exist. The examples below follow the current class-based interface; if something fails, compare it with the help output and the project’s documentation for the version you installed.

Always set a user agent

Every waybackpy class takes a user agent string. Use one that identifies your project and gives a way to contact you, for example "site-recovery-script/1.0 ([email protected])". It is polite, it helps the archive understand automated traffic, and anonymous default agents are more likely to be throttled.

Find the oldest, newest or nearest capture

The availability interface answers the simple question “is there a capture, and where is it?”:

from waybackpy import WaybackMachineAvailabilityAPI

url = "https://example.com/about"
user_agent = "site-recovery-script/1.0 ([email protected])"

availability = WaybackMachineAvailabilityAPI(url, user_agent)
availability.near(year=2016, month=6)
print(availability.archive_url)

Replace the near call with oldest() or newest() when you want the first or last capture instead. This is a quick way to confirm that a page exists in the archive before planning anything bigger.

List every capture with the CDX API

The CDX server is the archive’s index of captures. Waybackpy wraps it so you can loop over snapshots and read their details:

from waybackpy import WaybackMachineCDXServerAPI

cdx = WaybackMachineCDXServerAPI(
    "https://example.com",
    user_agent,
    start_timestamp=2015,
    end_timestamp=2017,
)

for snapshot in cdx.snapshots():
    print(snapshot.timestamp, snapshot.statuscode, snapshot.archive_url)

Each snapshot gives you its timestamp, the original URL, its status code and the archived address. Filtering by status code 200 skips redirects and error captures, which is usually what you want when recovering content. For the raw API parameters behind this, see our Wayback Machine API guide.

List the known URLs of a domain

Waybackpy can also return the archived addresses for a whole domain, which is a fast way to build a page inventory before a restoration. That list is the starting point for rebuilding a sitemap and planning redirects; our guide on finding historical URLs and rebuilding a sitemap shows what to do with it. Expect noise: tracking parameters, duplicate variants and asset files all appear and need filtering.

Example: build a clean page inventory for one year

A common first job before restoring a site is a list of the real pages that existed at a given time. This short script keeps only successful HTML captures and one entry per page:

from waybackpy import WaybackMachineCDXServerAPI

cdx = WaybackMachineCDXServerAPI(
    "example.com/*",
    user_agent,
    start_timestamp=2019,
    end_timestamp=2019,
)

pages = {}
for snapshot in cdx.snapshots():
    if snapshot.statuscode == "200" and snapshot.mimetype == "text/html":
        pages[snapshot.original] = snapshot.archive_url

for original, archived in sorted(pages.items()):
    print(original, archived)

The result is one archived address per page, which you can save to a spreadsheet, review with the site owner, and use to decide which pages to restore. Keep the raw output too: it records where each recovered page came from.

Save a page to the archive

The save interface asks the Wayback Machine to capture a page now:

from waybackpy import WaybackMachineSaveAPI

save_api = WaybackMachineSaveAPI("https://example.com/new-page", user_agent)
print(save_api.save())

Use this sparingly and only for pages you have a reason to preserve, such as your own site before a redesign. Saving is rate limited, and repeated automated saves are likely to be refused.

From capture lists to downloaded files

Waybackpy tells you where captures are. To fetch the original content of a capture without the archive’s toolbar and rewritten links, request the archived address with id_ added after the timestamp. In practice, a small script reads the CDX list from waybackpy, builds those addresses and downloads them with a short pause between requests. That approach works well for a handful of pages or for text recovery. For hundreds of pages with images and stylesheets, a dedicated downloader is less work; our guide on how to use Wayback Machine Downloader covers that route, and fixing links in a downloaded Wayback website explains the cleanup that every route needs.

Limits and good manners

• Pace your requests. Add a delay between calls and avoid running many scripts in parallel. Errors such as HTTP 429 mean you are going too fast.
• Cache results. Save CDX results to a file so you do not query the same URL repeatedly while testing.
• Expect gaps. Not every page or image was captured; a missing result is often a real gap, not a bug. Our guide to restoring from incomplete snapshots covers what to do next.
• Respect content rights. Archived pages remain the work of their owners. Recovering your own site is one thing; republishing someone else’s content is another.

Waybackpy compared with similar tools

Tool	Best for	Not designed for
Waybackpy	Querying captures, listing URLs, saving pages from Python	Downloading and rebuilding a full site
Waybackpack	Downloading every capture of one URL	Crawling a whole site
Whole-site downloaders	Saving all pages and assets from a snapshot period	Clean, ready-to-host output without cleanup
Download service	A cleaned, working copy without coding	Custom research scripts

For a wider comparison, see our overview of Wayback Machine downloader alternatives.

Once waybackpy has listed the captures you need, a Wayback Machine scraper can pull titles, prices or other fields out of those archived pages.

Frequently asked questions

Is waybackpy an official Internet Archive tool?

No. It is an independent open-source project that uses the Internet Archive’s public APIs. The archive’s own documentation describes the underlying endpoints.

Can waybackpy download a whole website?

Not by itself. It lists and locates captures; you need your own download script or a downloader to fetch the files, and cleanup to make them work.

Why do old waybackpy examples not work?

The library changed its interface between major versions. Check the version you installed and use the class names and methods shown in its current documentation and help output.

Do I need an API key?

No key is needed for the public lookup and CDX features. Saving pages and heavy use are subject to the archive’s rate limits.

Prefer not to code it yourself? Request a free archive review and we will tell you what can be downloaded from your site.