Email is one of those systems that looks simple until your tests need to interact with it. Sending a message is easy. Verifying that the right message arrived, in the right inbox, with the right token, and that the token still works inside a CI run is where most teams discover they have built a miniature distributed system.

A good test email harness with IMAP and Mailgun can make email verification testing boring in the best way. It can cover signup confirmations, password resets, magic link testing, 2FA codes, and transactional notifications without requiring a human to watch a mailbox. But if you are not disciplined, the harness becomes a side project with its own bugs, retries, cleanup jobs, and documentation that nobody wants to own.

This article shows the build path I would use if I needed stable inbox automation for a product team. It starts with the minimum viable harness, then moves through parsing, credential isolation, retries, cleanup, and the maintenance tradeoff that eventually decides whether to keep custom plumbing or move to a maintained platform.

What the harness needs to do

At a minimum, the harness has to answer four questions:

  1. Did the application send the message?
  2. Did it land in the right test inbox?
  3. Can the test extract the relevant value, such as a link or token?
  4. Can the test complete the flow without leaking secrets or leaving inbox clutter behind?

That sounds small. It is not.

The test has to coordinate the app under test, the mail provider, and the browser or API client that will consume the message. In Test automation terms, this is a classic boundary problem, because the thing you are validating is not just UI behavior, it is a workflow that spans multiple systems. IMAP is the access protocol that lets your harness inspect mailboxes, while Mailgun can give you a controlled way to send test messages or route application mail depending on your setup.

The most important design choice is not how to parse an email. It is what failure should mean when the email does not arrive.

If your harness treats every delay as a hard failure, you will get flaky tests. If it waits forever, CI becomes a parking lot. The right answer is a bounded wait with a clear error that tells the next engineer exactly what failed.

Start with a narrow scope

Do not try to support every email scenario at once. Pick one flow that matters to the product and is easy to recognize in the mailbox.

Good first candidates:

  • password reset
  • account verification
  • magic link login
  • 2FA code delivery
  • one transactional notification with a stable template

Each of these has a predictable event on the backend and a predictable payload in the email. That predictability is what makes the harness maintainable.

A practical scope for the first version:

  • one provider, such as Mailgun
  • one mailbox namespace for test runs
  • one parser for the exact template you care about
  • one browser test that consumes the extracted link or code
  • one cleanup step after the test finishes

If you need to verify multiple templates, use the same harness shape, but keep the per-template logic thin. Avoid one giant email parser that understands every notification in the product. That is how test code turns into application code.

A simple architecture that does not overreach

A stable version usually has four pieces:

1. Trigger the app action

Your test suite performs the user action that sends the email, for example, submitting a signup form or requesting a password reset.

2. Poll the inbox over IMAP

The harness connects to a dedicated mailbox and looks for the newest unseen message that matches a subject line, sender address, or correlation token.

3. Parse the target value

The harness extracts a link, OTP, or verification token from the message body or HTML.

4. Continue the test flow

The browser opens the link, enters the token, or performs the next authenticated step.

Here is a compact Playwright example for the browser side, assuming your mailbox helper returns the reset URL:

import { test, expect } from '@playwright/test';
import { waitForResetEmail } from './mailbox';

test('password reset email flow', async ({ page }) => {
  await page.goto('https://app.example.com/forgot-password');
  await page.getByLabel('Email').fill('test.user@example.com');
  await page.getByRole('button', { name: 'Send reset link' }).click();

const resetUrl = await waitForResetEmail(‘test.user@example.com’); await page.goto(resetUrl);

await expect(page.getByRole(‘heading’, { name: ‘Reset your password’ })).toBeVisible(); });

The browser test stays readable because the mail logic is isolated behind a helper. That helper is where the operational complexity lives.

IMAP polling without making CI miserable

IMAP is a reasonable choice when you need direct mailbox access and want to avoid scraping webmail. You do not need a huge abstraction layer, but you do need a few hard rules.

Use a dedicated mailbox per test lane

Do not share one inbox across all environments and all parallel jobs. Shared mailboxes create race conditions, especially when two jobs send the same template at the same time.

Useful isolation patterns:

  • one mailbox per branch or environment
  • one mailbox per parallel worker
  • one mailbox plus a unique correlation token in the email subject or body

If you cannot isolate per worker, at least tag the email request with a request ID and search for that ID later. Correlation beats clever polling.

Filter by what the app can control

Subject, recipient, sender, and embedded token are the easiest filters. Arrival time is the weakest filter, because CI timing is variable.

A decent IMAP search strategy is:

  • search for unseen messages
  • narrow by recipient or subject prefix
  • select the newest matching message
  • parse only after a definite match

Keep polling bounded

A common wait pattern looks like this:

import imaplib
import email
import os
import time

IMAP_HOST = os.environ[‘IMAP_HOST’] IMAP_USER = os.environ[‘IMAP_USER’] IMAP_PASS = os.environ[‘IMAP_PASS’]

def wait_for_message(subject_fragment, timeout_seconds=90, interval_seconds=5): deadline = time.time() + timeout_seconds while time.time() < deadline: with imaplib.IMAP4_SSL(IMAP_HOST) as client: client.login(IMAP_USER, IMAP_PASS) client.select(‘INBOX’) status, data = client.search(None, f’(UNSEEN SUBJECT “{subject_fragment}”)’) if status == ‘OK’ and data[0]: latest_id = data[0].split()[-1] status, msg_data = client.fetch(latest_id, ‘(RFC822)’) raw = msg_data[0][1] return email.message_from_bytes(raw) time.sleep(interval_seconds) raise TimeoutError(f’No email arrived for subject {subject_fragment}’)

This is intentionally plain. Fancy code tends to hide the important parts, which are timeout policy, mailbox selection, and error reporting.

Parse HTML and text bodies carefully

Many templates have both plain text and HTML. Links might appear in only one version, and some tokens are wrapped in tags that are easy to miss if you only parse the raw body.

Practical rules:

  • prefer HTML parsing when the link is rendered as a clickable anchor
  • fall back to text parsing when the plain text template is authoritative
  • normalize URLs before comparing them
  • decode HTML entities before extracting tokens

If you only care about a token or magic link, use one parser per message template. Avoid generic regular expressions that attempt to understand the whole email body. They are brittle, especially when the template team updates copy.

Mailgun as a control point, not a crutch

Mailgun can be useful in a harness in two different ways.

First, it can be the mail service your application uses in test and staging, which gives you a realistic path from app to inbox. Second, it can be part of the verification setup if you are using its APIs or routing rules to make delivery observable. The exact integration depends on your application architecture and the environments you control.

What matters operationally is that the harness should not require a human to inspect Mailgun dashboards.

A reliable pattern is to create a test-specific sending domain or subdomain, then route messages to dedicated test mailboxes. That keeps test mail separate from human mail and makes cleanup more tractable.

Isolate credentials and permissions

This is where many teams get sloppy.

Do not put production mail credentials in test code. Do not reuse personal inbox credentials. Do not make the harness depend on one engineer’s mailbox password stored in a local secret manager.

Prefer:

  • environment-specific secrets in your CI vault
  • a test-only Mailgun domain or API key
  • an IMAP account limited to the test inbox
  • rotation documented in the repo, not in someone’s head

A harness that can read real mail is sensitive infrastructure. Treat it that way.

The most common email-flow checks are verification link clicks and code entry. Both need extraction logic that is strict enough to catch bad emails, but tolerant enough to survive harmless template changes.

If the email contains a link with a tokenized path or query parameter, parse the URL and validate the expected host and route before navigating to it.

typescript

const url = new URL(resetUrl);
expect(url.hostname).toBe('app.example.com');
expect(url.pathname).toContain('/reset-password');

Do not blindly visit any URL found in the email. That makes the test vulnerable to malformed content and gives you weaker assertions.

OTP or code extraction

For a code like a 6-digit OTP, extract only the code shape you expect and assert the length and character class.

import re

match = re.search(r’\b(\d{6})\b’, email_body) if not match: raise ValueError(‘No 6-digit OTP found’) code = match.group(1)

This catches a class of template regressions, such as when the code format changes or the wrong message is sent.

Correlation tokens

If your application can include a request identifier in the subject or body, do it. That makes it far easier to avoid grabbing the wrong message when multiple tests are sending mail at once.

A simple correlation token can be the test-run ID, branch name, or a generated nonce written into the reset request and echoed in the email subject.

Retries are necessary, but only in the right place

Email delivery is asynchronous. Your harness should expect delay, not failure.

That does not mean every step gets a retry loop. Be selective.

Retry these:

  • mailbox polling
  • transient IMAP disconnects
  • short-lived browser navigation after clicking a link

Do not retry these blindly:

  • invalid email templates
  • missing tokens
  • wrong recipient addresses
  • expired reset links caused by bad test sequencing

A good retry strategy is based on failure mode:

  • if no message has arrived, keep polling until the deadline
  • if the mailbox returns a transient error, reconnect and retry
  • if the message arrived but parsing failed, fail immediately with the raw subject and a scrubbed snippet

Retries should absorb network noise, not hide product bugs.

If you retry parsing after a malformed email, you are usually just wasting CI time.

Cleanup matters more than people expect

Inbox automation turns into an operational problem when messages accumulate.

At a minimum, clean up by:

  • deleting processed test emails
  • expiring old test inboxes
  • rotating mailbox aliases
  • resetting state between environments

If the mail provider supports foldering or labels, move consumed messages out of the main inbox so a later run does not accidentally reuse them.

If messages are not deleted, make sure search queries exclude older messages by timestamp or unique token. Old mail is a subtle source of false positives.

Also clean up the app side:

  • delete test accounts if the flow creates them
  • reset password tokens if your app stores them
  • ensure repeated runs do not collide with prior state

An email harness that only cleans up the mailbox but leaves the application state dirty will still produce flaky tests.

Where custom harnesses fail in practice

The custom route fails for predictable reasons.

1. The email template changes often

If content, sender, or markup changes frequently, parsers break. That means your test infrastructure is coupled to product copy and design decisions.

2. Parallel runs collide

A shared inbox plus parallel CI jobs is a bad combination. One test picks up another test’s message and the failure is confusing.

3. Credential management becomes somebody’s part-time job

If the test harness depends on secrets nobody owns, it rots. The code still exists, but nobody trusts it enough to use it.

4. Debugging crosses too many systems

When an email flow fails, you may need to inspect app logs, Mailgun logs, mailbox contents, IMAP state, and browser traces. That is a lot of context switching for a routine smoke test.

5. Maintenance cost is hidden

The harness looks cheap because it is just a few utility functions. The real cost is the ongoing review burden, the occasional provider API change, and the time spent making flaky timing issues reproducible.

This is the point where engineering managers should ask a blunt question, not a hopeful one: is this harness core product value, or is it undifferentiated plumbing?

A decision rule for keeping or replacing the harness

Keep the custom harness when all of these are true:

  • the email flow is unique to your product logic
  • the team needs tight control over mailbox routing or message format
  • the harness is small enough for the same engineers who own the feature to maintain it
  • the failure modes are understood and infrequent

Consider replacing or reducing the custom plumbing when:

  • the team is spending more time on inbox automation than on the actual product tests
  • the inbox setup is duplicated across multiple services or repos
  • failures are mostly infrastructure noise, not product defects
  • new engineers need a long explanation just to understand why the mailbox exists

There is no prize for owning the most bespoke email harness. The goal is dependable coverage with the least maintenance that still gives you signal.

A practical CI pattern

A simple CI job can validate the flow without becoming fragile.

name: email-flow-check

on: push: branches: [main] pull_request:

jobs: test: runs-on: ubuntu-latest timeout-minutes: 15 steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: node-version: 20 - run: npm ci - run: npm test – –grep “email flow” env: IMAP_HOST: $ IMAP_USER: $ IMAP_PASS: $ MAILGUN_DOMAIN: $

Notice what this does not do. It does not hardcode credentials, and it does not try to be clever about test ordering. The job should fail fast if the mailbox is unreachable and should emit enough logs to tell you whether the issue is delivery, parsing, or browser navigation.

A lighter alternative when the plumbing is the problem

If the team wants stable email-flow coverage but not the ongoing burden of managing inbox plumbing, a maintained platform can be a simpler path. For example, Endtest, an agentic AI test automation platform, offers email and SMS testing with real inboxes, real phone numbers, and built-in extraction of links, codes, and message content. That can remove a lot of mailbox setup and reduce the maintenance surface, especially when your real goal is to verify the flow, not to own a mail subsystem.

The same tradeoff applies more broadly to UI maintenance. If the harness starts breaking because of locator drift, a platform with self-healing tests can reduce the babysitting load by adapting to UI changes and keeping the run going when element locators shift.

I would still keep custom code where the email workflow is truly unique. But if your team mainly needs reliable email-flow checks, and the custom harness is becoming operational overhead, using a maintained tool is often the more honest allocation of engineering time.

The version worth shipping

If I were reducing all of this to a team standard, it would look like this:

  • use a dedicated test mailbox or mailbox namespace
  • isolate credentials in CI secrets and rotate them
  • poll IMAP with a hard timeout
  • search by correlation token, not by timing
  • parse only the message template you care about
  • validate extracted links and codes before using them
  • clean up mailbox and application state after each run
  • revisit the build-vs-buy decision when maintenance starts competing with product work

That is enough to make email verification testing useful without turning it into a side project.

The real job is not to build a clever harness. The real job is to make sure your team can trust email flow checks, keep them running, and move on to the parts of the product that actually need human judgment.