Replacing ethereal.email with Cloudflare Workers and Email Routing

End-to-end testing is an integral part of our Quality Assurance (QA) process. It ensures that our applications behave as expected, and helps us identify any issues before they reach the end user. These tests are written with Cypress simulate the end users actions on the browser to verify that all parts of the application function correctly.

To test the end to end workflow fully, we also automate the validation of email content that is triggered by certain application actions, such as password reset emails, appointment confirmation emails, reminders, etc.

The Problem

When we started building out our end to end test suite, we used a free service ethereal.email, which was at the time a de facto standard with Cypress (see Cypress blog here)

Recently, without notice from ethereal.email, they removed the ability to create new email address accounts for the purpose of receiving emails. Therefore, our continuous integration pipeline started to raise alerts for the team to look into.

We quickly identified the problem and needed a new drop-in replacement without major modifications that offers reliability and would not break the bank.

High-level Requirements

• Near drop-in replacement for existing Cypress tests, with little to no changes to the individual tests. That means exposing a similar HTTP API to fetch emails from an arbitrary email address, and also store incoming emails to a database.
• Reliable and scalable; the email service should consistently work, and have 100% uptime. It should also be able to handle large volumes of emails.
• Cost-effective; the solution should not require significant investment of time or resources, whether it be initial setup or ongoing maintenance.

Solution

With the above high-level requirements in mind, we decided to do some research to see if there are alternatives out there that solves our problem. Unfortunately, we were only able to find paid solutions, or if they were open source will involve significant effort from our DevOps team to spin up and maintain. It seems like an in-house solution is the only remaining viable option.

Email Routing with Cloudflare

Fortunately, Cloudflare had a solution for this - by using a combination of Email Routing and Cloudflare Workers, we were hopeful to come to a very quick solution.

We were able to route all incoming emails to our QA owned domain to a Cloudflare Worker trigger by setting the catch all address to direct to a custom created Cloudflare Worker.

With the entrypoint set, all we had to do now was to create a Cloudflare Worker that can:

1. Intercept all emails and store into the Cloudflare serverless SQL solution (Cloudflare D1)
   
2. Expose an API to fetch messages per address

Intercepting emails with Workers

To intercept emails, we created a simple listener that would decode the email message using PostalMime. The storage layer is powered by Cloudflare's serverless solution, Cloudflare D1 and Drizzle ORM.

import PostalMime from "postal-mime";
import { app } from "./app";
import { drizzle } from "drizzle-orm/d1";
import { emails } from "./schema";

export default {
  async email(message, env: Env) {
    const db = drizzle(env.DB, { logger: true });
    const parser = new PostalMime();
    const email = await parser.parse(message.raw);
    
    // whitelisted senders only
    if (email.from.address !== VERTO_EMAIL) {
      await db
        .insert(emails)
        .values({
          from: email.from.address,
          to: email.to[0].address,
          subject: email.subject,
          html: email.html,
          createdAt: new Date(),
        })
      .run();
    }
  },
  fetch: app.fetch,
  scheduled: onScheduledTrigger,
};

Emails API to fetch emails

The app.ts file contains the main API for fetching received emails, powered by Hono. This allows us to make the following API call to fetch emails.

GET /[email protected]

import { Hono } from "hono";
import { injectDb } from "./db";
import { Bindings } from "../worker-configuration";
import { isAuthenticated } from "./middleware";
import { eq, desc } from "drizzle-orm";
import { emails } from "./schema";

export const app = new Hono<{ Bindings: Bindings }>();
app.use("*", injectDb);

app.get("/getEmails", isAuthenticated, async (c) => {
  const toEmail = c.req.query("email") as string;
  if (!toEmail) {
    return c.json({ error: "Missing email" }, 400);
  }
  const matchedEmails = await c.db
    .select()
    .from(emails)
    .where(eq(emails.to, toEmail))
    .orderBy(desc(emails.createdAt))
    .run();
  return c.json({
    total: matchedEmails.results.length,
    emails: matchedEmails.results,
  });
});

Scheduled deletions

Because it is so easy to define a scheduled task on scheduled workers, we decided to also periodically delete old emails from the database to manage storage and ensure efficiency. In this case, every 24 hours; emails older than a day are deleted.

const db = drizzle(env.DB);
await db
    .delete(emails)
    .where(
        // Delete emails older than 1 day
        lt(emails.createdAt, new Date(Date.now() - 24 * 60 * 60 * 1000))
    )
    .run();

With the above replicated functionality, we were all set to replace all ethereal.email references in our Cypress tests.

Cost

The total cost of the Cloudflare Workers solution came to $0, as we were paying for it with other use-cases already (such as a Sendgrid Webhook routing worker, coming in another blog).

For new Cloudflare users, its free tier will give you 100,000 requests per day. That's enough for 50,000 inbound emails and 50,000 API calls to fetch the email. If you pay $5 per month, that will give you only 10 million requests per month. Hopefully, your test suite does not exceed this limit!

Developing this MVP application took less than a day - as you can see, the whole application is not more than 500 lines of code. Furthermore, since it's powered on top of Cloudflare Workers, there is no DevOps effort other than performing security review and

This solution is therefore - cost effective ✅ and satisfies Requirement 3.

Custom Cypress email plugin

In the end, we needed to ensure that it did not require significant effort or refactor to fix the failing Cypress tests. In this case, we were actually able to simplify the approach!

Previously, with ethereal.email, we had to first execute createTestAccount to create a new inbox before sending emails to it. In the new approach, we no longer need to create inboxes for each test - because we now use a catch-all address, we can simplify generate a random UUID as the email username, such as [email protected].

The API call to fetch emails a drop-in replacement.

Our final email.js is an adaption from the original Cypress blog post, without the nodemailer dependency:

const axios = require('axios');
const { randomUUID } = require('crypto');

/**
 * Function to create a new email account using a generated UUID.
 */
const makeEmailAccount = async () => {
  const emailUUID = randomUUID();
  const email = `${emailUUID}@domain.com`;
  const apiUrl = `https://domain.com/getEmails?email=${email}`;
  const authToken = process.env.INBOX_AUTH_TOKEN; // Use Node.js environment variable

  if (!authToken) {
    throw new Error('INBOX_AUTH_TOKEN environment variable is not set.');
  }

  const userEmail = {
    email,

    /**
     * Utility method for getting the last email.
     */
    async getLastEmail() {
      try {
        const response = await axios.get(apiUrl, {
          headers: {
            Authorization: `Bearer ${authToken}`,
          },
        });

        const emails = response.data.emails;
        if (!emails.length) {
          return null;
        } else {
          // grab the latest email (first in the array)
          const mail = emails[0];
          return {
            subject: mail.subject,
            text: mail.text,
            html: mail.html,
          };
        }
      } catch (e) {
        console.error(e);
        return null;
      }
    },
  };

  return userEmail;
};

module.exports = makeEmailAccount;

Conclusion

Automated QA testing for email inboxes is essential for validating user actions such as account creation and password resets by ensuring that emails are correctly sent and received.

By leveraging Cloudflare Email Workers, we successfully created an ethereal.email alternative in our QA stack, and at the same time, benefited from higher reliability and with near to zero cost. By adopting this approach, we no longer have to rely on a uptime of third party service to run our continuous integration tests on email sending.

A nice 2 day experiment indeed :)