Coding Required

Lesson at First Abuse

2024-07-213 min read

When we first rolled out YoPrint, we added the ability to send emails through the system as part of our “mini” CRM. You can notify your customer of important events such as “Approval Required” or “Payment Needed.”

We also added the ability to send custom emails, but we restricted the “Subject” field so it wouldn’t be abused, or so we thought. More on this later

The Motivation

To entice prospects, we made this feature available in our trial. Anyone can create a trial account and experience the true YoPrint experience. We have a full-featured trial account because we are confident that we have winning software and nothing to hide.

The Incident

So, one day, we started receiving support tickets about emails not being sent out. We also noticed that our email spiked by almost 60,000 emails within minutes.

After investigation, it seems an abuser created a trial account and took advantage of our API access to send fraudulent emails about some scam-coin.

We immediately removed the account, but we have a mess on our hands.

Auto-Brakes by Postmark!

Postmark, our email provider, noticed the unusual activity and paused all emails sent to our account. We were counting on this to happen, and we’re happy they didn’t disappoint.

Postmarks swift and sane defaults were one of the many reasons we chose them over any other providers.

The Fix

We quickly implemented additional restrictions.

  1. No emails can be sent in the trial mode by default.
  2. Email access during the trial can be enabled on a per-account basis through customer support.
  3. API access is no longer allowed during trial mode.
  4. Paid accounts now have a maximum daily and monthly email caps.

Time to Restore Access

We contacted Postmark, and they responded almost immediately. It turns out Postmark simply disabled the sending but left the queue intact. They helped us remove any unsent fraudulent emails and restarted sending them.

24 Hours Partial Outage

It almost took us a day to resolve the incident from when we first discovered it to when we finally restored email access. Despite the long time, our customers understood and appreciated our efforts to resolve the issue as quickly as possible. We love our customers, and we’re happy that they love us too!

Being Transparent

We didn’t have an established incident reports system yet, so we simply published an article detailing the incident and the steps we took to rectify it moving forward. We also sincerely apologize for the issue.

We also completely forgot our status page and should probably update our incidents through that.

The Takeaway

Marching Ahead

If I am being honest, I am not happy with our restriction on having no emails during the trial. I want businesses to have the opportunity to try all of our features and make an informed decision.

I plan to allow a small number of emails in trial mode so that businesses can contact customer support to remove the limit.

Anbin Muniandy
CEO & Principal Engineer, YoPrint