The purpose of software is to make life easier, so I made MailLift’s API a little more approachable to the average human. This saved us untold development hours.
The CEO can even talk customers through the integration process. I love reading these messages:
– Brian Curliss, CEO of MailLift in a group SMS, after writing zero code
Why Email?
While designing and costing integrations between MailLift and our customers’ CRMs, we made an interesting discovery. Many CRMs support sending emails after certain events, but very few (not even Salesforce) supported sending configurable HTTP(S) requests without expensive development.
In the old days, any integration boiled down to a simple-sounding answer: build a 3rd party website and user experience to facilitate integrations. When a developer hears those words he or she starts thinking: oh… that includes OAuth (sometimes), managing API keys (other times), figuring out how contacts are stored, building and maintaining a data store to capture state, sorting out when and how to poll for relevant events… in short: each integration is going to cost us a lot.
When we stepped back and thought about what we were trying to do (make MailLift as accessible as possible, as easily as possible), we noticed three interesting things about CRMs, emails, and our user base:
- Many CRMs support sending automated emails on events (e.g. deal closed) – Those same CRMs do not support sending custom REST calls, storing structured responses, and retrying if a downstream service experiences issues
- Email is permanently stored, backed by incredibly reliable global systems, and (recently) relatively easy to access programmatically
- Every one of our customers can write an email (some better than others) – Very few of our customers can (or should) write code to POST to a HTTP(S) endpoint.
So we decided to treat email as an API endpoint. I call it… Maildrop
90’s text for emphasis
Here’s how we did it
Step 1: Create a recognizable format
JSON is easy for developers to read, but we needed something our audience of non-developers can parse. We created a few criteria for an easy to understand format:
- The user can easily recognize field delimiters
- The user can easily type the field delimiters on a keyboard
- Field delimiters are unlikely to appear in the data you’re trying to get (e.g. a handwritten letter likely won’t have 3 equal signs in a row). This removes the burden of communicating escaping (like \” for quotation marks in JSON) to our users.
In our case, we chose equal signs for three reasons. People recognize an equal sign, an equal sign doesn’t require the user to hold shift (or any other key) while typing it on a US English keyboard, and it is very unlikely that a user would use six equal signs with no spaces in a handwritten letter.
Bear in mind that your user is creating this content, not a serializer. So the format should be as intuitively obvious as possible.
We settled on a straightforward format:
1
2
3
4
5
6
7
8
9
===NAME1===
VALUE1
======
===NAME2===
VALUE2...
VALUE2 CONTINUES ON A NEW LINE
AD INFINITUM
AD NAUSEUM
======
The parser looks for 3 equal signs, a variable name, 3 more equal signs, any amount of text and whitespace, followed by 6 equal signs. Since we know we’re working with handwritten letters we can safely trim leading and trailing whitespace.
It’s easy to build a parser for this format. You can even do it confidently with regular expressions. For more difficult API requests, such as those that require the user to encapsulate data in several layers, you may have to consider ways of encapsulation that are readable and easy to understand.
Step 2: Create an authentication scheme
Users who send email to your maildrop account need a way of authenticating so you know which account to associate with a request. In our case, we created one inbox (e.g. maildrop) with a random and unique string after the mail prefix (e.g. maildrop+1a2b3c4d@maillift.com) for each account. Users can reset the key suffix or disable the maildrop feature to reduce their attack surface if they only wish to use the REST API.
When an email arrives at a maildrop address, we check for the random suffix string and try to match it to a user account. If the string matches, we attempt to parse the email and queue up the letter under the user’s account. If the string does not match any users, we do nothing.
In the case of MailLift, we know our customers and their businesses. We are very confident that MailLift can spot anyone attempting to impersonate a maildrop account. You may want to consider email encryption or signature verification (e.g. IRM) if your data is more sensitive.
Step 3: Connect to your email
If you’re like us, you use Google Apps to host and manage your email. Gmail has its own API and, more broadly, Nilas’ API can interface with Google and other inboxes. If you have time on your hands, you can write your own code or use a 3rd party library to interface with IMAP.
Make sure you keep track of the emails you receive and parse, so you don’t process an email twice. If possible, note in your system which items were created from your maildrop API so you can understand how your maildrop traffic is being handled downstream in your system. In the case of MailLift, hand writers have to consider and act on any special formatting of a letter and email can lead to strange formatting in certain circumstances.
Parsing text and HTML — It gets fuzzy
Our users don’t want to spend time configuring their mail clients or remembering to turn HTML formatting on or off. So we opted to parse both text/plain and text/html MIME types.
We try parsing text/plain first for required fields and fall back on HTML if text doesn’t work. In general, text keeps the spacing and formatting your user intended while HTML more readily mangles your user’s content. If you fall back to HTML, strip out all HTML tags first and then parse the message like text.
Document formatting was very important at MailLift, it may not be as important for your needs.
Step 4: Watch everything that comes in
Watch the maildrop box like you watch your early API calls. Make sure that users understand the format and are using it appropriately. Provide gentle feedback in cases where the format is incorrect. You may eventually want to automate this feedback loop by having the maildrop daemon reply to emails when it detects an error.
You will also discover that many email clients have different ideas of how content should be formatted for a user’s message. As you learn more about your users and their habits, adjust your formatting and parser to accommodate how your users understand your system. As always, text is best!
If you have some more time on your hands you can also take part in heated debates about Google’s adherence to various RFCs for email formatting.
Step 5: Get and Give Feedback
In addition to your software’s dashboard, your email checking routine should provide feedback to the user about whether their email was properly received and processed. Remember to keep your messages friendly and offer human help.
Email was originally conceived as an interface for humans to communicate with other humans. So be friendly and have someone from your team respond whenever it makes sense.
Have fun!
Email makes CRM integrations much easier. Consider giving it a shot!