How to support your Engineering team as a Product Manager when shipping APIs

What engineers expect from their product managers (PMs) and how can they help each other.

Hey folks, Ilija here! 👋 Welcome to the 2nd issue of Captain’s Codebook, published on captainscodebook.com.

Every week I write advice on how to become a product engineer, backed by my own experience of over a decade, my own research and learning from others’ experiences. Last week I wrote about:

Why user empathy is a product engineering skill & five tactics how to gain it

To receive all articles and support Captain’s Codebook, consider subscribing 👇

Now, onto the latest issue…

---

I subscribe to Technically, among a few other newsletters. In it, Justin (the author) demystifies tech concepts, and jargon software engineers use. I think it's handy for non-engineering folk that works/interact with engineers. Even though I build software for a living, I still enjoy Technically's explanations. You should check it out!

In the “Ask Technically #3” issue, a reader asked:

What is the contribution that a developer expects from a PM when building an API? What requirements should the PM provide?

I recommend reading Justin's answer because it's a good one. He answers well even though the topic doesn't align strongly with Technically's theme. But then I thought: "wait a sec… I am a developer that builds APIs and that works with Product Managers. So I could answer the question from firsthand experience."

So here it goes — my answer, with Justin's blessings.

What’s an API?

API stands for "Application Programming Interface." Not sure of the term's etymology or genesis, but I've always found it ambiguous. What's an "Interface" here? Is the "Application" a mobile app, or a web app?

Let’s talk about something not so abstract — microwave ovens.

Image courtesy of OpenAPI’s Dall-E

Microwave ovens are kitchen appliances packed in small boxes containing a metal box where we place the food. Inside the metal box, there's an electron tube called a magnetron. When we turn on the oven, the magnetron emits microwaves, and the metal interior of the oven reflects the waves. The emitted microwaves bouncing inside the metal box are absorbed by the food and cause the water molecules in the food to vibrate, producing heat that cooks the food.

Phew! That's quite an explanation. Luckily we don't have to understand how the microwave works to use it. Using it is relatively straightforward:

1. Open the door

2. Place the food in the box

3. Close the door

4. Turn on the microwave

5. Wait, what seems like an eternity

6. Eat warm(ish) food

We all know how to use a microwave because its API is dead simple to use. The API of the microwave oven is the few buttons on its front side and the oven door. The API is the way through which users interact with the oven. Remember the magnetron and the metal box, and the water molecules? Don't you? You don't have to! To use the microwave oven, you must remember only its API.

What is an API, technically?

Now that we understand the concept let's talk about actual APIs.

Software engineers use the term for the surface through which other programs can use a piece of software. If the software is a library, then the API is usually a combination of functions, methods, or types. If the software is an application running on a server somewhere, then the API is typically a set of endpoints (URLs) that clients can call via network protocols. Finally, if the software is a database server, its API can also be its data definition and query languages. I'll admit the last example is a bit of a stretch. Still valid.

The most common APIs are of the second kind: the set of endpoints callable via network protocols. We’ll focus on those.

Network-accessible APIs

Similarly to a website, APIs are accessible via a network protocol. The protocol is almost always HTTP. HTTP is great because it doesn't require particular clients to access the API. Instead, you can do it through a web browser like Firefox or Chrome. All you have to do is navigate your browser to the API's URL, and you'll see its results (assuming it doesn't require any authentication).

One such API is CoinDesk’s API which allows you to grab the latest Bitcoin price against multiple currencies:

Click here to see the API result in your browser.

When the browser requests the API, the server returns the results formatted as JSON, which the browser renders.

Other servers can also call APIs, not just clients (browsers). For example, you might use Stripe's APIs to charge your customers for using your product. Stripe has a well-documented API that allows anyone to move money, create invoices, manage subscriptions, refund money, and much more. Google has a swath of APIs that you can integrate into your products (sometimes free, but more often for a price). See Google's API explorer for a list of their public APIs. Similarly, Spotify has a Web API that requires authentication (so clients can't access it without signing up) that you can use in your applications.

APIs are like infrastructure; they allow services and products to build on top of each other. For example, building your service to accurately convert coordinates to addresses, called geocoding (and reverse-geocoding), is a tricky problem. Especially if that's not your product's competitive advantage, e.g., Shopify does e-commerce well and has no business building geocoding in-house¹. But, if you need geocoding, you can use Google's Geocoding API, and for a small fee, you'll get highly accurate (reverse) geocoding.

Why do PMs find APIs hard to navigate?

By now, it should be evident that APIs are technical products. However, API products can be tricky to reason about because they live in the lower levels of the stack. Their low-level nature requires an engineer to lead the API development or a technical product manager. In addition, leading API projects requires understanding the domain's technical complexity, which can be outside some PMs' areas of interest or comfort.

For product managers some challenging API concepts can include understanding REST v.s. GraphQL v.s. gRPC, authentication and authorization, generating documentation, errors shapes and handling, API versioning, and so on. Also, APIs are hellishly hard to change without breaking the existing integrations. They require a lot of upfront exploration and design, which is at odds with the iterative approach to building products.

Photo by Christina @ wocintechchat.com on Unsplash

Also, for a product manager to steer the product, they must use the product. So they'd have to (at least) navigate Postman well to use an API. And that's the bare minimum. To truly understand how multiple endpoints of an API play together, they'd need to "feel out" the integration. Integrating APIs is a technical endeavor that few PMs can do right out of the gate. This is usually what engineers do.

APIs don't have a UI - the next best thing is the API documentation (hopefully powered by OpenAPI). It must be structured and provide runnable examples. But what's drastically different with APIs, relative to other products, is that the documentation is as necessary as the API itself. The actual users of an API are other applications calling it via a network, but the success of the integration depends on the quality of the API documentation.

There's a slight disconnect between what makes API ergonomic and pleasant to work with and the feature set of the API itself. Wild.

How can a PM contribute to building an API?

Now that we understand APIs and why PMs find them challenging, we can look into what the engineering team expects from PMs.

Always leading with a job to be done (JTBD)

APIs expose endpoints that clients can request - these endpoints usually implement data retrieval (e.g., "get the customer with ID=1") and manipulation (e.g., "update the email of the customer with ID=1 to ‘jdoe@example.com'") or some behavior (e.g. "refund the payment with ID=3"). Each data retrieval/manipulation or behavior in REST design is exposed through a single endpoint.

API endpoints expose functionalities, but they are more low-level relative to forms or screens with a UI. For example, a multi-step wizard might fulfill a whole job for the user, but it'd be a wrong design choice for a single endpoint to mimic a wizard. If we'd get in the engineer's head while designing the API, they are thinking about resources and behaviors, not user flows. This thinking is expected - even senior engineers can go too deep into the rabbit hole.

The tediousness of API design is where PMs can help - always keep the job-to-be-done front and center. If developers are designing an endpoint, always try to tie that with a job to be done. In other words, always bring to the table the user's desired outcome by using the API. For example, while designing the 'GET /v1/users/:id' endpoint, explain to your developers in which scenarios the user might call the endpoint and how they'd use the data they get back from the API.

Photo by Cathryn Lavery on Unsplash

Maintain a list of scenarios & integrations

In line with the JTBD above, PMs must maintain a list of scenarios or user flows they want to support. It would be a bad look if the engineers have spent weeks or months designing and implementing an API endpoint to realize it doesn't support all different scenarios. Reminding the engineering team of the scenarios is always good to avoid disappointment later. Just keep asking, "how does this field in the response help scenario X?".

Besides scenarios, it's crucial to understand how users will integrate your new API. Users can call an API from servers, browsers, or mobile apps. Every integration is different. For example, if web browsers can call the API, the API keys must have minimal abilities and be scoped to the user session. Such integration requires very different API key issuing and management than a server-side-only integration.

Disambiguating use-cases

When building a product, a bazillion things are going on. It requires good mental power to navigate it all. Most of your engineers will remember the different scenarios, but it will soon become a big mush. FUD, wrong assumptions and ambiguity are common telltales of an overwhelmed team. Yet, all are normal and, frankly, expected.

It is your job as a PM to bring clarity to the discussions. If your team is remote, ask your team to write their reasoning behind their design decisions. Stuff like what use-case their design choice supports, or perhaps simpler, was the need behind the choice. Writing is thinking; with things written down, it'll be easy to see where the ambiguity is. Then it's straightforward to disambiguate and clear out any confusion.

Getting user feedback

While the engineering team is busy designing and building the API, you should tell your users about the API. Reach out to different users and introduce the API to them. Learn about their business and current product challenges, and explain how the new API solves (some of) their problems.

Also, get an engineer on the call with you. They can support you and, if needed, go in-depth on the API design with the customer. Don't make the engineer present the API - that should be your job! The engineer is there to support and clarify, not sell.

Beta programs

Some more established companies have structured customer relationships, like beta programs. A beta program is just a short list of enthusiastic customers about your product or company. Such folks are usually willing to discuss early iterations of products or APIs. It gives them insight into how your company is working, but also gives them influence on the shape of the product.

Enroll good beta members: diverse companies that will commit to integrating and testing the API earlier than the rest of your customers. You, as a PM, can talk to the CEO or product folks, while your API engineering team can speak with their engineers. In addition, it will give your team direct access to motivated customers that are technical enough to evaluate the API.

How can engineers help PMs when building an API?

Make tradeoffs together

The perfect API, or for that matter, software, does not exist. We design APIs with what makes sense now, biased by past experiences, while looking towards the future as far as we can. Engineers make tons of decisions while building and designing APIs. Micro-decisions even: "let's not call the field 'author' instead of 'blogger.'" Inconsequential.

It's in these small details where the devil lies, as they say. So if you are an engineer and are unaware of your decision's downstream effects, raise it. Your product manager would love to discuss it with you. Most often, it'll be a 5-minutes-long chat. But there will be some decisions that might take you to the whiteboard.

Recently, while designing an API, I needed to decide on a parameter: should I allow inlining object data in the parameter or pass a reference to said object and snapshot the data?

The dilemma was between:

{
  "detective_data": {
    "name": "Sherlock Holmes",
    "address": {
      "street": "221B Baker St",
      "city": "London",
      "postal_code": "NW1 6XE",
      "country": "UK"
    }
  }
}

and

{
  "detective": "detective_ID_here"
}

You might conclude that #2 is better, but I'd wager you need to talk with your PM about this dilemma. You need customer insight from your PM that can help you here. For example, maybe providing 'detective_data' will be enough, as your PM doesn't expect customers to keep detectives stored in your database. Or perhaps you want to support both options? If yes, you'd like to discuss which will take precedence if the user provides both parameters. Or do you fall back on the 'detective' object if the address provided in the 'detective_data' is invalid?

Dogfood the API, together

Product managers’ primary focus is users. It's their job to consider the users 'needs' and 'wants.' They would love it if you two were to sit down and simulate an integration with your new API. Remember that list of scenarios that you should ask your PM to make? Those would be handy if you did the exercise, as they'd set the context of the integration.

There might be similarities in the integration scenarios, but each one will have its quirks. You want to trip up during the simulation. You want to make mistakes that your customers might. And as scary as it sounds, it's great if you realize that an API endpoint doesn't support a scenario. It's better to discover pitfalls before your API gets into your customers' hands. Then it might be a bit too late.

---

And that’s it for this week! If you liked this, consider doing any of these:

1) ❤️ Share it — Captain’s Codebook lives thanks to word of mouth. Share the article with your team or with someone to whom it might be useful!

2) ✉️ Subscribe — if you aren’t already, consider becoming a subscriber. Seeing folks read the newsletter energizes me and gives me ideas to write.

3) 🧠 Ping me — I am always on the lookout for questions to answer, to help out with challenges, or topic ideas to cover. Let me know what’s on your mind!

Until next time,
Ilija

1. No clue if Shopify has an in-house geocoding service, it was just an example. Once a company reaches a particular scale, some previously ludicrous ideas might appear reasonable.