Introduction

SEI's approach to APIs is changing. APIs are products and products of the SEI brand SHOULD aim to follow the same design patterns across our various development teams.  For ALL NEW interfaces being built by the development teams, the standards outlined SHOULD be adhered to in order to maintain that brand consistency.  The benefits of creating and adhering to consistent design guidelines will continue to aggregate as the SEI API program grows.  It will allow for more collaboration, reusable code and patterns around common problems, such as authentication, and allow teams to focus on building brand new interfaces, rather than a new version of a problem already solved.

The guide aims to achieve the following:

• Define consistent approach and design patterns to APIs across SEI

• Align with REST/HTTP industry best practices

• Deliver SEI services to be used both internally and externally via RESTful interfaces leading to higher quality 'product' for all consumers and faster collaboration

• Provide the capability for developers to reuse methods of implementation, testing, and documentation of API endpoints consistently

Suggested Reading

Understanding the philosophy behind the REST Architectural Style is recommended for developing good HTTP-based services. If you are new to RESTful design, here are some good resources:

REST on Wikipedia -- Overview of common definitions and core ideas behind REST.

• Link to Microsoft API standards: 3.1. Recommended reading

​Interpreting the SEI API Guidelines

The keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.

ALL new APIs being developed SHOULD try to enforce these guidelines so external features, such as authentication or data authorization and entitlements, will allow a valuable minimum viable product to be produced regardless of the circumstance.

For existing services, it is not recommended that a breaking change is introduced into the current version of the service to try to comply with the standards.

Consistency Fundamentals

URL Structure

• Humans SHOULD be able to read the URL construction and participate in the design.

• The URL MUST NOT contain any verbs.

• Endpoint URLs (including the base path) must be lower-case (ex: https://api.seic.com/v1/portfolios).

• Keep Resource URL's simple. A path parameter used in the resource URL MUST be mandatory and meet one or more of the following criteria:

  • Help narrow down the hierarchy of the resource 

  • Help identify the resource uniquely

  • Be a resource or a sub resource exposed by the API

Canonical Identifier

• Resources, when applicable, SHOULD contain a unique identifier. 

• This identifier MAY be unique in the context of a SEI client or firm. 

• The developer SHOULD consider if the service is to be used with a complement of other services developed by a different SEI team. 

• In this case, the developer MUST coordinate with that team to identify a common canonical or unique identifier that can be used across services successfully by the consumer.

Supported Methods

The below methods/operations are recognized and MUST be used when possible in the context defined below.  The below list of methods SHOULD be supported for SEI's REST services.  The new services developed MUST conform to the below definitions.

Method Explanations

What is safe and idempotent?

Idempotent: HTTP methods which can be called multiple times and will produce the same result.
Safe: Safe methods are HTTP methods that do not modify the resource

Request Headers

Response Headers

Response Formats

Standardize Response Structure

All responses MUST adhere to the following structure:

• The API data MUST be set in the "data" attribute while the pagination metadata (when applicable) MUST be set in the "paging" attribute.  

• The data attribute MUST be included in the response only for GET, POST, PUT, PATCH, and DELETE.

• The paging attribute MUST be required for responses which return a collection of results.

• The data object SHOULD be returned empty [{"data":{}}] only if all the fields contained under the data object have no data in the target data store. This applies for the paging object as well; if there is no data there will not be any pagination. 

• For scenarios where there is data for some fields returned by the API, the below standards MUST be followed based on the datatype. 

  • If the datatype of the value is an array, and there is no value – return an empty array. This gives clients some safety (no useless null checks during iteration) but overrides default serialization behavior.

  • If the data of the value is an object and there is no value – return null (lower-case).

  • If the data type of the value is a Boolean, it MUST be true or false, null is not an option.

  • If the data type of the value is a Trilean, it MUST be true, false, or null.

  • If the data type of the value is a Number and there is no value, return null (lower-case).

  • If the data type of the value is a String and there is no value, return null (keep in mind this will be most of the use cases where null will be seen in the data store).

Target API Response

​​Link to Microsoft API standards: 3.1. Recommended reading

Customizing Response Representation

Some API consumers may not need the full representation of the resource. Allow clients to specify the fields/sections which are of interest to them. This is especially important if doing so provides clients with performance benefits.

Think: What if I could ask my client to explicitly tell me if they want to traverse the database relationship to get them additional data? This is especially important if additional joins/traversals cause API performance to degrade. We SHOULD offer the API client a choice if they want to incur that performance penalty.

This section documents two strategies to achieve the same behavior. The choice of a strategy depends on

• The "known" usage of an API - If you know how your clients will use the API, use Strategy 1. If the usage patterns are unknown, use Strategy 2.

• The complexity of the API - Legacy systems which have complex data relationships tend to prefer Strategy 2.

Strategy 1: Explicitly request fields/sections

Fields vs. Sections: We consider fields to be scalars (value with no dimension). Sections represent a group of attributes. Section groupings must be driven by the relationships expressed within the resource and not by technical factors.

For e.g. Filter by Sections

Strategy 2: Control Response Depth or Expand Certain Sections

Most APIs return a limited view of a resource by default.  This may be common in requests that return large lists of objects or in the case of a persistence layer that has some child objects set to lazy fetching. In such cases, the client must be explicitly allowed to request the expansion of certain "well-known" sections. The list of such sections must be published in the API documentation.

An alternate approach for limiting how much data is returned for each object is to specify the depth of the data you want returned. For this, use the depth parameter on the query string.  The value starts at 0 and increments with each nested level of data.

Note: While each API provider can implement their own understanding of the "depth" parameter, we highly recommend using the root of the functional response as depth zero.

Use query parameters for filtering, sorting and searching

Simplify Resource URLs as much as possible.

Complex result filters, sorting requirements and advanced searching (when restricted to a single type of resource) can all be easily implemented as query parameters on top of the base URL. Let's look at these in more detail:

Filtering: Use a unique query parameter for each field that implements filtering. For example, when requesting a list of calendars from the /calendar endpoint, you may want to limit these to only applicable for NASDAQ stock exchange in the US. This could be accomplished with a request like GET /calendar/US/NASDAQ?year=2016. Here, year is a query parameter that implements a filter.

Sorting: Similar to filtering, a generic parameter sort can be used to describe sorting rules. Accommodate complex sorting requirements by letting the sort parameter take in list of comma separated fields, each with a possible unary negative to imply descending order. Let's look at some examples:

• GET /portfolios?sort=createdAt - Retrieves a list of portfolios in descending created date.

• GET /portfolios?sort=priority, createdAt - Retrieves a list of tickets in descending order of created date. Within a specific priority, older potfolios are ordered first.

Searching: Sometimes basic filters aren't enough and you need the power of full text search. Perhaps you're already using ElasticSearch or another Lucene based search technology. When full text search is used as a mechanism of retrieving resource instances for a specific type of resource, it can be exposed on the API as a query parameter on the resource's endpoint. Let's say q. Search queries SHOULD be passed straight to the search engine and API output SHOULD be in the same format as a normal list result.

Combining these together, we can build queries like:

1. GET /portfolios?sort=updatedAt - Retrieve recently updated tickets

2. GET /portfolios?state=closed&sort=updatedAt - Retrieve recently closed portfolios

3. GET /tickets?q=ibm&state=open&sort=-priority,createdAt - Retrieve the highest priority portfolio mentioning the word 'IBM'

Properties and field names

• Query parameters SHOULD be camelCased

• There SHOULD be no special characters such as underscores within your request or response fields/objects

Refer to General Guidelines and Property Name Guidelines.

URL Versioning

Always version an API. Versioning helps you iterate faster and prevents invalid requests from hitting updated endpoints. It also helps smooth over any major API version transitions as you can continue to offer old API versions for a period of time.

Every API implementation must be versioned using the semantic versioning scheme. The summarized scheme is included below

Given a version number MAJOR.MINOR.PATCH, increment the:

1. MAJOR version when you make incompatible API changes,

2. MINOR version when you add functionality in a backwards-compatible manner

3. PATCH version when you make backwards-compatible bug fixes.

Additional labels for pre-release and build metadata are available as extensions to the MAJOR.MINOR.PATCH format.

In this case, the major version provides structural stability of the API as a whole while the sub-versions accounts for smaller changes (field deprecations, endpoint changes, etc.)

The URL of the API MUST have a major version number (v1 or v2). The API MAY choose to support sub-versions via an HTTP Request Header named SEIC-API-Version

Aliases for common queries

To make the API experience more pleasant for the average consumer, consider packaging up sets of conditions into easily accessible RESTful paths. For example, the recently closed portfolio query above could be packaged up as GET /portfolios/recentlyclosed

Here 'recentlyclosed' is an alias for ?state=closed&sort=updatedAt. These aliases are often created as certain query patterns find popularity or are known to be well used.

Target API Response Structure

Status Codes

Leveraging HTTP Status codes to convey outcome semantics can be very powerful because their intent is well documented in the W3C specification. A standard set of W3C defined status codes enable faster debugging and wider API adoption because the status codes (or status code ranges) are well defined and understood. An effective use of these status codes can help operation/support personnel triage issues faster. Custom error structures on the wire SHOULD be used only to convey additional information to the developers/level 1 support team.

While the W3C specification defines a varied set of HTTP status codes, the following twelve have emerged as the most commonly used HTTP codes while developing APIs.

Pagination

• Although, RFC 5988 defines Link Headers for pagination, this isn't a prevalent pattern in Web APIs just yet.

• However, as and when this pattern finds prominence, simple changes on the Apigee side can make the pagination information available via Headers as defined in the RFC. The need is to standardize pagination metadata to allow Apigee to inspect and translate the links.

• Companies like Facebook and Twitter follow Cursor based pagination to allow real time access to data.

  • https://dev.twitter.com/ads/basics/pagination 

  • https://developers.facebook.com/docs/graph-api/using-graph-api/

• Please review the link to understand the benefits (and limitations) of cursor based pagination.

  • Cursor based pagination simply requires API providers to include a "before" and "after" cursor in the pagination metadata.

  • Cursors are computed at run-time and are not stored with the entity.

  • Cursors are a base64 encoded result of other properties on the entity, typically just the id or row_num. The property must be sequential. 

• When receiving a cursor in query it must be decoded from base64 back to the serialized properties and then used to find the associated item.

The pagination information must be attached to the response as follows:

Note 1: The "paging" attribute sent by the Target API may be a fully qualified URL. If the URLs are not fully qualified, the full URL qualification is done by Apigee before the response is sent to the client.  

Note 2: The paging attribute is required for responses which return a collection of results

Note 3: To reduce the risk that user manipulation might cause invalid cursors for pagination, the backend API provides an environment variable flag in docker-compose.yml named "PaginatedParams":"f" to make “before” and “after” query parameters visible/invisible from swagger perspective. This way we ensure valid cursors will be constructed during runtime along with the flexibility to turn on/off this feature without code changing based on user requirement down the road.

Note 4:

Scenario 1: When a single page of data returned with respect to the limit, In the paging object [prev and next URL must be null] & [first,last,self URL will be the same]. 
Scenario 2: When more than one page of data is returned:
On the first page [prev URL must be null] & [first,self URL will be the same]. 
On the last page [next URL must be null] & [last,self URL will be the same].

Logging

As an API consumer, developers SHOULD log their API requests, responses and errors must be logged.  Specifically, at a minimum, capture the requestTrackingId that is returned in each API response for troubleshooting purposes:

RequestTrackingId:

A unique identifier (guid) that is generated for every API request, response and error. 

• The target APIs exposed through the API Platform must log a RequestTrackingId that they will receive in the header for every request, response, and error. 

• The RequestTrackingId is also sent to the application consuming the APIs and they must log the RequestTrackingId in their application logs.  

During troubleshooting when we have multiple systems in play the RequestTrackingId helps tie all the points of contact together and can be leveraged commonly to look up the different system logs for a particular request, response or error. 

Encoding

UTF-8 is the standard character encoding format for APIs.  It represents every character in the Unicode character set and is required by the IETF for all protocols.

Compression

The HTTP Specification supports response compression. Response compression is often achieved by specific configuration on the web/application server which hosts the API. The servers must at least support Gzip compression.

Security and Entitlements

Authentication

Data access using the Platform APIs MUST be authenticated and authorized. 

• Apigee secures all Target APIs by asking clients for Site Minder Credentials and a Consumer Key & a Consumer Secret (also known as App Key and Secret). 

• Target APIs MAY choose to implement basic authentication as an additional security measure. In such a scenario Apigee will inject a Base 64 encoded Digest into the HTTP Authorization Header.

Authorization

• APIs MUST be authenticated by leveraging a valid Site Minder id.

• The required business authorization MUST be established at the time of API design. 

• APIs exposed through the API Platform need to follow one of the many infosec approved patterns for authentication and authorization. 

• All APIs SHOULD perform user entitlements checks in the backend service to verify the end user or service account has access to the API.

• All APIs SHOULD perform data authorization checks in the backend service to verify the end user or service account has data access in the API.

• Some examples may include:

  • Transaction filtering by end client.

  • Balance filtering by account.

  • Reference data filtering by firm.

Standard Formats

Date Formats

• Any date attribute in the request/response of the API MUST always use the UTC format. The API must throw an error if the dates passed in the request are in non UTC format. The error thrown MUST comply with the API Standards.

• Note: Some APIs may choose to support a subset of formats supported in the UTC format. In such cases, APIs must perform an upfront validation on invalid formats and report errors which comply to API Program Standards.

• For more information on UTC format please refer to https://www.w3.org/TR/NOTE-datetime

Date Parameter Naming Convention

• If the API accepts a single date as a request query parameter the key must be called "asOfDate".

• If the API accepts a date range as request query parameters the keys must be called "startDate" & "endDate". 

• Any exceptions to this naming convention is permitted if the domain demands it. 

Handling of Personally Identifiable Information (PII)

• PII MUST NOT be used as query, header, or path parameters.

• Explicit PII Examples that MUST NOT be used are:

  • Full name

  • Home Address

  • Email Address

  • Social Security Number

  • Passport Number

  • Driver's License Number

  • Credit Card Number

  • Date of birth

  • Login Credentials

  • Account Numbers

  • Medical Information

  • Insurance Information

• Linkable (Explicit + Linkable), implicit PII Examples that SHOULD NOT be used are:

  • Mother's maiden name

  • Telephone number

  • Gender

  • Ethnic origin

  • Names of minors

  • Place of birth

  • Religious affiliation

• Non-PII Examples that SHOULD be used are:

  • Internal system identifiers

  • Client provided or populated user defined fields (UDFs)