The Twitter Network Layer (TNL) is a framework for interfacing with the Apple provided
NSURLSession stack that provides additional levels of control and insight over networking requests,
provides simple configurability and minimizes the cognitive load necessary to maintain a robust and
wide-reaching networking system.
The Twitter Network Layer sits on top of the connection/session layer provided by the Apple NSURL framework. Those frameworks are build on top of the HTTP/1.1 and HTTP/2. The layer chart appears like this:
/--------------------------------------\
| |
| User Layer |
| The actual user (Layer 8) |
| |
|--------------------------------------|
| |
| Application Layer |
| MVC, MVVM, etc (Layer 7e) |
| |
|--------------------------------------|
| |
| Concrete Operation/App Layer | <------ Operations, Requests &
| TNL (Layer 7d) | Responses built on TNL
| |
|--------------------------------------|
| |
| Abstract Operation/App Layer |
| TNL (Layer 7c) | <------ TNL
| |
|--------------------------------------|
| |
| Connection/App Layer |
| NSURL Stack (Layer 7b) |
| |
|--------------------------------------|
| |
| Protocol/App Layer |
| HTTP/1.1 & HTTP/2 (Layer 7a) |
| |
|--------------------------------------|
| |
| Presentation Layer |
| Encryption & Serialization (Layer 6) |
| |
|--------------------------------------|
| |
| Session Layer |
| A Feature of TCP (Layer 5) |
| |
|--------------------------------------|
| |
| Transport Layer |
| TCP (Layer 4) |
| |
|--------------------------------------|
| |
| Routing/Network Layer |
| IP (Layer 3) |
| |
|--------------------------------------|
| |
| Data Link Layer |
| IEEE 802.X (Layer 2) |
| |
|--------------------------------------|
| |
| Physical Layer |
| Ethernet (Layer 1) |
| |
\--------------------------------------/Twitter Network Layer provides a framework on top of Apple's NSURLSession framework with
numerous benefits. Here are some of the features provided by TNL:
NSURLSession, simplified where appropriateNSOperation based request operations (for NSOperation features)TNLHTTPRequest) and configurations (TNLRequestConfiguration)TNLResponse)NSData storage, callback chunking or saving to file)The high level concept of how to use TNL is rather straightforward:
TNLRequestConfiguration instancesTNLRequestDelegate (if added functionality is desired beyond just handling the result)TNLRequestConfiguration for reuseTNLGlobalConfigurationTNLRequestOperationQueue objects once (ex: one for API requests, one for image requests, etc.)
[TNLRequestOperationQueue initWithIdentifier:]TNLRequestOperation with the following objects:
TNLRequest conforming object (including TNLHTTPRequest concrete class and NSURLRequest)TNLRequestConfiguration (optional)TNLRequestDelegate (optional)TNLResponse
TNLRequestOperation's TNLRequestDelegateTwitter Network Layer documentation starts with this README.md and progresses through the APIs via their documentation.
The core objects of a service based architecture are request, response and operation/task/action (referred to as an operation from here on). The request encapsulates data to send and is not actionable; the response encapsulates the data received and is not actionable; and the operation is the object that delivers the request and retrieves the response and is the only actionable object in the set of core objects.
This high level concept translates directly into a network architecture as we will have requests that encapsulate the data of an HTTP request which are Headers and a Body, responses that encapsulate the data of an HTTP response which are Headers and a Body, and last the operation that executes delivering the request and retrieving the response.
TNLRequest is the protocol for requests in TNLTNLHTTPRequest and NSURLRequest are concrete classes (both are immutable/mutable pairs)TNLResponse is the object for responses in TNL (composite object that includes an NSHTTPURLResponse)TNLRequestOperation is the operation in TNL (subclasses NSOperation) and is backed by NSURLSessionTaskIn addition to a service architecture having requests, operations and responses; support objects are often present that aid in the management of the executing operations, configuration of their behavior and delegation of decisions or events.
The configuration object encapsulates how an operation behaves. It will have no impact on what is sent in the operation (that's the request), but can modify how it is sent. For instance, the configuration can indicate a maximum duration that the operation can take before it should fail.
The delegate object provides the extensibility of on demand decision making when prudent as well as the delivery of events as the operation executes.
The manager object coordinates the execution of multiple operations within a logical grouping.
TNLRequestConfiguration is the config in TNL (applied per operation)NSURLSessionConfiguration is the config in NSURL stack (applied per manager)TNLRequestDelegate is the delegate in TNL (provided per operation)NSURLSessionDelegate is the delegate in NSURL stack (provided per manager)manager
TNLRequestOperationQueue is the manager object in TNLNSURLSession is the manager object in NSURL stackNote: You can already see there is a fundamental architecture difference between NSURLSession networking
and Twitter Network Layer. The configuration and delegate per operation approach in TNL
is much more scalable when dealing with dozens or hundreds of unique configurations and/or delegates
given a plethora of requests and their needs. Coupling the configuration and/or delegate to the
reusable manager object(s) is unwieldy and can lead to mistakes w.r.t. correct configuration and event
on a per request basis.
TNL uses the TNLRequest as the interface for all network requests. In practice, the protocol
is used in one of 3 ways:
TNLHTTPRequest
TNLHTTPRequest object (or TNLMutableHTTPRequest)NSURLRequest
NSURLRequest explicitely conforms to TNLRequest protocol via a category in TNL making it supported as request object.NSURLRequest are observed and the configuration properties of NSURLRequest are ignored.Implementing a custom TNLRequest
TNLRequest as an original request for an operation.APPRetrieveBlobRequest that has 1 property, the identifier for the "Blob" call blobIdentifier.TNLRequest. This can be done in 2 ways:
TNLRequest and have its methods that populate the values by the relevant properties (in our example, the blob identifier)TNLRequest ready for HTTP transport.When it comes to making the choice, it can boil down to convenience vs simplicity of reuse. If you have a one shot request that has no reuse, options 1 and 2 will suffice. If you have a request that can be reused throughout the code base, option 3 clearly offers the cleanest interface. By having the caller only need to know the class of the request and the relevant values for populating the request, any concern over the HTTP structure is completely eliminated.
TNLHTTPRequest:
NSString *URLString = [NSString stringWithFormat:@"http://api.myapp.com/blob/%tu", blobIdentifier];
NSURL *URL = [NSURL URLWithString:URLString];
TNLHTTPRequest *request = [TNLHTTPRequest GETRequestWithURL:URL
HTTPHeaderFields:@{@"User-Agent": [MYAPPDELEGATE userAgentString]}];TNLMutableHTTPRequest:
NSString *URLString = [NSString stringWithFormat:@"http://api.myapp.com/blob/%tu", blobIdentifier];
NSURL *URL = [NSURL URLWithString:URLString];
TNLMutableHTTPRequest *mRequest = [[TNLMutableHTTPRequest alloc] init];
mRequest.HTTPMethodValue = TNLHTTPMethodValueGET;
mRequest.URL = URL;
[mRequest setValue:[MYAPPDELEGATE userAgentString] forHTTPHeaderField:@"User-Agent"];NSString *URLString = [NSString stringWithFormat:@"http://api.myapp.com/blob/%tu", blobIdentifier];
NSURL *URL = [NSURL URLWithString:URLString];
NSMutableURLRequest *mRequest = [[NSMutableURLRequest alloc] init];
mRequest.HTTPMethod = @"GET";
mRequest.URL = URL;
[mRequest setValue:[MYAPPDELEGATE userAgentString] forHTTPHeaderField:@"User-Agent"];1) Request Hydration
APPRetrieveBlobRequest *request = [[APPRetrieveBlobRequest alloc] initWithBlobIdentifier:blobIdentifier];
// ... elsewhere ...
- (void)tnl_requestOperation:(TNLRequestOperation *)op
hydrateRequest:(APPRetrieveBlobRequest *)request // we know the type
completion:(TNLRequestHydrateCompletionBlock)complete
{
NSString *URLString = [NSString stringWithFormat:@"http://api.myapp.com/blob/%tu", blobRequest.blobIdentifier];
NSURL *URL = [NSURL URLWithString:URLString];
TNLHTTPRequest *newReq = [TNLHTTPRequest GETRequestWithURL:URL
HTTPHeaderFields:@{@"User-Agent": [MYAPPDELEGATE userAgentString]}];
complete(newReq);
}2) Request with HTTP support
APPRetrieveBlobRequest *request = [[APPRetrieveBlobRequest alloc] initWithBlobIdentifier:blobIdentifier];
// ... elsewhere ...
@implementation APPRetrieveBlobRequest
- (NSURL *)URL
{
NSString *URLString = [NSString stringWithFormat:@"http://api.myapp.com/blob/%tu", self.blobIdentifier];
return [NSURL URLWithString:URLString];
}
- (NSDictionary *)allHTTPHeaderFields
{
return @{@"User-Agent":[MYAPPDELEGATE userAgentString]};
}
// POINT OF IMPROVEMENT:
// utilize polymorphism and have an APPBaseRequest class that implements
// the "allHTTPHeaderFields" so that all subclasses (including APPRetrieveBlobRequest)
// will inherit the desired defaults.
// This can apply to a wide range of HTTP related TNLHTTPRequest properties
// or even composition of subcomponents that are aggregated to a single property.
// For example: the host of the URL (api.myapp.com) could be provided
// as a property on APPBaseRequest that permits subclasses to override the host, and then
// the construction of the `URL` uses composition of variation properites that the subclasses
// can provide.
@endWhen an operation completes, a TNLResponse is populated and provided to the completion block
or completion callback (depending on if you use a delegate or not). The TNLResponse has all
the information necessary to understand how the operation completed, as well as what information
was retrieve. Additionally, with response polymorphism, the response can be extended to
provide better contextual information regarding the result, such as parsing the response body as
JSON or converting the response body into a UIImage.
The way you deal with a TNLResponse should be systematic and straighforward:
anyError property for quick access to any error in the response.deal with the response payload
NSData or a file on disk), etcOne benefit to using response polymorphism is the ability to handle the response and populate
the hydrated response with the information that's pertinent to the caller.
For example: if your network operation yields JSON, and all you care about is if that JSON came
through or not, at hydration time you could check for any error conditions then parse out the JSON
and if everything is good have a property on the custom TNLResponse subclasss that holds the
NSDictionary result property (or nil if anything along the way prevented success).
Things you can inspect on a response by default:
data or temporarySavedFile if the operation was configured to maintain the data)NSURLRequest that loaded the responseNSURLResponse objectNSURLTwitter Network Layer provides a highly robust API for building network operations with a great deal of flexibility and extensibility. However, there are often occasions when you just need to execute an operation and need things to be as simple as possible. Twitter Network Layer provides all the convenience necessary for getting what needs to be done as simply as possible.
NSString *URLString = [NSURL URLWithString:@"http://api.myapp.com/settings"];
NSURLRequest *request = [NSURLRequest requestWithURL:URLString];
[[TNLRequestOperationQueue defaultOperationQueue] enqueueRequest:request
completion:^(TNLRequestOperation *op, TNLResponse *response) {
NSDictionary *json = nil;
if (!response.operationError && response.info.statusCode == 200) {
json = [NSJSONSerialization JSONObjectWithData:response.info.data options:0 error:NULL];
}
if (json) {
[self _myapp_didCompleteSettingsRequestWithJSON:json];
} else {
[self _myapp_didFailSettingsRequest];
}
}];Often the vanila configuration for an operation will suffice, however it is common to need particular behaviors in order to get specific use cases to work. Let's take, as an example, firing network operation when a specific metric is hit. In this case, we don't care about storing the response body and we also want to avoid having a cache that could get in the way.
NSURL *URL = [NSURL URLWithString:@"http://api.myapp.com/recordMetric?hit=true"];
TNLHTTPRequest *request = [TNLHTTPRequest GETRequestWithURL:URL HTTPHeaderFields:nil];
TNLMutableRequestConfiguration *config = [TNLMutableRequestConfiguration defaultConfiguration];
config.responseDataConsumptionMode = TNLResponseDataConsumptionModeNone;
config.URLCache = nil; // Note: 'URLCache' is now 'nil' by default in TNL, but the illustration still works
TNLRequestOperation *op = [TNLRequestOperation operationWithRequest:request
configuration:config
completion:^(TNLRequestOperation *o, TNLResponse *response) {
assert(response.info.source != TNLResponseSourceLocalCache);
const BOOL success = response.info.statusCode == 202;
[self didSendMetric:success];
}];
[[TNLRequestOperationQueue defaultOperationQueue] enqueueOperation:op];Now, sometimes, you may want to have the same defaults for certain kinds of operations. That can easily be accomplished with a category or some other shared accessor.
@interface TNLRequestConfiguration (APPAdditions)
+ (instancetype)configurationForMetricsFiring;
@end
@implementation TNLRequestConfiguration (APPAdditions)
+ (instancetype)configurationForMetricsFiring
{
static TNLRequestConfiguration* sConfig;
static dispatch_once_t onceToken;
dispatch_once(&onceToken, ^{
TNLMutableRequestConfiguration *mConfig = [TNLMutableRequestConfiguration defaultConfiguration];
mConfig.URLCache = nil; // Note: 'URLCache' is now 'nil' by default in TNL, but the illustration still works
mConfig.responseDataConsumptionMode = TNLResponseDataConsumptionModeNone;
sConfig = [mConfig copy];
});
return sConfig;
}
@end
@implementation TNLMutableRequestConfiguration (APPAdditions)
+ (instancetype)configurationForMetricsFiring
{
return [[TNLRequestConfiguration configurationForMetricsFiring] mutableCopy];
}
@endTwitter Network Layer was designed from the ground up with REST APIs in mind. From simple APIs to complex API layers that require a complicated system for managing all operations, TNL provides the foundation needed.
As a pattern for creating concrete API operations, one of the first places to extend TNL for
your API layer is by concretely building API requests and responses. For requests, you
implement a TNLRequest for every request your API provides with properties that configure each
request appropriately. Those requests should be subclassing a base request that does the busy
work of pulling together the generic properties that the subclasses can override to construct the
HTTP properties of the request. Each subclassed request then overrides only what is
necessary to form the valid HTTP request. For things that are context or time sensitive, such as
request signing, request hydration should be used to fully saturate the custom API request at
the time the request is going to sent (vs at the time it was enqueued).
Following from custom API requests are custom API responses. At a minimum, it makes sense to
have an API response that subclasses TNLResponse. To provide an even simpler interface to
callers, you can implement a response per request. For response hydration, you merely
extract whatever contextually relevant information is valuable for an API response and set those
properties on you custom subclass of TNLResponse (such as API error, JSON result, etc).
If the API layer is advanced enough, it may warrant further encapsulation with a managing object which is often referred to as an API client. The API client would manage the queuing of requests, the delegate implementation for operations (including hydration for requests and subclassing responses so they hydrate too), the vending of operations, authentication/signing of requests, high level retry plus timeout behavior, custom configurations and oberving responses for custom handling.
With an API client architecture, the entire HTTP structure is encapsulated and callers can deal with things just the objects they care about. No converting or validating. No configuring. The power of TNL is completely utilized by API client freeing the caller of any burden.
APISendMessageRequest *request = [[APISendMessageRequest alloc] init];
request.sender = self.user;
request.receiver = otherUser;
request.message = message;
self.sendOp = [[APIClient sharedInstance] enqueueRequest:request
completion:^(TNLRequestOperation *op, APIResponse *response) {
[weakSelf messageSendDidComplete:op withResponse:(id)response];
}];
// ... elsewhere ...
- (void)messageSendDidComplete:(TNLRequestOperation *)op withResponse:(APISendMessageResponse *)response
{
assert(self.sendOp == op);
self.sendOp = nil;
if (!sendMessageResponse.wasCancelled) {
if (sendMessageResponse.succeeded) {
[self updateUIForCompletedMessageSendWithId:sendMessageResponse.messageId];
} else {
[self updateUIForFailedMessageSendWithUserErrorTitle:sendMessageResponse.errorTitle
errorMessage:sendMessageResponse.errorMessage];
}
}
}
// Insight:
// Presumably, APISendMessageResponse would subclass a base response like APIBaseResponse.
// Following that presumption, it would make sense that APIBaseResponse would expose
// wasCancelled, succeeded, errorTitle and errorMessage while APISendMessageResponse would
// expose messageId (since that is part of the response payload that is specific to the request).
// It would likely make sense that if the API used JSON response bodies,
// the base response would also expose a "result" property (NSDictionary) and
// APISendMessageResponse's implementation for messageId is just:
// return self.result[@"newMessageId"];Twitter Network Layer includes a target for building a macOS tool called tnlcli. You can build this tool
run it from Terminal from you Mac, similar to cURL or other networking command line utilities.
Usage: tnlcli [options] url
Example: tnlcli --request-method HEAD --response-header-mode file,print --response-header-file response_headers.json https://google.com
Argument Options:
-----------------
--request-config-file <filepath> TNLRequestConfiguration as a json file
--request-headers-file <filepath> json file of key-value-pairs for using as headers
--request-body-file <filepath> file for the HTTP body
--request-header "Field: Value" A header to provide with the request (will override the header if also in the request header file). Can provide multiple headers.
--request-config "config: value" A config setting for the TNLRequestConfiguration of the request (will override the config if also in the request config file). Can provide multiple configs.
--request-method <method> HTTP Method from Section 9 in HTTP/1.1 spec (RFC 2616), such as GET, POST, HEAD, etc
--response-body-mode <mode> "file" or "print" or a combo using commas
--response-body-file <filepath> file for the response body to save to (requires "file" for --response-body-mode
--response-headers-mode <mode> "file" or "print" or a combo using commas
--response-headers-file <filepath> file for the response headers to save to (as json)
--dump-cert-chain-directory <dir> directory for the certification chain to be dumped to (as DER files)
--verbose Will print verbose information and force the --response-body-mode and --responde-headers-mode to have "print".
--version Will print ther version information.Copyright 2014-2020 Twitter, Inc.
Licensed under the Apache License, Version 2.0: https://www.apache.org/licenses/LICENSE-2.0
Please report sensitive security issues via Twitter's bug-bounty program (https://hackerone.com/twitter) rather than GitHub.