Creating an API Client
Creating an API Client
Using a client we can create a dynamic connection between the Connector and your API. In this step we’ll build an HTTP client so the data objects provided to the Connector will be from your system.
Key considerations
The data your client gets from the API should look similar to the data objects needed for your connector, but if the data does not match precisely, we can map the API data to the exact data object in the Data Reader.
Your client code will be executed by the App Xchange platform and as such will need to have access to your API.
CLI Command Options
Before running the CLI command to create a client, you have to know about the existing options it uses.
-
--type (aliases: -t)
- It indicates the type of client to create. HTTP is used by default
- Required: False
-
--auth-type (aliases: --auth)
- It indicates the type of authentication to use. Possible values include
ApiKey,Basic,Custom,OAuth2CodeFlow,OAuth2ClientCredentialsandOAuth2Password. - Required: True
- It indicates the type of authentication to use. Possible values include
If your client needs ApiKey, Basic, OAuth2CodeFlow, OAuth2ClientCredentials or OAuth2Password authentication, use the --auth-type option with the specific authentication type. If you need a type of authentication that is not listed, you can use Custom to implement what you need. Read more at Auth Types
The CLI command:
xchange client new --type Http --auth-type {YOUR_AUTH_TYPE}
While in the root of your Connector project run the CLI command to create a new client. Depending on which --auth-type was specified, different files will be created (the files for each case will be explained in detail shortly). A new Client directory and a client named ApiClient.cs will be created. This will also add the client to your dependency injection found in the ConnectorRegistration file.
The client will by default come with a retry policy. Policies can be removed or added as needed, more on this can be found in the Custom Policies section.
Defining Connections for Authentication
When a connector builder using the App Xchange CLI tools wants to create a client with authentications, the connector will need the proper credentials to create valid requests, as well as a handler that adds the authentication headers the requests may need.
All the configuration needed to get authenticated requests will be contained in an AuthType class, and, depending on the type of authentication, its respective handler will take care of authenticating the requests. It is important to include everything for unique authentication in the connections section.
About AuthType classes
The AuthType class contains what the client will use in order to get authenticated requests. The AuthType class will contain the secrets, as well as the Connection Environment and Base Url the secret belongs to.
Depending on the type of authentication the API used in the client needs, a different AuthType class will be created.
Currently, there are 6 types of authentication whose creation is supported: ApiKeyAuth, BasicAuth, CustomAuth, OAuth2CodeFlow, OAuth2ClientCredentials and OAuth2Password. This page provides a generic overview, see Auth Types for details.
When the AuthType class is created, it includes a property called ConnectionEnvironment. This property will have the configuration of the possible environments. Depending on the ConnectionEnvironment, the BaseUrl will be set.
Upon creation, the BaseUrl will be filled with placeholders. Be sure to remove those and use your actual URLs. If you only have one baseUrl, please remove the switch statement and just set the baseUrl to avoid an unnecessary dropdown.
Connection definition classes are decorated with an attribute called ConnectionProperty. This attribute is necessary for the property to show up in the extracted metadata of the connector.
Key considerations
Currently, only properties of type string are supported.
About AuthType Handlers
To establish authentication in a transaction, an AuthType Handler will be created with the authentication logic you need. The AuthType Handler is a DelegatingHandler that decides how the requests should behave. Depending on the type of authentication needed, it adds the headers with its values.
The AuthType Handler exist in the Client folder, and connector builders can customize it depending of the API they're working with, such as by modifying the headers.
Working with connections
If you created a client with an authentication type, you will have two new folders in your repository, named Client (with the ApiClient and the AuthType handler) and Connections (with the AuthType class and a ConnectionTestHandler class). It will also implement IConfigureConnectorApiClient in the ConnectorRegistration file, where the injections needed for the client will be added. Note that the TestConnection method will not have access to the Connector Registration Config, so ensure everything needed to authenticate and implement that method is included as a connection property.
Additionally, the generated ApiClient class will implement an interface called ITargetSystemApiClient. This interface is implemented so that the CLI can identify which class is the intended API Client class needed for injection in subsequent code files.
Example of ApiClient
public class ApiClient : ITargetSystemApiClient
{
private readonly HttpClient _httpClient;
public ApiClient (HttpClient httpClient, string baseUrl)
{
_httpClient = httpClient;
_httpClient.BaseAddress = new System.Uri(baseUrl);
}
public async Task<ApiResponse<JsonDocument>> GetExampleRecord(CancellationToken cancellationToken = default)
{
var response = await _httpClient
.GetAsync("example", cancellationToken: cancellationToken)
.ConfigureAwait(false);
return new ApiResponse<JsonDocument>
{
IsSuccessful = response.IsSuccessStatusCode,
StatusCode = (int)response.StatusCode,
Result = response.IsSuccessStatusCode ? await response.Content.ReadFromJsonAsync<JsonDocument>(cancellationToken: cancellationToken) : default,
RawResult = await response.Content.ReadAsStreamAsync(cancellationToken: cancellationToken)
};
}
}
Example of ConfigureConnectorApiClient and RegisterConnectionTestHandler
public void ConfigureConnectorApiClient(IServiceCollection serviceCollection, IHostConnectionContext hostConnectionContext)
{
var activeConnection = hostConnectionContext.GetConnection();
serviceCollection.ResolveServices(activeConnection);
}
public void RegisterConnectionTestHandler(IServiceCollection serviceCollection)
{
serviceCollection.AddSingleton<IConnectionTestHandler, ConnectionTestHandler>();
}
RegisterConnectionTestHandler applies to version 1.2.3 or above with the introduction of Connections
Another file that will be created is the ClientHelper, which will help resolve what we need in case there's multiple connection definitions. Depending on the definitionKey of the active connection, it will inject the dependencies to support the AuthType in use. It will also instantiate the ApiClient, using the configuration of the connector and the Base Url of the active connection.
Example of a ClientHelper with ApiKey
public static class ClientHelper
{
public static class AuthTypeKeyEnums
{
public const string ApiKeyAuth = "apiKeyAuth";
}
public static void ResolveServices(this IServiceCollection serviceCollection, ConnectionContainer activeConnection)
{
serviceCollection.AddTransient<RetryPolicyHandler>();
var options = new JsonSerializerOptions
{
PropertyNamingPolicy = JsonNamingPolicy.CamelCase
};
switch (activeConnection.DefinitionKey)
{
case AuthTypeKeyEnums.ApiKeyAuth:
var configApiKeyAuth = JsonSerializer.Deserialize<ApiKeyAuth>(activeConnection.Configuration, options);
serviceCollection.AddSingleton<IApiKeyAuth>(configApiKeyAuth);
serviceCollection.AddTransient<RetryPolicyHandler>();
serviceCollection.AddTransient<ApiKeyAuthHandler>();
serviceCollection.AddHttpClient<ApiClient, ApiClient>(client => new ApiClient(client, configApiKeyAuth.BaseUrl))
.AddHttpMessageHandler<ApiKeyAuthHandler>()
.AddHttpMessageHandler<RetryPolicyHandler>();
break;
default:
throw new Exception($"Unable to find services for definition key {activeConnection.DefinitionKey}");
}
}
}
Example of a ClientHelper with OAuth2ClientCredentials
public static class ClientHelper
{
public static class AuthTypeKeyEnums
{
public const string OAuth2ClientCredentials = "oAuth2ClientCredentials";
}
public static void ResolveServices(this IServiceCollection serviceCollection, ConnectionContainer activeConnection)
{
serviceCollection.AddTransient<RetryPolicyHandler>();
var options = new JsonSerializerOptions
{
PropertyNamingPolicy = JsonNamingPolicy.CamelCase
};
switch (activeConnection.DefinitionKey)
{
case AuthTypeKeyEnums.OAuth2ClientCredentials:
var configOAuth2ClientCredentials = JsonSerializer.Deserialize<OAuth2ClientCredentials>(activeConnection.Configuration, options);
serviceCollection.AddSingleton<OAuth2ClientCredentialsBase>(configOAuth2ClientCredentials!);
serviceCollection.AddTransient<RetryPolicyHandler>();
serviceCollection.AddTransient<OAuth2ClientCredentialsHandler>();
serviceCollection.AddHttpClient<ApiClient, ApiClient>(client => new ApiClient(client, configOAuth2ClientCredentials!.BaseUrl))
.AddHttpMessageHandler<OAuth2ClientCredentialsHandler>()
.AddHttpMessageHandler<RetryPolicyHandler>();
break;
default:
throw new Exception($"Unable to find services for definition key {activeConnection.DefinitionKey}");
}
}
}
If another AuthType class is created, the case statement with the dependency injections needed will be added to the file.
Testing a Connection
NOTE: Applies to version
1.2.3-beta.2or above
When a connection is created, we need a way to test the connection to see if it works before we decide to use it. For that purpose, the CLI will also generate a class that implements IConnectionTestHandler called ConnectionTestHandler in the Connections folder. This class will implement a method called TestConnection, in which a test process for the connection should be implemented.
The ConnectionTestHandler generated by the CLI will look like this.
Example of an implemented ConnectionTestHandler
public async Task<TestConnectionResult> TestConnection()
{
// Make a call to your API/system to obtain the connection test result.
var response = await _apiClient.TestConnection();
// Depending on the response, make your own specific messages.
if (response == null)
{
return new TestConnectionResult()
{
Success = false,
Message = "Failed to get response from server",
StatusCode = 500
};
}
if (response.IsSuccessful)
{
return new TestConnectionResult()
{
Success = true,
Message = "Successful test.",
StatusCode = response.StatusCode
};
}
switch (response.StatusCode)
{
case 403:
return new TestConnectionResult()
{
Success = false,
Message = "Invalid Credentials: Forbidden.",
StatusCode = response.StatusCode
};
case 401:
return new TestConnectionResult()
{
Success = false,
Message = "Invalid Credentials: Unauthorized",
StatusCode = response.StatusCode
};
default:
return new TestConnectionResult()
{
Success = false,
Message = "Unknown Issue.",
StatusCode = response.StatusCode
};
}
}
The method TestConnection() should make a call to the ApiClient and it could use a simple GET endpoint to something more complicated; the purpose of the _apiClient.TestConnection() method is to ensure the connector can make authenticated calls to the API.
Depending on the response of the ApiClient call, Success should indicate if the test as successful, the StatusCode of the response should be set, and, if the test is unsuccessful, a Message should be set in order to give more information.
For running your connection test locally, follow the next guide: Testing A Connector Locally.
In summary what is being done is:
- Given an AuthType, a class and a handler will be created and have its dependencies injected.
- The ApiClient will receive the BaseUrl from the AuthType class configured.
- Since multiple Auth Types are supported, the ClientHelper will help resolve the dependencies needed for the active connection.
- A handler for testing a connection is created to ensure the connector can perform as needed with the given connection.
Custom Policies
The SDK comes with Polly. We use Polly for all of our HTTP pipeline policies; by default, all HTTP pipelines have a RetryPolicyHandler on them. Polly can be used for things other than retry policies or HTTP.
Retry Policy
By default there is a retry policy added to the generated API client's HTTP pipeline. This retry policy is named RetryPolicyHandler. It has the following policy:
- Retries up to 3 times on:
- Transient HTTP status codes:
- 5xx
- 408
- HTTP Request Exceptions
- Transient HTTP status codes:
- Each attempt will have an exponential backoff
- Attempt 1: 1 second
- Attempt 2: 4 seconds
- Attempt 3: 9 seconds
With this policy this means that there will be a total of 4 attempts (initial attempt + the policy's retries).
Info
It's important to be aware of that all 4 of these requests happen within the same cancellation token. By default HTTP clients come with a timeout of 100 seconds. So after 100 seconds the token will be canceled. Look at timeout policies for guidance.
Custom Retry Policy
You may want to have your own retry logic for your Connector. This section will explain how to create a policy and how to utilize it. Starting with the policy creation, below is a policy that handles all transient errors plus 429 status codes.
Example of a retry policy with 429 support
public class CustomRetryPolicy : DelegatingHandler
{
private const int RetryTimes = 3;
private static readonly TimeSpan MaxRetryAfter = TimeSpan.FromMinutes(2);
private static TimeSpan Jitter() => TimeSpan.FromMilliseconds(Random.Shared.Next(0, 250));
protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
var policy = GetRetryPolicy(RetryTimes);
return policy.ExecuteAsync(policyCancellationToken => base.SendAsync(request, policyCancellationToken), cancellationToken);
}
private static AsyncRetryPolicy<HttpResponseMessage> GetRetryPolicy(int retryTimes)
{
return HttpPolicyExtensions
.HandleTransientHttpError()
.OrResult(r => r.StatusCode == HttpStatusCode.TooManyRequests)
.WaitAndRetryAsync(
retryTimes,
(attempt, outcome, _) =>
{
if (outcome.Result is { StatusCode: HttpStatusCode.TooManyRequests } response)
{
var header = response.Headers.RetryAfter;
if (header != null)
{
TimeSpan? specified = null;
if (header.Delta.HasValue) specified = header.Delta.Value;
else if (header.Date.HasValue)
{
var calc = header.Date.Value - DateTimeOffset.UtcNow;
if (calc > TimeSpan.Zero) specified = calc;
}
if (specified.HasValue)
{
if (specified.Value > MaxRetryAfter) specified = MaxRetryAfter;
return specified.Value + Jitter();
}
}
}
var backoff = TimeSpan.FromSeconds(Math.Pow(2, attempt));
return backoff + Jitter();
},
onRetryAsync: (_, _, _, _) => Task.CompletedTask);
}
}
The policy above also supports the retry-after header convention. However, the delay may be longer than the timeout of the HTTP client. A timeout policy can help here; a more robust solution would be to implement a circuit breaker. Polly does provide APIs to build one if needed.
To use this policy you will need to update the HTTP pipeline construction in client/ClientHelper.cs
client/ClientHelper.cs updates
// ... rest of code
switch (activeConnection.DefinitionKey)
{
case AuthTypeKeyEnums.BasicAuth:
var configBasicAuth = JsonSerializer.Deserialize<BasicAuth>(activeConnection.Configuration, options);
serviceCollection.AddSingleton<IBasicAuth>(configBasicAuth!);
serviceCollection.AddTransient<CustomRetryPolicy>();
serviceCollection.AddTransient<BasicAuthHandler>();
serviceCollection
.AddHttpClient<ApiClient, ApiClient>(client => new ApiClient(client, configBasicAuth!.BaseUrl))
.AddHttpMessageHandler<BasicAuthHandler>()
.AddHttpMessageHandler<CustomRetryPolicy>();
break;
// ... rest of code
}
Timeout Policy
You may have cases where requests can take a long time or network issues cause a request to take an indefinite amount of time. In those cases you can wrap your delegating handlers in a timeout policy, so that if a request is taking too long it can be canceled and retried.
For example: in the case where a target system's network gateway has issues such that it will not respond to a connection at times, e.g. a connection is open but not doing anything. We'll want to close out that connection and try again. In order to do so we need to cancel out the ongoing request and let the inner retry policy attempt it again. We can cancel that request via a timeout policy.
Example of a timeout policy
public class TimeoutPolicyHandler : DelegatingHandler
{
private readonly int _timeoutInSeconds;
public TimeoutPolicyHandler()
{
_timeoutInSeconds = (int)TimeSpan.FromMinutes(3).TotalSeconds;
}
public TimeoutPolicyHandler(TimeSpan timeout)
{
_timeoutInSeconds = (int)timeout.TotalSeconds;
}
protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
var policy = GetRetryPolicy(_timeoutInSeconds);
return policy.ExecuteAsync(token => base.SendAsync(request, token), cancellationToken);
}
private static AsyncTimeoutPolicy<HttpResponseMessage> GetRetryPolicy(int timeoutInSeconds)
{
return Policy.TimeoutAsync<HttpResponseMessage>(timeoutInSeconds, TimeoutStrategy.Optimistic);
}
}
To use this policy the following updates need to be done on the client/ClientHelper.cs file:
client/ClientHelper.cs updates
// ... rest of code
switch (activeConnection.DefinitionKey)
{
case AuthTypeKeyEnums.BasicAuth:
var configBasicAuth = JsonSerializer.Deserialize<BasicAuth>(activeConnection.Configuration, options);
serviceCollection.AddSingleton<IBasicAuth>(configBasicAuth!);
serviceCollection.AddTransient<RetryPolicyHandler>();
serviceCollection.AddTransient<BasicAuthHandler>();
serviceCollection.AddTransient<TimeoutPolicyHandler>();
serviceCollection
.AddHttpClient<ApiClient, ApiClient>(client => new ApiClient(client, configBasicAuth!.BaseUrl))
.AddHttpMessageHandler<BasicAuthHandler>()
.AddHttpMessageHandler<RetryPolicyHandler>()
.AddHttpMessageHandler<TimeoutPolicyHandler>();
// Example for configuring timeout at client level instead of global
// .AddHttpMessageHandler(_ => new TimeoutPolicyHandler(TimeSpan.FromMinutes(3)));
break;
// ... rest of code
}
Info
Be aware that policies are applied in the reverse order that you add them using .AddHttpMessageHandler<>.
The last thing to do is update the ApiClient's timeout to be something higher than the timeouts + retry policy. This is done by:
client/ClientHelper.cs updates
public ApiClient (HttpClient httpClient, string baseUrl)
{
_httpClient = httpClient;
_httpClient.BaseAddress = new System.Uri(baseUrl);
_httpClient.Default
// 4 total attempts each with a 3 minute timeout + delays between retries and some buffer
_httpClient.Timeout = TimeSpan.FromMinutes(13);
}
Endpoint Specific Policies
Sometimes there are endpoints on the target system that are asynchronous. Such that when you make a POST to create a resource, doing a GET afterwards may return a 404 or a 202. Instead of introducing a static delay in the code, a policy can be put in place to keep trying to GET until the record returns. This allows for a more responsive Connector due to it being able to dynamically wait instead of a static wait.
Here's an example of creating a policy to handle an endpoint that may return back a 202 until the data is ready, and once ready returns a 200.
client/ApiClient.cs updates
public class ApiClient : ITargetSystemApiClient
{
private readonly HttpClient _httpClient;
private readonly AsyncRetryPolicy<HttpResponseMessage> _waitUntilReadyPolicy;
public ApiClient(HttpClient httpClient, string baseUrl)
{
_httpClient = httpClient;
_httpClient.BaseAddress = new Uri(baseUrl);
_waitUntilReadyPolicy = HttpPolicyExtensions
.HandleTransientHttpError()
.OrResult(resp => resp.StatusCode == HttpStatusCode.Accepted)
.OrResult(resp => resp.StatusCode == HttpStatusCode.NotFound)
.Or<TimeoutRejectedException>()
.WaitAndRetryAsync(20, _ => TimeSpan.FromMilliseconds(100));
}
public async Task<ApiResponse<T>> GetDelayedCreationRecord<T>(
string relativeUrl,
CancellationToken cancellationToken = default)
{
var response = await _waitUntilReadyPolicy.ExecuteAsync(
async policyCancellationToken => await _httpClient
.GetAsync(relativeUrl, cancellationToken: policyCancellationToken)
.ConfigureAwait(false),
cancellationToken);
return new ApiResponse<T>
{
IsSuccessful = response.IsSuccessStatusCode,
StatusCode = (int)response.StatusCode,
Data = response.IsSuccessStatusCode ?
await response.Content.ReadFromJsonAsync<T>(cancellationToken: cancellationToken)
: default,
RawResult = await response.Content.ReadAsStreamAsync(cancellationToken: cancellationToken)
};
}
}