Handling binary content with API Gateway

This tutorial shows how to configure API Gateway using Claudia API Builder to send or receive binary files.


Basics of binary handling with API Gateway

Since November 2016, API Gateway has basic support for binary transfers. It can transform incoming binary files into base64 encoded strings before passing on to a Lambda function, and transform base64 encoded Lambda resutls into binary files.

Incoming request are transformed into base64 strings if:

Results are transformed into binary content from base64 if:

Lambda integration quirks

Although the API Gateway documentation suggests otherwise, it seems that for Lambda integrations, setting the incoming request content handling to 'CONVERT_TO_TEXT' is not necessary, as any matching binary type is automatically encoded into base64 for AWS Lambda integrations. You can set it to 'CONVERT_TO_BINARY' to prevent the transformation.

Similarly, although the documentation suggests that multiple content types can be specified in the Accept header for binary responses, it seems that this breaks the conversion. This makes the current implementation useless for browsers, which by default request complex Accept headers. This means that it’s currently not possible to use the API Gateway/AWS_PROXY integration to return images that can be just included into a web page using the img tag.

How Claudia API Builder helps

Claudia API Builder makes it easier to handle binary content by doing several things automatically for you:

### Configuring an API


api.post('/thumb', (request) => {
}, { 
  requestContentHandling: 'CONVERT_TO_TEXT', 
  success: { 
    contentType: 'image/png', 
    contentHandling: 'CONVERT_TO_BINARY' 

## Responding with binary content

To respond with binary content, make sure to set the response success.contentHandling to 'CONVERT_TO_BINARY', and return a content type matching the configured binary types. Claudia API Builder will automatically convert a binary buffer to a base64 string. Here is how you can serve a binary file to API clients – Note that we’re directly using the results of fs.readFile (a Promise version), which provides a buffer.

api.get('/img', () => {
	'use strict';
	return fs.readFilePromise(path.join(__dirname, 'img.png'));
}, { success: { contentType: 'image/png', contentHandling: 'CONVERT_TO_BINARY'}});

To retrieve this file, you must set the Accept header to image/png:

curl https://<API-ID>.execute-api.us-east-1.amazonaws.com/latest/img -H "Accept: image/png" > 1.png

Requesting with binary content

You can also use the binary support to process incoming binary files. Claudia API Builder will populate request.body with a binary buffer automatically in case of base64 encoded binary content.

For example, we can use the ImageMagick identify tool to get basic information about image files. This example will save the incoming file, execute identify, clean up, and return the result. Note that we’re directly using fs.writeFile (a Promise version) to save the request body into a temporary file, because it is a binary buffer. No specific API endpoint configuration is needed here.

api.post('/info', (request) => {
	'use strict';
	const tempFileName = path.join(os.tmpdir(), request.lambdaContext.awsRequestId);
	let result;
	return fs.writeFilePromise(tempFileName, request.body) // dump the request body into a temporary file
		.then(() => childProcessPromise.spawn('/usr/bin/identify', [tempFileName])) // execute `identify` and return the response
		.then(picInfo => result = picInfo.replace(/[^\s]*\s/, '')) // strip the temp file name out of the result
		.then(() => fs.unlinkPromise(tempFileName)) // remove the temporary file
		.then(() => result);
}, { success: { contentType: 'text/plain' } });

To use this service, make sure to include the Content-Type header into your request.

curl --request POST -H "Content-Type: image/png" --data-binary "@img.png" https://<API-ID>.execute-api.us-east-1.amazonaws.com/latest/info

Using binary content for both request and response

We can combine both techniques to create a service for thumbnails. The endpoint will receive an image, resize it, and return the result. In this case, we’ll need to configure the response content handling. Note that we can directly store request body into a file (Claudia API builder will process and convert the incoming body into a binary buffer), and that we can directly return the results of a binary file read (Claudia API Builder will convert it into a base64 string).

api.post('/thumb', (request) => {
	'use strict';
	const tempFileName = path.join(os.tmpdir(), request.lambdaContext.awsRequestId),
		thumbFileName = tempFileName + '-thumb.png';
	let result;
	return fs.writeFilePromise(tempFileName, request.body) // save the body into a temporary file
		.then(() => childProcessPromise.spawn('/usr/bin/convert', ['-resize', '150x', tempFileName, thumbFileName])) // create thumbnail
		.then(() => fs.readFilePromise(thumbFileName)) // read out the conversion results
		.then(fileContents => result = fileContents) // store into a variable for later
		.then(() => fs.unlinkPromise(tempFileName)) // remove temporary files
		.then(() => fs.unlinkPromise(thumbFileName))
		.then(() => result); // return the file contents
}, { success: { contentType: 'image/png', contentHandling: 'CONVERT_TO_BINARY' } });

To use this service, you’ll have to set both the request content type and the accepted response content type:

curl --request POST -H "Content-Type: image/png" -H "Accept: image/png" --data-binary "@img.png" https://<API-ID>.execute-api.us-east-1.amazonaws.com/latest/thumb > thumb.png

To see this in action, check out the Binary Content Handling example project.

Did you like this tutorial? Get notified when we publish the next one.

Once a month, high value mailing list, no ads or spam. (Check out the past issues)