Service Discovery Through Lambda Extension


I started to read more and more about serverless architecture and the power of AWS Lambda. Amongst other things, Lambda extensions caught my eye. In a few words, this allows you to run code before or after your Lambda executes and also during the initialization or shutdown of your Lambda. There are already quite a few extensions in the wild. The first scenario I wanted to explore was a poor-man Service Discovery Through Lambda Extension.

As always, the code associated with this experiment is available on my GitHub repository.

Lambda extensions

Lambda extensions are a relatively new thing: they were introduced at the end of 2020. In order to keep things simple, Amazon built extensions on top of layers. Basically, you’ll need a special directory structure to signal to the runtime that your layer is an extension. Then, you register your extension like any other lambda layer.

I could not find any official documentation on that directory structure and had to rely on samples to figure it out. It turns out that you will need the following structure in your ZIP for your layer:

  • extensions: this directory needs to contain the executable that will start your extension
  • myExtensionName: this directory will contain your extension code

It is worth noting that nothing forces you to use a specific language to write your extension. You could write your extension in GO and still use it in a nodejs lambda.

Basically, you can see the extension as an external process that runs in the same context as your lambda. A little like a Kubernetes Sidecar.

My actual extension

Service discovery is always a complex topic in a microservices system. Therefore the extension I wrote tries to solve this problem. It allows a deployment tool to insert services, their version, and ARN into DynamoDB. Users of the extension can then set an environment variable named  NEEDED_SERVICES on their lambda. This environment variable supports a comma-separated list of services. It even handles a quick notation  MyService==1.2.0 to get a specific version and not the latest one.

When the extension is initialized, it does a few things:

  1. Read the environment variable to get the needed services
  2. Get the ARNs for the services from DynamoDB
  3. Write a file containing the discovered services
  4. The lambda can then read the file and knows how to contact the other services

Of course, none of this is usable as-is and is simply me playing. There are better ways of doing the same thing as this extension is doing.

Writing the extension

Entry point

The entry point of your extension is an executable file found in the extensions/ directory of your layer’s ZIP file. Make it easy for yourself and name it the same as the directory containing your code. A sample of this executable can be found here. The name of this executable will be important later, choose it well. Remember to set that file as executable, else your extension won’t work.

Registering the extension

One of the first things to do in your extension is to register it to the lambda service. This is done through an HTTP call. For example, the register call in my extension can be found here. There are two points to note in this call:

  • The events types to listen for, the only valid values are  INVOKE and SHUTDOWN.
  • Most importantly, the  Lambda-Extension-Name header needs to be exactly the same string as your executable from the extensions directory

If you are registering for SHUTDOWN events and are getting errors like ShutdownEventNotSupportedForInternalExtension: you haven’t passed the right extension name in the register call.

Polling for events

Once your extension is registered, you will be able to poll for events associated with it. Doing so is, again, done through an HTTP call and can be found here in my playground. You can then do a good old  while ... true to poll and process the events, like here. In my example, I do not care about invocations. Therefore, only the shutdown event is handled.

Remember to be a good neighbor and handle SIGINT and SIGTERM. These two signals could be received in some cases and you should shut down your extension. Handling process signals in nodejs is done trivially using process.on(signal, function), for example: here.

Using the extension

Amazon made it really easy to use extensions: they are configured as layers on a lambda. Therefore, you can add an extension really easily through any deployment tool. In my case, I decided to use serverless, so adding the layer is done when declaring the function, see this section.

Related links

Tagged , . Bookmark the permalink.

Leave a Reply