Deploying Golang Code to AWS Lambda on Windows

Since January 2018 AWS Lambda supports go as a first-class citizen. That’s good news for engineers that love the simplicity and power of Pike’s and Thompson’s baby. However, things are sometimes more complicated than they should when you work on Windows. The usual deployment strategy for AWS Lambda is to wrap your code with all the 3rd party dependencies in a zip archive, upload that bundle to S3 and point Lambda function resource to it. Go has a valuable benefit – when compared to scripting languages – of being able to produce a statically-linked binary which you can just drop on a target machine and run. No hassle with virtualenv/pip, bundler or npm. So when it comes to Lambda deployment of your go code, what do you do? Yeah, right: compile, zip and upload. Well, that’s not the whole story.

You need to be sure that your binary will work on the target platform (which is an Amazon Linux). Ok, no problem: you can cross-compile on Windows for Linux. Done? Unfortunately not.

Remember what is the next step for Lambda function deployment? Archiving. You need to wrap your binary in a zip file before you upload it to S3. And here comes the surprise. Once you package your cross-compiled binary with a zip archiver on Windows, you’ll have a file without execution bits set and it will not run once deployed to AWS. I’ve spent quite some time trying to find a zip archiver that could allow me to set linux file permissions manually, with no luck. However, there are smart folks out there that have already solved this issue. Python to the rescue.

The way I deploy to AWS is by using CloudFormation. I use make as a task runner, so below is a makefile snippet that illustrates how I deal with compiling & archiving go lambda code. And yes, once again: this is for Windows.

# NOTE: python *3* code
lambda/target/%.zip: ${GOPATH}/src/github.com/aws/aws-lambda-go ${GOPATH}/src/github.com/aws/aws-sdk-go
	GOOS=linux GOARCH=amd64 go build -o $(subst .zip,,$@) $(subst .zip,.go,$@)
	python -c "import time; import zipfile as zf; z = zf.ZipFile('$@', mode='w'); \
	  zi = zf.ZipInfo('$(subst .zip,,$@)'); \
	  zi.external_attr = 0o0755 << 16 ; \
	  zi.date_time = time.localtime() ; \
	  zi.create_system = 3 ; \
	  z.writestr(zi, open('$(subst .zip,,$@)', mode='rb').read(), zf.ZIP_DEFLATED)"

Ok let’s dissect this piece of code now line by line.

lambda/target/%.zip: ${GOPATH}/src/github.com/aws/aws-lambda-go ${GOPATH}/src/github.com/aws/aws-sdk-go

This line declares that the result of this makefile target is a zip file in lambda/target directory. Prerequisites for running are two directories with library code within GOPATH.

GOOS=linux GOARCH=amd64 go build -o $(subst .zip,,$@) $(subst .zip,.go,$@)

This instructs go compiler to cross-compile for 64-bit Linux platform. The subst magic manipulates with file extensions.

$(subst .zip,,$@)

takes $@ (which would be a full target name like lambda/target/my_go_lambda.zip) and removes .zip extension. Much like

echo 'lambda/target/my_go_lambda.zip' | sed 's/\.zip//'

So the first subst provides an argument for the -o commandline option, that is, the name of the binary we’re going to produce. Second subst replaces .zip extension with .go. As you can guess, I follow a rule: one lambda function equals to one go sourcefile. And if I have my_go_lambda.go, it will be compiled to my_go_lambda and archived to my_go_lambda.zip.

python -c "import time; import zipfile as zf; z = zf.ZipFile('$@', mode='w'); \

ZipFile is the hero.

z = zf.ZipFile('$@', mode='w')

First thing is to open a file with ‘write’ mode. $@, again, is the name of this makefile target (something like lambda/target/my_go_lambda.zip). So we create an empty zipfile here.

zi = zf.ZipInfo('$(subst .zip,,$@)')

ZipInfo is the class that is used to add files to an archive. Here, we instantiate an object of this class for our compiled binary file.

zi.external_attr = 0o0755 << 16

This is the crucial part: file attributes. Oo prefix means we have an octal number, because this is the way linux file permissions are usually represented. Zip format, being cross-platform, supports MS-DOS and UNIX attributes as external file attributes.

Your usual windows zip archiver will simply ignore the UNIX part while creating archives, but here we are explicitly setting those UNIX permissions bits. The << 16 thing just adds enough zeroes to our 0755 octal number so that it’s positioned correctly in a 32-bits external attributes field. 0755, by the way, means rwxr-xr-x when represented in a human-readable way.

zi.date_time = time.localtime()

Setting file’s last modification time explicitly.

zi.create_system = 3

This line sets a bit in zip file’s metadata that denotes which OS the archive originates from. Zip format allows you to pick from 20 different OSes, among them – already extinct CP/M and a bunch of mainframe systems like MVS and OS/400. In our case number 3 means UNIX. To my experience, this field is important for the unpacking to work correctly.

z.writestr(zi, open('$(subst .zip,,$@)', mode='rb').read(), zf.ZIP_DEFLATED)"

And finally, we write our ZipFile object to a file. ZIP_DEFLATED demands compression (it is also possible to store uncompressed files inside a zip archive, just like with tar).

Thank you for reading!