Talk

Serverless from scratch with CI/CD on AWS: it’s a lot easier than you think (and mostly free).

Friday, May 26

11:40 - 12:10
RoomRisotto
LanguageEnglish
Audience levelIntermediate
Elevator pitch

We’ll cover the fundamentals of the Serverless approach to running code in AWS, deploying a FastAPI web application, with CI/CD, quickly, using the Infrastructure as Code tool Terraform. Crucially, we’ll cover the fundamentals of Serverless AWS, and make comparisons to server-based approaches.

Abstract

In this talk, we’ll learn about the Infrastructure as Code (IaC) tool Terraform - how, and crucially, why, to use it with python and AWS. We’ll then take a tour of how to use terraform to build a lightweight, mostly-free, serverless setup inside AWS, that allows you to get full stack web applications up and running quickly. What’s more, because you’re managing your own infrastructure with repeatable patterns, you are not tied into more opinionated solutions like Heroku, or set frameworks, and you can use this IaC approach to scale web and other cloud apps from any size - from fun side projects, to multi-million euro commercial applications with thousands of requests-per-second. Crucially though, we’ll cover the fundamentals of the various components in the AWS Serverless ecosystem, and compare them to how a more conventional web app might work on your laptop, or a more traditional server setup, to demystify Serverless technologies, and make other frameworks more transparent and easier to use.

All code has to run, somewhere. Typically that ‘somewhere’ will be your laptop, followed by a server. At the most basic level, maybe you copy your code into a free-tier server somewhere in “the cloud”, or a Raspberry Pi in your basement, set it to run in the background, and then you’re sorted: your API running a framework like FastAPI is running and you can start making queries. However, if the code is something you care about, you’re probably going to want to use a version control system like git, and related tools like GitHub, and implement continuous integration/deployment, meaning your code is automatically deployed without the need for fiddly manual steps. What’s more, you’re likely to need to use other services such as a database, queues, background jobs, or caches.

In the 2010s, tools like Heroku shone in allowing users to quickly and easily deploy these projects to the cloud, without having (much) to worry about provisioning or managing servers. However, in recent years, the appeal, and cost advantages of such tools has waned. Meanwhile, a new class of cloud technology has emerged - the “serverless” cloud.

You may be familiar with the archetypal piece of serverless infrastructure, the AWS Lambda Function. These are easy to create, deploy, and play around with, and you can get them up and running and connected to other services by following tutorials online, simply clicking around in the AWS Web Console, generally with minimal swearing as you mutter under your breath about things like “IAM policies” and “invocation ARNs” that were correct when you copied then 10 minutes ago, but have since, subtly changed. This “ClickOps” approach is great if you deploy your infrastructure once and then either never change it, or leave your company and make it someone else’s problem, but if you want to manage infrastructure and deploy your python code cheaply, we need a better approach to using serverless technologies with Python.

This is where Terraform comes in. Terraform is an Infrastructure as Code (IaC) tool that allows you to define all your infrastructure inside a configuration language called HCL. In this talk, I’ll explain why you should always do this, and stop making any changes to your cloud resources through the AWS Console (probably). We’ll first off take a quick tour of the history and philosophy of IaC, before delving into an open source repo accompanying the talk, which shows you how you can deploy a full stack serverless web application using FastAPI, with a full CI/CD pipeline, a relational database including migrations, and for added bonus points, a front end, for as little as a Euro or so a month inside AWS, often less.

The talk will also focus on the fundamentals of how each serverless cloud component works, by comparison to a more conventional FastAPI setup running simply on your laptop. What’s more, we’ll also explore some of the limitations of this approach, and see how you may wish to start scaling up this approach as your service grows, or alternatively, keep costs as low as possible, as your little fun side project continues to provide you with joy, but at almost no cost (so you can forget about it slowly, as you start several others).

TagsAPIs, AWS, Microservices
participant photo

Dom Weldon

Dom is a full stack cloud software engineer based in central London. He works as an independent contractor, primarily in financial services, and is a Principal Associate at Decision Lab, a consultancy with expertise in simulation, optimisation, and machine learning. Dom’s primary expertise are in Python and Javascript, graph and relational databases, and IaC in AWS using Terraform. Dom studied at the University of Cambridge and King’s College London.