There are several ways to get the latest codebase uploaded into your testing and production servers. Manually copying them over is fine and all, but having something take care of running tests and deploying the code will save your team countless hours of manual labor.
This will also lessen the chance of human error. This article covers some steps you can take to add continuous integration and deployment (CI/CD) to the serverless project created in Part 1 of the AWS Serverless App, but the actions can be applied to any serverless project. For a more detailed breakdown of building a Serverless API on AWS, check out the book by the author, Building Serverless Node.js Apps on AWS.
Building Serverless Node.js Apps on AWS: A Brief Guide on Where to Start
Welcome to Part 2 of the AWS Serverless App tutorial set. By now, you should already have a serverless project you’ve been working with. If not, feel free to fork or clone the sample drug-search project from GitHub from Part 1 of the tutorial to move forward.
We’ll be wiring up our projects with Travis CI and Codeship, to give you an idea of how both work. Feel free to follow whichever one works best for you. If your project is an open-source project, Travis CI and CircleCI both provide free builds.
We’ll be adding the following:
To get started, create your account with whichever CI/CD service provider you prefer, and hook it up to your GitHub account. The account will need access to your repositories. You can either allow access to all by default or add them individually (recommended to reduce potential charges).
If you have a serverless project in GitHub already, go ahead and add that. If you cloned the same drug search project, you will need to add it to the listing of repositories for the CI/CD service. We’ll be adding the rest momentarily.
You’ll also need an AWS service account or user account in order to perform the deployments.
There are several ways you can leverage CI/CD with your team — one being the verification of feature branches prior to a merge.
When a developer pushes up their changes, you can configure your CI/CD build to verify code formatting with a linter, run unit tests, and check its ability to start. In the case of your serverless app, the last two are a definitive way to know whether the feature build will break your main branch and live resources prior to deployment.
Upon a merge into the master or main branch, the service can run through the linter and perform unit tests again, but you can also get your service to deploy to your project wherever it’s being hosted (AWS, in our case).
Go into your project settings, and in the Tests tab, update your Setup Commands to include the following:
This will install serverless globally to execute your tests and to allow for deployment. It’ll also install the dependencies for the project.
Next, add npm test into the test-configuration box:
Remember the service or user account we created in Part 1? If you haven’t created one yet, now will be the time.
Once you create it, grab the AWS access key and secret generated, and add them to your environment variable section:
This is required to be able to deploy and use AWS services in your scripts.
We’ll be setting up the linter and testing foundation files next. If you don’t intend on setting up Travis, you may skip the next section.
First, enable Travis for the repository. Go into your dashboard, and click on the plus symbol. It will provide the search bar.
Start searching for the project, and flip the switch on the project:
Toggle the switch to enable the repository
Click on the settings button to provide any customizations you need, like your AWS access and secret keys for deployment and AWS services usage.
Place them in your Environment Variables section, like so:
Travis CI allows you to display in build log, but it’s not recommended
You will then add the following contents to your .travis.yml file in your project’s root directory:
language: node_js
node_js:
- "10"cache:
directories:
- node_modules
install:
- npm install -g serverless
jobs:
include:
- stage: "Tests"
name: "Unit Tests"
script:
- npm install
- npm test
- stage: deploy
name: "Deploy to AWS"
script: sls deploy -v
if: NOT type = pull_request
on:
branch: master
Code snippet from GitHub
To run serverless offline for your tests or even to perform the deployment, you will need serverless. You will need to do an npm install in order to install the dependencies needed to run the tests. The npm test will run your linter and unit tests that we’ll set up in the next sections.
Notice the deploy stage has a conditional on whether it’s the master branch (which for PRs, it’ll automatically set it like that) and whether or not its a PR. We don’t want it attempting to deploy during a PR, just run the tests. It should only deploy merges into the master branch.
We don’t have tests yet, but we can wire up a linter real quick to execute with our npm test command. Install ESLint dependencies with the following terminal command:
You will need an ESLint configuration. There are tons of examples in GitHub you can grab.
This is the one I used for the drug search project:
module.exports = {
"extends": ['eslint:recommended'], // extending recommended config and config derived from eslint-config-prettier
"plugins": ['prettier'], // activating esling-plugin-prettier (--fix stuff)
"rules": {
'prettier/prettier': [ // customizing prettier rules (unfortunately not many of them are customizable)
'error',
{
"singleQuote": true,
"trailingComma": 'none',
"useTabs": false,
"tabWidth": 4
},
],
"no-constant-condition": 2, // disallow use of constant expressions in conditions
"no-control-regex": 2, // disallow control characters in regular expressions
"no-debugger": 2, // disallow use of debugger
"no-dupe-args": 2, // disallow duplicate arguments in functions
"no-dupe-keys": 2, // disallow duplicate keys when creating object literals
"no-duplicate-case": 2, // disallow a duplicate case label.
"no-empty": 2, // disallow empty statements
"no-ex-assign": 2, // disallow assigning to the exception in a catch block
"no-extra-boolean-cast": 2, // disallow double-negation boolean casts in a boolean context
"no-extra-parens": 0, // disallow unnecessary parentheses (off by default)
"no-extra-semi": 2, // disallow unnecessary semicolons
"no-func-assign": 2, // disallow overwriting functions written as function declarations
"no-inner-declarations": 2, // disallow function or variable declarations in nested blocks
"no-invalid-regexp": 2, // disallow invalid regular expression strings in the RegExp constructor
"no-irregular-whitespace": 2, // disallow irregular whitespace outside of strings and comments
"no-negated-in-lhs": 2, // disallow negation of the left operand of an in expression
"no-obj-calls": 2, // disallow the use of object properties of the global object (Math and JSON) as functions
"no-regex-spaces": 2, // disallow multiple spaces in a regular expression literal
"no-sparse-arrays": 2, // disallow sparse arrays
"no-unreachable": 2, // disallow unreachable statements after a return, throw, continue, or break statement
"use-isnan": 2, // disallow comparisons with the value NaN
"valid-jsdoc": 2, // Ensure JSDoc comments are valid (off by default)
"valid-typeof": 2, // Ensure that the results of typeof are compared against a valid string
},
"parserOptions": {
'ecmaVersion': 2017
},
"env": {
'es6': true,
'node': true
}
};
Code snippet from GitHub
You’ll also need to update your package.json scripts listing to appear like the following:
"scripts": {
"start": "sls offline --noAuth",
"test": "NODE_ENV=test PORT=9100 ./node_modules/.bin/nyc ./node_modules/.bin/mocha --opts mocha.opts",
"offline": "sls offline start",
"precommit": "eslint .",
"pretest": "eslint --ignore-path .gitignore ."
},On your local machine, run the following to start the linter:
You may or may not find yourself with a handful of linting errors — I know I did. In order for your build to succeed, you’ll need to fix the linting issues marked in red (aka errors) or modify the ESLint config as needed to reduce the errors to zero.
As you probably already noticed, the test command also has the dependency on the NYC module used to determine test coverage, and the test command uses mocha.opts, which we don’t have.
Install NYC for code coverage:
You will need to create a mocha.opts file in the root of the project directory with the following:
--timeout 5000
--reporter list
--ui bdd
--exit
test/bootstrap.test.js
test/**/*.test.js
test/**/**/*.test.jsNow when you run npm test, it’ll attempt to give you the code coverage, but it’ll also come across a failure message. Let’s add some supporting files in the test directory.
At the root of the test directory, add the following bootstrap.test.js file:
/* eslint-disable no-undef */
const { spawn } = require('child_process');
const slsOfflineTestConfig = require('./support/slsOfflineTestConfig');let slsOfflineProcess;
before(function(done) {
// increase mocha timeout for this hook to allow sls offline to start
this.timeout(30000);
console.log('[Tests Bootstrap] Start');
startSlsOffline(function(err) {
if (err) {
return done(err);
}
console.log('[Tests Bootstrap] Done');
done();
});
});
after(function() {
console.log('[Tests Teardown] Start');
stopSlsOffline();
console.log('[Tests Teardown] Done');
});
// Helper functions
function startSlsOffline(done) {
slsOfflineProcess = spawn('sls', [
'offline',
'start',
'--noAuth',
'--port',
slsOfflineTestConfig.getSlsOfflinePort()
]);
console.log(
`Serverless: Offline started with PID : ${slsOfflineProcess.pid}`
);
slsOfflineProcess.stdout.on('data', data => {
if (data.includes('Offline listening on')) {
console.log(data.toString().trim());
done();
}
});
slsOfflineProcess.stderr.on('data', errData => {
console.log(`Error starting Serverless Offline:\n${errData}`);
done(errData);
});
}
function stopSlsOffline() {
slsOfflineProcess.kill();
console.log('Serverless Offline stopped');
}
Code snippet from GitHub
Create a new directory called support in the test directory.
Add a file called slsOfflineTestConfig.js, and place the following function into it:
module.exports.getSlsOfflinePort = () => {
return process.env.PORT || '3005';
};Now when you run the npm test command, the output should look like the following:
No errors should show here, just 0 passing and 0 coverage for now
Yikes! Looks like it’s time to write some tests. We’ll save that for the next article.
With all your latest changes and linter fixes, you now have a fair amount of changes to push up. If this isn’t on a feature branch yet, go ahead and do a git checkout -b branchName, commit your changes, and push them up to GitHub. Codeship, Travis, or whatever you are using will be triggered with the changes and attempt a build.
Note: If you didn’t set up Codeship or Travis to have this repo before you attempted to push up, it may not trigger a build as expected.
If you create a pull request, your page will show the status of the build similar to the image below:
The following is what your Codeship will look like when it begins the build:
Similarly, Travis CI will appear like the image below during a branch or PR build:
Your project is reporting a successful build — yay! Let’s remove one more manual step out of the way.
The Travis CI configuration already has the deployment step defined and will execute sls deploy -v. This will upload your latest files to your AWS resources. You can specify in this step what stage this is going to. By default, the stage will be dev, but it could be test, prod, etc.
For Codeship, select the Deploy tab, and create the deployment pipeline:
The branch can either match exactly or given a pattern (handy if your team does release tags). Now select Scripts from the Add Deployment section. You will need to enter the following:
And you're done. If you merge your pull request, it’ll immediately trigger a final build and deployment. Your resources should be available automatically after you get the success email from either service provider.
All gists and the entire drug-search project repository is publically available on GitHub.
There you have it. Your project should now be building and deploying with the expected triggers. Think of how much time this will save you or your team. I, personally, couldn’t choose between the CI/CD service providers in the beginning and ended up going through the steps for both Travis and Codeship to see how they worked with my project.
We’ll be exploring testing and debugging in the next post, AWS Serverless App: Testing.
Thanks for reading!