This week I released version 1.0 of RxJS Primitives
to NPM - but the
journey to get there was not as easy as I hoped.
In the past I’ve used CircleCI
and at work I use Jenkins
but with this project I wanted to try out Github Actions
.
After some trial an error (and many failed builds) I managed to get the workflow working, I’ve decided to share here the
steps taken to hopefully save you from the same pain.
Setting up your monorepo
The NX
CLI is a set of tools that make managing your repository of JavaScript and TypeScript code
easy. Originally built on top of the Angular CLI
the tooling now supports more project types
including Node, React and Web Components.
In my day-job most of my work is building Angular Enterprise applications, but I also work on my own open source
software and in both cases I use NX, including using it to publish Angular and TypeScript libraries to NPM.
Each way of setting this up is subtly different. In this example I’ll show the steps of how I built a pipeline for
releasing my TypeScript RxJS library.
The easiest way to start is to create your repo
using <code>create-nx-workspace</code>
npx create-nx-workspace <project-name>
When running this, you will be asked a few questions depending on your setup. For RxJS Primitives
I used a plain
workspace, and then once created I added <code>@nrwl/node</code>
to the project. The
default @nrwl/workspace
plugin allows the creation of libraries, but does not provide a publishable output (adding package.json
) as an option.
Once set up you can now create libraries that can be built and published as NPM modules, using the @nrwl/node
plugin
type to create them. For rxjs-string
I used:
1
| > nx g @nrwl/node:library --name=string --directory=rxjs --publishable
|
If it’s an Angular library use:
1
| > nx g @nrwl/angular:library --name=my-component --directory=ngx --publishable
|
After a few seconds, inside the libs
folder will be a default output of a TypeScript library, with
various tsconfig.json
files for testing and building, a Jest
config for unit testing,
a index.ts
file as the entry point and package.json
for publishing.
One issue with nx
is that with this configuration in the package.json
you’ll find a 2-level
name for your library (e.g. @tinynodes/rxjs-string
) however in the root tsconfig.json
file you’ll see the following
in the paths
property
1
2
3
4
5
6
7
| {
"paths": {
"@tinynodes/rxjs/string": [
"libs/rxjs/string/src/index.ts"
]
}
}
|
If you are only using this library internally it’s not really an issue, but when you intend to publish the library I
recommend changing the path to @tinynodes/rxjs-string
to match the NPM import path.
Preparing for publishing
Once you have developed your library, it’s time to publish to NPM! First of all, make sure your public API
(Functions, Classes, and Types) are exported in the library index.ts
:
1
2
3
4
| // index.ts
export { myFactoryFunction } from './libs/factory';
export { MyThing } from './classes/my-thing'
export { MyInterface, SOME_CONSTANT } from './types/thing-types'
|
To see the output of this library you can run nx build library-name
, this will output a NPM module to the dist
folder.
Setting up Github Actions for Pull Requests
For an open-source library on GitHub, it’s good practice for you to get pull requests from other developers, and to have
confidence in those PRs having a pull request checker is ideal.
First create a folder .github
at the root of your workspace, and inside the workflows
folder. This is the folder
that GitHub checks to see if there are any YAML files containing action flows.
Create a file pr_on_master.yml
and inside this file set up the following steps:
1
2
3
4
5
6
7
8
9
| # File for Pull Request on master branch
name: PR on master
# When a PR is opened to master
on:
pull_request:
branches:
- master
types: [ opened, reopened, synchronize ]
|
This first section provides the name and triggers for when the action runs: when a pull request opened or re-opened
against master
.
Next the steps need set up:
1
2
3
4
5
6
7
8
| jobs:
build:
# Setup OS and Node Version
runs-on: ubuntu-latest
strategy:
matrix:
# Latest nodes only
node-version: [ 13.x ]
|
This sets up a Github Action runner and makes sure NodeJS 13 is installed. Here you can add more versions and run
parallel tests against different versions if you plan to support more than one.
1
2
3
4
5
6
7
8
9
10
11
12
| # Define Steps
steps:
# Checkout code
- uses: actions/checkout@v2
with:
ref: ${{ github.event.pull_request.head.ref }}
fetch-depth: 0
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v1
with:
node-version: ${{ matrix.node-version }}
|
The first two steps first check out the code from the pull request branch, and then sets up NodeJS to run.
The next piece of step was the part of setting up the pipeline that took the longest to fix.
To use the nx affected
commands it needs a base branch to compare changes against - by default Github checkout does
not pull all the branches, only the current one - and this includes the master branch. This command tells git to pull
the master branch from the origin:
1
2
3
| # Make sure we have all branches
- name: Fetch other branches
run: git fetch --no-tags --prune --depth=5 origin master
|
Finally, we run some NPM command for installing the dependencies, running linting and test coverage.
1
2
3
4
5
6
7
8
| - name: Install environment
run: npm ci
- name: Run lint
run: npm run affected:lint -- --base="origin/master"
- name: Tests coverage
run: npm run affected:test -- --base="origin/master" --codeCoverage
|
Using the affected commands, the pipeline will only run linting and testing against libraries that have changed against
the master branch. More steps such as SonarQube
or other webhooks can be performed here.
Publishing to NPM
When publishing libraries, there’s a few additional steps needed before we write the pipeline. First of all we need two
access tokens, one for NPM and one for GitHub.
NPM Token
The NPM token will be used to publish the library to the public NPM registry. Log into NPM and under your profile go to
the “Auth Tokens”
section and create a new token with “Publish” access. Next go to your Github repository and under “Settings -> Secrets”
add a new token called NPM_AUTH_TOKEN
and paste in the value.
If publishing to a private registry follow it’s instructions on generating an API token.
Github Token
To publish changes back to GitHub from the pipeline you also need a personal access token - this can be created under
your account settings in “Developer Settings -> Personal Access Token”. The only permissions you need to give this token
are the repo
permissions.
Add this under “Settings -> Secrets” as ACTION_AUTH_TOKEN
(it seems you cannot name them with GITHUB_
in the name at
all) 🤷♂️
Setting up the action
Like before we are going to add a YAML file to the .github/workflows
folder - this time called publish.yml
First set the action up to trigger only when a PR is closed on master
:
1
2
3
4
5
6
| name: Merge on master
on:
pull_request:
branches:
- master
types: [ closed ]
|
The job
section is the same as above, but for the steps there is a slight change - to allow merges to master that *
don’t* trigger a release, each command will be wrapped in a if
block this block checks the commit message for the
string [skip-ci]
to avoid running these tasks (unfortunatly it seems you can’t just put a block around the entire set
of steps so has to be added to all of them)
1
2
3
4
5
| steps:
# Checkout code
- name: Checkout Code
if: github.event.pull_request.merged == true && contains(github.event.commits[0].message, '[skip-ci]') == false
uses: actions/checkout@v2
|
Instead of running lint and test, we’ll now run a set of different tasks - hold on tight because we’re about to dive
into some bash
code!
Deployment Step
In our publish.yaml
add the following step which first exports our registry publishing setting with the auth token we
set up earlier, then we call our publish bash script
1
2
3
4
5
6
7
| - name: Deploy
if: github.event.pull_request.merged == true && contains(github.event.commits[0].message, '[skip-ci]') == false
env:
NPM_AUTH_TOKEN: ${{ secrets.NPM_AUTH_TOKEN }}
run: |
npm config set //registry.npmjs.org/:_authToken=$NPM_AUTH_TOKEN
./.github/scripts/publish-libraries.sh
|
Create the .github/scripts/publish-libraries.sh
file and make sure it’s set to executable by
typing chmod +x .github/scripts/publish-libraries.sh
before committing the script.
The first part of the script is setting up our variables and getBuildType
function to check what type of release we
are doing. It’s good to use SemVer
to publish based on changes that happen in the library and
with this function using the following words in the MERGE COMMIT message will determine the release type.
For example: (fix): Fixed that annoying bug in issue #123
will set the release type to patch
. The default is minor
. Using (feat)
will make a major change.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
| #!/usr/bin/env bash
set -o errexit -o noclobber -o nounset -o pipefail
# This script uses the parent version as the version to publish a library with
getBuildType() {
local release_type="minor"
if [[ "$1" == *"feat"* ]]; then
release_type="major"
elif [[ "$1" == *"fix"* || "$1" == *"docs"* || "$1" == *"chore"* ]]; then
release_type="patch"
fi
echo "$release_type"
}
PARENT_DIR="$PWD"
ROOT_DIR="."
echo "Removing Dist"
rm -rf "${ROOT_DIR:?}/dist"
COMMIT_MESSAGE="$(git log -1 --pretty=format:"%s")"
RELEASE_TYPE=${1:-$(getBuildType "$COMMIT_MESSAGE")}
DRY_RUN=${DRY_RUN:-"False"}
|
The script also cleans up the dist folder, and has an optional DRY_RUN
to set if you want to test the pipeline before
release.
After this add the following:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
| AFFECTED=$(node node_modules/.bin/nx affected:libs --plain --base=origin/master~1)
if [ "$AFFECTED" != "" ]; then
cd "$PARENT_DIR"
echo "Copy Environment Files"
while IFS= read -r -d $' ' lib; do
echo "Setting version for $lib"
cd "$PARENT_DIR"
cd "$ROOT_DIR/libs/${lib/-//}"
npm version "$RELEASE_TYPE" -f -m "RxJS Primitives $RELEASE_TYPE"
echo "Building $lib"
cd "$PARENT_DIR"
npm run build "$lib" -- --prod --with-deps
wait
done <<<"$AFFECTED " # leave space on end to generate correct output
cd "$PARENT_DIR"
while IFS= read -r -d $' ' lib; do
if [ "$DRY_RUN" == "False" ]; then
echo "Publishing $lib"
npm publish "$ROOT_DIR/dist/libs/${lib/-//}" --access=public
else
echo "Dry Run, not publishing $lib"
fi
wait
done <<<"$AFFECTED " # leave space on end to generate correct output
else
echo "No Libraries to publish"
fi
|
This part of the script allows us to control the build and release - the first line gets a single line list of affected
libraries from the current master
to the last HEAD
in master
- if you use only PRs to make changes into master
this is very effective - but can break if you make changes directly to master
.
This is done instead of using nx affected
as there is currently no task for publishing, so this gives a way to provide
a loop for both building and publishing.
Both while
loops parses this string and splits on the space to allow a loop to be run.
In each loop there is a string replacement ${lib/-//}
- this changes the library name (e.g. rxjs-string
) into a path (rxjs/string
)
The first while
loop ensures we are in the correct directory and does npm version
- bumping the package.json
for
release - it then runs the build task for production, and ensures any dependencies are also build.
The second while
loop iterates over the same list but runs the npm publish
command in each directory, setting the
access to public.
Congratulations
If you’ve made it this far, well done - this is quite a long post! At this point your pipeline should have successfully
published your NPM module for other developers to use! We’re not done yet though!
Before running this we are going to add some additional steps so that we can publish documentation and changes back to
Github:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
| - name: Build Docs
if: github.event.pull_request.merged == true && contains(github.event.commits[0].message, '[skip-ci]') == false
run: npm run docs
- name: Commit files
if: github.event.pull_request.merged == true && contains(github.event.commits[0].message, '[skip-ci]') == false
run: |
git config --local user.email "[email protected]"
git config --local user.name "GitHub Action"
git add .
git commit -m "Release [skip-ci]" -a || true
- name: Push changes
if: github.event.pull_request.merged == true && contains(github.event.commits[0].message, '[skip-ci]') == false
uses: ad-m/github-push-action@master
with:
github_token: ${{ secrets.ACTION_AUTH_TOKEN }}
tags: true
force: true
|
For RxJS Primitives I used TypeDoc
to generate static content to the docs
folder, which is
then used by GitHub Pages, but here you can set up whatever documentation system you prefer.
Once the docs have been generated we then commit all changes including the package.json
version bumps back to git and
then finally push it back to master
using the GitHub token generated earlier.
Within a few seconds your GitHub Page should update with the latest content, the master branch reflect all changes made.
That’s a wrap! You’ve now successfully published your TypeScript library for other developers to use, along with
documentation.
If you’ve found this article useful, let me know - and if you find any issues or improvements please get in touch!