JS monorepos in prod 3: commit enforcement and changelog generation

Conventional Commits introduces a structured format for commit messages. It standardizes the messages among all the contributors. This makes them more readable and easy to automate. It simplifies the management of a monorepo and contributes to better DevOps practices. Additionally, it enables the automatic generation of changelog files.

In the previous article on project versioning and publishing, I briefly covered the structure of the commit messages. This article shows how to enforce and automatically validate them with Conventional Commits and how to use them for changelog generation. In the following articles we will continue with unit tests and CI/CD integration:

Conventional Commits definition

By now, we have already mentioned Conventional Commits several times, but we still need to define them. The Conventional Commits is a lightweight convention, or a specification, to structure your commit massages to make them readable by humans and interpretable by machines.

Here’s how a commit message looks like:

[optional scope]: 
[optional body]
[optional footer(s)]

You may have noticed that our commit messages respect the format on the first name. Types match predefined values and have a special meaning towards Semantic Versioning (SemVer):

• patch fixes a bug and corresponds to PATH in semantic versioning.
• feat stands for a feature and corresponds to MINOR in semantic versioning.
• BREAKING CHANGE introduces a change in the API and corresponds to MAJOR in semantic versioning.

Other types are allowed. A majority of projects follow the Angular convention, which defines the build, chore, ci, docs, style, refactor, perf, test types.

We have already made use of the build, chore, and docs types. I admit to using my own interpretation of build, which covers anything related to project management.

Scopes must also match predefined values. They act like topics or categories. In a monorepo, it is common to associate them with the package names.

Conventional Commits usage and changelog generation

Since Conventional Commits are interpretable by machine, it is tempting to rely on the commit message to generate a changelog or choose the appropriate version number.

The gatsby-remark-title-to-frontmatter package also deserves a README, so let’s create it:

echo \
 '# Package `gatsby-remark-title-to-frontmatter`' \
  > gatsby-remark/title-to-frontmatter/README.md
git add gatsby-remark/title-to-frontmatter/README.md
git commit -m "docs(title-to-frontmatter): new readme file"

This time, we run lerna version with the --conventional-commits flag. Basing on the commit types, it proposes the most appropriate version for validation.

lerna version --conventional-commits
info cli using local version of lerna
lerna notice cli v3.22.1
lerna info versioning independent
lerna info Looking for changed packages since [email protected]
lerna info ignoring diff in paths matching [ '**/test/**' ]
lerna info getChangelogConfig Successfully resolved preset "conventional-changelog-angular"
Changes:
 - gatsby-remark-title-to-frontmatter: 1.0.0-alpha.1 => 1.0.0-alpha.2
? Are you sure you want to create these versions? Yes
lerna info execute Skipping releases
lerna info git Pushing tags...
lerna success version finished

A new file, the changelog file, is also created inside our published package. Its Markdown content is extracted from the commit history and looks like:

# Change Log
All notable changes to this project will be documented in this file.
See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.
# [1.0.0-alpha.2](https://github.com/adaltas/remark-gatsby-plugins/compare/gatsby-remark-title-to-frontmatter@[email protected]) (2020-12-03)
**Note:** Version bump only for package gatsby-remark-title-to-frontmatter

More detailed information would have been available with some patch, feat, or BREAKING CHANGE commit messages.

It is, of course, possible to combine the changelog generation and the --conventional-commits flag with the enforcement of a particular version and a SemVer keyword.

Conventional Commits from the command line – Commitizen

Our commit messages became very important, and our collaborators must get some help to create them if we want to preserve consistency across all our commits.

Commitizen is a CLI tool that guides you in the process of creating compliant commit messages. The user is prompted to fill out any required commit field at commit time.

There are multiple ways to use Commitizen. Running npm install -g commitizen installs the package globally, and it hooks it into Git. Now, you can simply use the git cz instead of git commit. However, I prefer to install my dependencies locally.

Here is how to initialize your repo with Commitizen:

npx commitizen init cz-conventional-changelog -D -E

This installs the commitizen dependency and configures it with the Commitizen adapter of your choice, Conventional Commits in our case. The -D flag is to save the adapter to devDependencies and the -E flag is to set an exact version instead of a range.

diff --git a/package.json b/package.json
index f853695..15e431f 100644
--- a/package.json
+++ b/package.json
@@ -12,5 +12,13 @@
   "workspaces": [
     "gatsby/*",
     "gatsby-remark/*"
-  ]
+  ],
+  "devDependencies":
+    "cz-conventional-changelog": "^3.3.0"
+  ,
+  "config":
+    "commitizen":
+      "path": "./node_modules/cz-conventional-changelog"
+
+
 }

Commitizen works on staged changes. Before testing it, make some changes and stage them with git add. Since Commitizen has already modified our package.json file, we can just stage that file and use the npx cz command instead of git commit:

git add package.json
npx cz

First, it starts to prompt for the type of change. Possible values are:

• feat
  A new feature
• fix
  A bug fix
• docs
  Documentation only changes
• style
  Changes that do not affect the meaning of the code (white-space, formatting, missing semi-colons, etc)
• refactor
  A code change that neither fixes a bug nor adds a feature
• perf
  A code change that improves performance
• test
  Adding missing tests or correcting existing tests
• build
  Changes that affect the build system or external dependencies (example scopes: gulp, broccoli, npm)
• ci
  Changes to our CI configuration files and scripts (example scopes: Travis, Circle, BrowserStack, SauceLabs)
• chore
  Other changes that don’t modify src or test files
• revert
  Reverts a previous commit

Note, my usage of the build in the previous articles might not be conventional, since I use it to set up the project.

In the second step, it asks us for an optional scope. Typically, we align the scope with the package name. This is covered later in more details.

Then come the principal commit message and description.

Finally, Commitizen must know if our change introduces any breaking change and if it is related to an open issue.

This is how the final npx cz command looks like:

npx cz
[email protected], [email protected]
? Select the type of change that you’re committing: build:    Changes that affect the build system or external dependencies (example scopes: gulp, broccoli, npm)
? What is the scope of this change (e.g. component or file name): (press enter to skip)
? Write a short, imperative tense description of the change (max 93 chars):
 (32) declare and configure commitizen
? Provide a longer description of the change: (press enter to skip)
? Are there any breaking changes? No
? Does this change affect any open issues? No
[master 9e33979] build: declare and configure commitizen
 1 file changed, 10 insertions(+), 2 deletions(-)

Conventional Commits validation – Commitlint

Our collaborators now have access to a nice CLI tool that helps and guides them in the creating their commit message. However, it does not preserve us from mistakes. Anyone not using git cz or npx cz – for example, those using their favorite graphical editor – will commit the message of their choice. We need to enforce good practices and not let invalid commits be created and pushed.

commitlint validates messages based on Conventional Commits. We must install the CLI tool along with its Conventional Commits adapter. We also need to create the commitlint.config.js file at the root of our project.

yarn add -D -W @commitlint/config-conventional,cli
cat <<CONFIG > commitlint.config.js
module.exports =
  extends: [
    "@commitlint/config-conventional"
  ]
;
CONFIG

The -W flag bypasses a Yarn check, which prevents you from declaring dependencies in the root package.

Let’s try to see how it works. An invalid message, such as an invalid type, must raise an error:

echo 'invalid: error are expected sometimes' | yarn commitlint
yarn run v1.22.5
$ /Users/david/projects/github/remark-gatsby-plugins/node_modules/.bin/commitlint
⧗   input: invalid: error are expected sometimes
✖   type must be one of [build, chore, ci, docs, feat, fix, perf, refactor, revert, style, test] [type-enum]
✖   found 1 problems, 0 warnings
ⓘ   Get help: https://github.com/conventional-changelog/commitlint/
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Our commit message is invalid, wonderful! The output even suggests us the valid types supported by Conventional Commits.

The package.json file is modified with the new dependencies, and a new commitlint.config.js file is ready to be committed. This is how to commit only if the validation passed:

echo 'build: enable commitlint' | yarn commitlint || exit 1
git add commitlint.config.js package.json
git commit -m 'build: enable commitlint'

Conventional Commits enforcement – Husky

A solution to validate commit messages is now at our disposal. However, we can’t expect every user to issue the command. Doing it later, such as inside your CI/CD, will be too late. The commit is already there and synchronized with your central remote repository. Therefore, commit validation must be automated. This is where Husky comes in. It plugs itself into the Git’s hook configuration to validate lint rules before committing.

We will use the latest release of Husky, version 5. Its layout differs from version 4, don’t be surprised. Note, however, that at the time of this writing (December 2020), the license of version 5 only permits Open Source usages of the library unless you become a sponsor. You can also continue to use version 4 at work.

yarn add -D -W husky@next
yarn husky install

Now, have a look at your .git/config file. It has been updated with:

cat .git/config | grep hook
	hooksPath = .husky

Husky is now set up, we continue configuring it to check our commits message. We want to hook commitlint just before a commit happens:

yarn husky add .husky/commit-msg 'yarn commitlint --edit $1'

A new .husky folder is created, and it must not be Git ignored. Only the .husky/_ folder inside must be ignored and there is already the .husky/.gitignore file just for that. Since the folder name starts with ., we need to modify our original .gitignore rules:

echo '!.husky' >> .gitignore

Inside the .husky folder, Husky-managed files are named after the Git hook name. The examples can be found in the .git/hooks folder. Husky has created a pre-commit file executing the yarn commitlint --edit $1 command on commit. Let’s commit the .husky folder along with our changes in the .gitignore and package.json files:

git add .gitignore .husky package.json
git commit -a -m 'disable ignore rule for husky configuration'
yarn run v1.22.5
$ /Users/david/projects/github/gatsby/node_modules/.bin/commitlint --edit
⧗   input: disable ignore rule for husky configuration
✖   subject may not be empty [subject-empty]
✖   type may not be empty [type-empty]
✖   found 2 problems, 0 warnings
ⓘ   Get help: https://github.com/conventional-changelog/commitlint/
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
husky - commit-msg hook exited with code 1 (error)

Oops, I forgot. Conventional Commits are now automatically enforced, and there was no type in the commit message:

git commit -a -m 'build: disable ignore rule for husky configuration'

It works. And not only it does work on the command line with git commit and npx cz, but everywhere including inside your favorite editor.

One more thing with Husky: to automatically setup Git hooks on install, edit the package.json file.

  "scripts": 
    "postinstall": "husky install"
  

And commit it:

git commit -a -m 'build: husky activation on install'

Conventional Commits scope personalization

It was mentioned earlier how the commit scope is a good fit for monorepos. By naming the scope with the package name, the global changelog can inform of the affected packages in the commit messages.

commitlint provides the @commitlint/config-lerna-scopes package adapter for Lerna:

yarn add -D -W @commitlint/config-lerna-scopes

It must also be registered inside the commitlint.config.js file:

module.exports = 
  extends: [
    "@commitlint/config-conventional",
    "@commitlint/config-lerna-scopes"
  ]
;

However, I find the list of scopes provided by the package name to be a little restrictive. Indeed, we already have a problem. Let’s try to commit our change and make a release.

First, we need some changes to the packages to trigger a new version. We add an instruction to both README files:

git commit -a -m 'build: commitlint with lerna scopes'
echo '' >> gatsby/caddy-redirects-conf/README.md
echo 'Generate a Caddy compatible config file.' >> gatsby/caddy-redirects-conf/README.md
git commit -a -m 'docs(gatsby-caddy-redirects-conf): introduction'
echo '' >> gatsby-remark/title-to-frontmatter/README.md
echo 'Move the title from the content to the frontmatter' >> gatsby-remark/title-to-frontmatter/README.md
git commit -a -m 'docs(gatsby-remark-title-to-frontmatter): introduction'

We can now try to create new versions:

lerna version --conventional-commits
info cli using local version of lerna
lerna notice cli v3.22.1
lerna info versioning independent
lerna info Looking for changed packages since [email protected]
lerna info ignoring diff in paths matching [ '**/test/**' ]
lerna info getChangelogConfig Successfully resolved preset "conventional-changelog-angular"
Changes:
 - gatsby-remark-title-to-frontmatter: 1.0.0-alpha.2 => 1.0.0-alpha.3
 - gatsby-caddy-redirects-conf: 0.1.0-alpha.2 => 0.1.0-alpha.3
? Are you sure you want to create these versions? Yes
...
lerna ERR! ✖   scope must be one of [gatsby-remark-title-to-frontmatter, gatsby-caddy-redirects-conf] [scope-enum]
...
lerna ERR! lerna husky - commit-msg hook exited with code 1 (error)
lerna ERR! lerna
git reset --hard HEAD

The output is verbose, but the important line is scope must be one of [gatsby-caddy-redirects-conf, gatsby-remark-title-to-frontmatter] [scope-enum]. This is because Lerna is configured to generate a commit message from the template chore(release): publish. The list of supported scopes is enforced by @commitlint/config-lerna-scopes and “release” is not one of them.

The solution is to plug ourselves into the rules.scope-enum function and add our custom scopes. Modify the commitlint.config.js file accordingly.

const utils: getPackages = require('@commitlint/config-lerna-scopes');
module.exports = 
  "extends": [
    "@commitlint/config-conventional",
    "@commitlint/config-lerna-scopes"
  ],
 rules: 
  'scope-enum': async ctx =>     [2, 'always', [...(await getPackages(ctx)),       // Insert custom scopes below:      'release'    ]] 

We can now commit the commitlint.config.js change and create a new release:

git commit -a -m 'build: add the release scope generated by lerna'
lerna version --conventional-commits
...
Changes:
 - gatsby-remark-title-to-frontmatter: 1.0.0-alpha.2 => 1.0.0-alpha.3
 - gatsby-caddy-redirects-conf: 0.1.0-alpha.2 => 0.1.0-alpha.3
...

Prerelease version management

We have made use of prerelease version previously when running lerna version. The command prompt us with the choice for creating prepatch, preminor and premajor. For example, if you are currently with version 0.1.0-alpha.3, the choices are:

lerna version
info cli using local version of lerna
lerna notice cli v3.20.2
lerna info versioning independent
lerna info Looking for changed packages since @nikitajs/[email protected]
lerna info version rooted leaf detected, skipping synthetic root lifecycles
? Select a new version for @nikitajs/db (currently 0.9.7) (Use arrow keys)
❯ Patch (0.9.8)
  Minor (0.10.0)
  Major (1.0.0)
  Prepatch (0.1.1-alpha.0)   Preminor (0.2.0-alpha.0)   Premajor (1.0.0-alpha.0)   Custom Prerelease
  Custom Version

The common prerelease stages are apha when the software may contain serious errors and not all of the features that are planned for the final version, beta when the software is feature complete but likely to contain a number of known or unknown bugs and rc when the software has the potential to be a stable release.

Lerna automates the selection of the prerelease with the --preid flag. For example, to jump from version 0.1.0 to a major version 1.0.0 in beta state:

lerna version --conventional-commits --preid beta premajor

However, asking Lerna to increment all packages to a major version kind of defeat the purpose of using the --conventional-commits flag to extract the next version from the commit logs.

Instead, we can switch to a prerelease version while letting Lerna choosing the appropriate version bump with the --conventional-prerelease flag:

lerna version --conventional-commits --conventional-prerelease

You can then exit the prerelease state and graduate to a final version with the --conventional-graduate flag:

lerna version --conventional-commits --conventional-graduate

Cheatsheet

• Commit validation with commitlint
  Install the dev dependency

  yarn add -D -W @commitlint/config-conventional,cli

  Modify commitlint.config.js with Conventionnal Commit and Lerna scopes:

  module.exports = 
    extends: [
      "@commitlint/config-conventional",
      "@commitlint/config-lerna-scopes"
    ]
  ;

  Commit the changes:

  git add commitlint.config.js package.json
  git commit -m 'build: enable commitlint'

• Enforce Conventional Commit on commit
  Install Husky

  yarn add -D -W husky@next
  yarn husky install

  Register the commit hook

  yarn husky add .husky/commit-msg 'yarn commitlint --edit $1'

  Modify package.json to automatically setup Git hooks on the install

  
    ...
    "scripts": 
      ...
      "postinstall": "husky install"
    
  

  Commit the changes:

  echo '!.husky' >> .gitignore
  git add .gitignore .husky package.json
  git commit -a -m 'build: disable ignore rule for husky configuration'

• Prerelease
  Enter prerelease state while controling the next version bump (use premajor, preminor, or prepatch):

  lerna version --conventional-commits --preid beta premajor

  Enter prerelease state with automatic version bump:

  lerna version --conventional-commits --conventional-prerelease

  Graduate to a final version:

  lerna version --conventional-commits --conventional-graduate

Conclusion

In this article we discovered how to create the Conventional Commits with Commitizen, validate them with commitlint, and automate the validation with Husky. In addition, we used them to automatically create the changelogs. In the upcoming articles, we will see how to run unit tests and how to automate the publication of packages in a CI/CD environment.