GitHub Actions has become very popular in the recent years. More so because of the functionality and the ease of use to just integrate it to automate something about an already existing repo. I had one such requirement. Creating a new release for a repo is easy but then going over the same process of generating the ditributables and then releasing them just becomes tiresome after a while.

This is when I decided to automate my PyPI release through GitHub Actions. This can be done for any other packaging index (if it's simple to publish to).

What I used to do

So before I created this pipeline, I used to make my PyPI releases manually. After drafting a release on GitHub, I would just go to my local clone of the repo and then build the dist using the following:

I also had to make sure that no previously added files are present in the dist directory, so I used to run the following command before running the one to create dists

rm -rf dist

Once done, I would create the latest versions distributables:

python setup.py sdist bdist_wheel

After this, we have our files ready so I just used to use twine to push these files to PyPI with the latest release version:

twine upload --config-file=~/.pypirc dist/*

In the above command I am passing the config file because that contains my credentials for PyPI to make the release.

Is it that tiresome?

Even though it seems like it is just the above three steps, it becomes tiresome after a while. Add onto that the possibility of forgetting to pass a flag to one of the above commands. Or let's say we used a different version of Python that my package is not compatible with.

What if I absolutely removed my .pypirc and now I cannot remember my credentials?

When should the pipeline run

We need to make sure that the pipeline doesn't run with every push. We just want the pipeline to run on releases. This can be done by making tag creation a trigger. This can be setup in the following way:

on:
  push:
    tags:
      - "*"

Setting up the pipeline

Now that we know what we need to do, let's see how we can set it up through a GitHub action file so it's easy to automate the releases.

For starters, we will need the following things:

Setup a proper version of python to build dist files
Install the dependencies to push to PyPI
Build the dist files using the installed packages
Publish the package using twine with GitHub Secrets

Setting up Python

Setting up a proper version of Python is simple, we just need to use the setup-python package. This will automatically setup the latest Python version in the system.

It is better to specify a version to use with this action since some packages do not support the latest python version as soon as it comes out. It can be used in the following way:

- name: Setup Python
  uses: actions/setup-python@v2
  with:
    python-version: "3.6"

We can also set the architecture in the above package which by default is set to x64.

Install dependencies

Now that we have Python setup, we need to install the packages that we will require to build the dist files and push them to PyPI. We will need the following packages:

setuptools: To build the dist files
wheel: To build wheel dist files
twine: To push the package to PyPI.

We can install the above using pip after setting up Python in the following way:

- name: Install dependencies
  run: |
    python -m pip install --upgrade pip
    pip install setuptools wheel twine

NOTE that we did an --upgrade pip here to make sure pip is latest and no issues are thrown.

Build the dist files

Now that we have our dependencies setup, we can go ahead and build the dist files using the command that I mentioned above:

- name: Build dist files
  run: python setup.py sdist bdist_wheel

Publish the package

Once we have our dist files ready, we can use twine to publish the package. Before we setup this step, we will need to provide two things to twine:

username: PyPI username
password: PyPI password

We can pass the above to twine through env. However, we will need to store them on GitHub Secrets to use them in envs and keep them safe at the same time. Read more about setting GitHub secrets here

The following variables can be saved on GitHub Secrets:

PYPI_USERNAME
PYPI_PASSWORD

Once we have our credentials ready, we can define the publish stage in the following way:

- name: Publish Package
  env:
    TWINE_USERNAME: ${{ secrets.PYPI_USERNAME }}
    TWINE_PASSWORD: ${{ secrets.PYPI_PASSWORD }}
  run: twine upload dist/*

Complete Pipeline

Considering all the above steps, we have our final pipeline file in the following structure:

name: PyPI Release

on:
  push:
    tags:
      - "*"

jobs:
  release:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout Repo
        uses: actions/checkout@v2
      - name: Setup Python
        uses: actions/setup-python@v2
        with:
          python-version: "3.6"
      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install setuptools wheel twine
      - name: Build the dist files
        run: python setup.py sdist bdist_wheel
      - name: Publish
        env:
          TWINE_USERNAME: ${{ secrets.PYPI_USERNAME }}
          TWINE_PASSWORD: ${{ secrets.PYPI_PASSWORD }}
        run: twine upload dist/*

Test it out

Once the pipeline is ready, it needs to be saved in the .github/workflows directory with the name release.yaml. After that's done, if a new release is made, this pipeline would be invoked automatically.

Live Example

The pipeline can be seen live on the downloader-cli repo.

Looking at the actions tab will show a recent run that was invoked due to a release and it was successfull. Moreover, the file is located inside .github/workflows directory in the repo. Check it out here.

Posts for action

Automate Releases With Github Actions for Python