Releasing a GitHub-Hosted, GitFlow-Release-Managed, Wheel-Packaged Python Package with a Markdown-formatted Readme to PyPI Using Twine
Registering a PyPi Account
Before you do anything else, you will need accounts on the test and production PyPi websites:
- Navigate to the Test PyPi site and create an account for test releasing
- Navigate to the PyPi site and create an account for officially releasing
.pypircfile stores the information locally so you don't have to input usernames and passwords with every upload. It needs to be created, as described here.
To include both the production and test PyPi sites, that might look like the following (swapping out your own usernames and passwords for
password where indicated:
[distutils] index-servers = pypi pypitest [pypi] repository: https://pypi.python.org/pypi username: username password: password [pypitest] repository: https://testpypi.python.org/pypi username: username password: password
Using a Markdown-formatted Readme
Note that a Markdown-formatted Readme file is possible but PyPi won't display the contents as the long description if the Readme file is being used as the long description. This is a nicety of PyPi which provides a browsable interface to extra information about a package.
To use a Markdown-formatted Readme file, assuming it is in the root of the project (analogous to the
setup.py script) in the
[metadata] description-file = README.md
Then, in the
setup.py script, the call to
setup() will not include a kwarg for
long_description. As opposed to an instance using a ReStructuredText-formatted Readme where that kwarg argument would be
Creating a Development Environment for a Python Package and Uploading it to PyPI
Assuming you've created a
.pypirc configuration file and registered for PyPI Live and PyPI Test as described above, the directions for using twine rather than sdist are pretty much a drop-in-place replacement.
The directions that follow assume that you have a project you've built and have been hosting on GitHub and are ready to release to PyPi.
Starting from the top, we will create a new virtual environment, clone in the repo, run tests and lint the project, register the package to PyPi Test and PyPi Live and then upload.
Clone the Repo
$ git clone firstname.lastname@example.org:myuser/myproject.git
This creates a directory tree as follows:
├── myproject ├── LICENSE.txt ├── README.md ├── myproject │ ├── __init__.py ├── setup.cfg ├── setup.py └── tests ├── __init__.py └── test_myproject.py
Create a Virtual Environment
(I'm using virtualenvwrapper):
$ cd myproject $ mkvirtualenv -a . myVirtualEnvName
Install the Package for Development
$ pip install -e .
Install Wheel, Twine and PyLint
$ pip install wheel twine pylint
Run the Tests
$ python ./tests/test_myproject.py $ pylint ./myproject
Setup Git Flow
I'm using Nvie's Git Flow to manage releases to GitHub. The first time the project is cloned and setup, GitFlow needs to be initialized.
By default my project clones the
develop branch which is my default working branch. Git Flow uses
master to track the latest release. This branch is also on GitHub so it needs to be checked out.
$ git checkout master
master branch exists alongside
develop, Git Flow can be initialized with the defaults.
$ git flow init -d
Create the Distribution
This needs to be done each time the version (or subpoint version) gets bumped.
Tag the version for release.
$ git flow release start versionNumberAndSubPoint
setup.py and bump the
download_url versions (my
download_url points to a specific tarball of a tag on GitHub).
$ git flow release finish versionNumberAndSubPoint
Push everything back to GitHub.
$ git push --all; git push --tags
Create the dist.
$ python setup.py sdist
Create the Wheel
Depending on whether the package is universal to Python 2 and 3 or not changes how this is run. Instructions can be found in the Python Packaging User Guide. To create a Universal package that works with both Python 2 and 3:
$ python setup.py bdist_wheel --universal
Register the package to PyPI Test
This only needs to be done the first time.
$ python setup.py register -r pypitest
Upload to PyPI Test
$ twine upload dist/* -r pypitest
Register the package to PyPI Live
This only needs to be done the first time.
$ python setup.py register -r pypi
Upload to PyPI Live
$ twine upload dist/*
There are a number of existing tutorials and articles on this subject. However, many tutorials and directions cover eggs and easy install rather than pip and wheel, or use sdist rather than twine. The Python Packaging and Users Guide is the actual, most-up-to-date documentation on the process of releasing a package to the Python Package Index (PyPI). It covers the additional parameters of the setup file including the more complicated variables of
classifiers and possible values therein. It also includes instructions for uploading files using the better practice of using twine instead of sdist as well as the wheel binary package distribution format intended to replace eggs.
The Python Project Howto, while aging, does a good job of covering general best practices before deciding to release a module.
How to submit a package to PyPI provides a good basic overview of what to include in the
setup.py script and how to link to documentation when it is in a markdown-formatted readme file. This also provids all the directions for creating PyPI Live and PyPI Test accounts and a
.pypirc configuration file. The official documentation is contained in the distutils documentation. The basic steps are:
- August 20, 2015 — Revised virtualenv instructions to utilize virtualenvwrapper. Added instructions for using wheel. Added GitFlow instructions.
- February 23, 2016 — Noted that Universal wheels support both Python 2 and Python 3; consolidated installation of Python packages necessary for linting and pushing to PyPi.
- March 20, 2017 — Added directions regarding Markdown formatted readmes as well as the
- June, 11, 2017 — Revised the language in the introduction of the article to be more straightforward and moved it to a resource section at the bottom of the article.