Frequently Asked Questions¶
In case you have a general question that is not answered here, please have a look at our discussions and consider submitting a new one for the Q&A.
- Does my project depend on PyScaffold when I use it to set my project up?
Starting from version 4, your package is completely independent from PyScaffold, we just kick-start your project and take care of the boilerplate. However, we do include some build-time dependencies that make your life easier, such as setuptools_scm. But don’t worry, if you distribute your project in the recommended wheel format those dependencies will not affect the final users, they are just required during development to assembling the package file.
That means if someone clones your repository and tries to build it, the dependencies in
pyproject.tomlwill be automatically pulled. This mechanism is described by PEP 517/PEP 518 and definitely beyond the scope of this answer.
- Can I use PyScaffold ≥ 3 to develop a Python package that is Python 2 & 3 compatible?
Python 2 reached end-of-life in 2020, which means that no security updates will be available, and therefore any software running on Python 2 is potentially vulnerable. PyScaffold strongly recommends all packages to be ported to the latest supported version of Python.
That being said, Python 3 is actually only needed for the
putupcommand and whenever you use
setup.py. This means that with PyScaffold ≥ 3 you have to use Python 3 during the development of your package for practical reasons. If you develop the package using six you can still make it Python 2 & 3 compatible by creating a universal
bdist_wheelpackage. This package can then be installed and run from Python 2 and 3. Just have in mind that no support for Python 2 will be provided.
- How can I get rid of PyScaffold when my project was set up using it?
First of all, I would really love to understand why you want to remove it and what you don’t like about it. You can create an issue for that or just text me on Twitter. But the good news is that your project is completely independent of PyScaffold, even if you uninstall it, everything will be fine.
If you still want to remove setuptools_scm (a build-time dependency we add by default), it’s actually really simple: Within
setup.pyjust remove the
use_scm_versionargument from the
setup()call which will deactivate the automatic version discovery. In practice, following things will no longer work:
python setup.py --versionand the dynamic versioning according to the git tags when creating distributions, just put e.g.
version = 0.1in the
That’s already everything you gonna lose. Not that much. You will still benefit from:
the smart project layout,
the declarative configuration with
setup.cfgwhich comes from
some sane defaults in Sphinx’
.gitignorewith some nice defaults and other dot files depending on the flags used when running
some sane defaults for pytest.
For further cleanups, feel free to remove the dependencies from the
pyproject.tomlas well as the complete
- Why would I use PyScaffold instead of Cookiecutter?
PyScaffold is focused on a good out-of-the-box experience for developing distributable Python packages (exclusively). The idea is to standardize the structure of Python packages. Thus, PyScaffold sticks to
“There should be one– and preferably only one –obvious way to do it.”
from the Zen of Python. The long-term goal is that PyScaffold becomes for Python what Cargo is for Rust. Still, with the help of PyScaffold’s extension system customizing a project scaffold is possible.
Cookiecutter on the other hand is a really flexible templating tool that allows you to define own templates according to your needs. Although some standard templates are provided that will give you quite similar results as PyScaffold, the overall goal of the project is quite different.
Still, if you so desire, PyScaffold allows users to augment PyScaffold projects with certain types of cookiecutter templates, through its pyscaffoldext-cookiecutter extension.
File Organisation and Directory Structure¶
- Why does PyScaffold ≥ 3 have a
srcfolder which holds the actual Python package?
This avoids quite many problems compared to the case when the actual Python package resides in the same folder as
setup.py. A nice blog post by Ionel gives a thorough explanation why this is so. In a nutshell, the most severe problem comes from the fact that Python imports a package by first looking at the current working directory and then into the
PYTHONPATHenvironment variable. If your current working directory is the root of your project directory you are thus not testing the installation of your package but the local package directly. Eventually, this always leads to huge confusion (“But the unit tests ran perfectly on my machine!”).
Moreover, having a dedicated
srcdirectory to store the package files, makes it easy to comply with recent standards in the Python community (for example PEP 420).
Please notice that PyScaffold assumes all the files inside
srcare meant to be part of the package.
- Can I have other files inside the
srcfolder that are not meant for distribution?
PyScaffold considers the
srcdirectory to be exclusively dedicated to store files meant to be distributed, and relies on this assumption to generate configuration for the several aspects of your project. Therefore it is not recommended to include any file not meant to distribution inside the
srcfolder. (Temporary files and directories automatically generated by
setuptoolsmight appear from times to times though).
- Where should I put extra files not meant for distribution?
You can use the
docsfolder (if applicable) or create another dedicated folder in the root of your repository (e.g.
examples). The additional project structure created by the pyscaffoldext-dsproject is a good example on how to use extra folders to achieve good project organisation.
- How can I get rid of the implicit namespaces (PEP 420)?
setup.cfgto ensure setuptools will follow PEP 420. If this configuration particularly messes up with your package, or you simply want to follow the old behavior, please replace
packages = find_namespace:with
packages = find:in the
[options]section of that file.
You should also remove the
--implicit-namespacesoption in the
cmd_line_templatevariable in the
Finally, if want to keep a namespace but use an explicit implementation (old behavior), make sure to have a look on the packaging namespace packages official guide. If there are already other projects with packages registered in the same namespace, chances are you just need to copy from them a sample of the
__init__.pyfile for the umbrella folder working as namespace.
- My namespaced package and/or other packages that use the same namespace broke after updating to PyScaffold 4. How can I fix this?
That is likely to be happening because PyScaffold 4 removed support for pkg_resources namespaces in favour of PEP 420. Unfortunately these two methodologies for creating namespaces are not compatible, as documented in the packaging namespace packages official guide. To fix this problem you (or other maintainers) will need to either (a) update all the existing “subpackages” in the same namespace to be implicit (PEP 420-style), or (b) get rid of the implicit namespace configuration PyScaffold automatically sets up during project creation/update. Please check the answers for question 8 and question 10 and the updating guides for some tips on how to achieve that.
- How can I convert an existing package to use implicit namespaces (PEP 420)?
The easiest answer for that question is to (a) convert the existing package to a PyScaffold-enabled project (if it isn’t yet; please check our guides for instructions) and (b) update your existing project to the latest version of PyScaffold passing the correct
The slightly more difficult answer for that question is to (a) make sure your project uses a src layout, (b) remove the
__init__.pyfile from the umbrella folder that is serving as namespace for your project, (c) configure
setup.cfgto include your namespace – have a look on setuptools, for packages that use the
src-layoutthat basically means that you want to have something similar to:
[options] # ... packages = find_namespace: package_dir = =src # ... [options.packages.find] where = src
setup.cfg– and finally, (d) configure your documentation to include the implicit namespace (for Sphinx users, in general that will mean that you want to run
--implicit-namespacesflag after extending the
The previous steps assume your existing package uses setuptools and you are willing to have a src layout, if that is not the case refer to the documentation of your package creator (or the software you use to package up your Python projects) and the PEP 420 for more information.
- Can I modify
requiresdespite the warning in
pyproject.tomlto avoid doing that?
You can definitely modify
pyproject.toml, but it is good to understand how PyScaffold uses it. If you are just adding a new build dependency (e.g. Cython), there is nothing to worry. However, if you are trying to remove or change the version of a dependency PyScaffold included there, PyScaffold will overwrite that change if you ever run
putup --updatein the same project (in those cases
git diffis your friend, and you should be able to manually reconcile the dependencies).
- What should I do if I am not using
pyproject.tomlor if it is causing me problems?
If you prefer to have legacy builds and get the old behavior, you can remove the
pyproject.tomlfile and run
python setup.py bdist_wheel, but we advise to install the build requirements (as the ones specified in the
pyproject.toml) in an isolated environment and use it to run the
setup.pycommands (tox can be really useful for that). Alternatively you can use the
setup_requiresfield in setup.cfg, however, this method is discouraged and might be invalid in the future.
For the time being you can use the transitional
--no-pyprojectoption, when running
putup, but have in mind that this option will be removed in future versions of PyScaffold.
Please check our updating guide for extra steps you might need to execute manually.
Best Practices and Common Errors with Version Numbers¶
- How do I get a clean version like 3.2.4 when I have 3.2.3.post0.dev9+g6817bd7?
Just commit all your changes and create a new tag using
git tag v3.2.4. In order to build an old version checkout an old tag, e.g.
git checkout -b v3.2.3 v3.2.3and run
tox -e buildor
python setup.py bdist_wheel.
- Why do I see unknown as version?
In most cases this happens if your source code is no longer a proper Git repository, maybe because you moved or copied it or Git is not even installed. In general using
pip install -e .,
python setup.py installor
python setup.py developto install your package is only recommended for developers of your Python project, which have Git installed and use a proper Git repository anyway. Users of your project should always install it using the distribution you built for them e.g.
pip install my_project-3.2.3-py3-none-any.whl. You build such a distribution by running
tox -e build(or
python setup.py bdist_wheel) and then find it under
- Is there a good versioning scheme I should follow?
The most common practice is to use Semantic Versioning. Following this practice avoids the so called dependency hell for the users of your package. Also be sure to set attributes like
- Is there a best practice for distributing my package?
First of all, cloning your repository or just coping your code around is a really bad practice which comes with tons of pitfalls. The clean way is to first build a distribution and then give this distribution to your users. This can be done by just copying the distribution file or uploading it to some artifact store like PyPI for public packages or devpi, Nexus, etc. for private packages. Also check out this article about packaging, versioning and continuous integration.
- Using some CI service, why is the version unknown or my_project-0.0.post0.dev50?
Some CI services use shallow git clones, i.e.
--depth N, or don’t download git tags to save bandwidth. To verify that your repo works as expected, run:
git describe --dirty --tags --long --first-parent
which is basically what setuptools_scm does to retrieve the correct version number. If this command fails, tweak how your repo is cloned depending on your CI service and make sure to also download the tags, i.e.
git fetch origin --tags.