Following system colour scheme Selected dark colour scheme Selected light colour scheme

Python Enhancement Proposals

PEP 771 – Default Extras for Python Software Packages

Author:
Thomas Robitaille <thomas.robitaille at gmail.com>, Jonathan Dekhtiar <jonathan at dekhtiar.com>
Sponsor:
Pradyun Gedam <pradyunsg at gmail.com>
Discussions-To:
Discourse thread
Status:
Draft
Type:
Standards Track
Topic:
Packaging
Created:
13-Jan-2025
Post-History:
15-Jan-2025, 06-Feb-2025, 09-Jun-2025, 15-Oct-2026
Table of Contents

Abstract

PEP 508 specifies a mini-language for declaring package dependencies. One feature of this language is the ability to specify extras, which are optional components of a distribution that, when used, install additional dependencies. This PEP proposes a mechanism to allow one or more extras to be installed by default if none are provided explicitly, and specifies that an empty set of extras (package[]) requests a minimal installation without any default extras.

Motivation

Various use cases for default extras and possible solutions in this PEP were discussed extensively on this discuss.python.org thread. These fall into two broad cases that provide the motivation for the present PEP.

Packages supporting multiple backends or frontends

Another common use case for using extras is to define different backends or frontends and dependencies that need to be installed for each backend or frontend. A package might need at least one backend or frontend to be installed in order to be functional, but may be flexible on which backend or frontend this is. Concrete examples of such frontends or backends include:

With current packaging standards, maintainers have to either require one of the backends or frontends or require users to always specify extras, e.g., package[backend], and therefore risk users having an unusable installation if they only install package. Having a way to specify one or more default backend or frontend and providing a way to override these defaults would provide a much better experience for users, and the approach described in this PEP will allow this.

Note that this PEP does not aim to address the issue of disallowing conflicting or incompatible extras, for example if a package requires exactly one frontend or backend package. There is currently no mechanism in Python packaging infrastructure to disallow conflicting or incompatible extras to be installed, and this PEP does not change that.

Examples of packages that require at least one backend or frontend to work and recommend a default extra to install a backend or frontend include:

In all three cases, installing the package without any extras results in a broken installation, and this is a commonly reported support issue for some of these packages.

Rationale

A number of possible solutions have been discussed extensively by the community for several years, including in this discuss.python.org thread as well as in numerous issues and pull requests. The solution that is presented below:

  • is an opt-in solution which means that package maintainers can choose whether or not to use it
  • is flexible enough to accommodate both of the major use cases described in Motivation
  • re-uses the syntax from PEP 508

It is the only solution, out of all those discussed, that meets all three criteria.

An alternative approach using existing infrastructure – splitting a package into a core package and a meta-package (discussed in detail in Using a meta-package for recommended installations) – can technically achieve similar outcomes but imposes a significant and ongoing maintenance burden, introduces user-facing issues such as naming mismatches and unintuitive uninstallation behavior, and has proven impractical enough that many projects have chosen not to adopt it.

The concept of default-but-optional dependencies has prior art in other ecosystems. Linux distributions such as Debian/Ubuntu and Fedora have long distinguished between required (Depends) and recommended (Recommends) dependencies, where recommended packages are installed by default but can be excluded. The Rust/Cargo ecosystem has had default features for over 10 years, which are widely used.

Specification

Default-Extra Metadata Field

A new multiple-use metadata field, Default-Extra, will be added to the core package metadata. For this field, each entry must be a string specifying an extra that will be automatically included when the package is installed without any extras being specified explicitly.

Only entries already specified in a Provides-Extra entry can be used in a Default-Extra entry. Each entry must be normalized following the same rules as Provides-Extra (see PEP 685) and must match the corresponding Provides-Extra entry exactly.

Examples:

Default-Extra: recommended

Default-Extra: backend1
Default-Extra: backend2
Default-Extra: backend3

Since this introduces a new field in the core package metadata, this will require Metadata-Version to be bumped to the next minor version (2.7 at the time of writing).

New key in [project] metadata table

A new key will be added to the [project] table in project metadata as originally defined in PEP 621 and now defined in the PyPA specifications. This key will be named default-optional-dependencies with the following description:

  • TOML type: Array of strings
  • Corresponding core metadata field: Default-Extra

Each string in default-optional-dependencies must be the name of an extra defined in optional-dependencies, and each extra in this array will be converted to a matching Default-Extra entry in the core package metadata. Build backends must raise an error if any of the strings do not match the name of a defined extra. Examples of valid usage which would produce the example Default-Extra entries presented in the previous section are:

[project]
default-optional-dependencies = [
    "recommended",
]

and:

[project]
default-optional-dependencies = [
    "backend1",
    "backend2",
    "backend3"
]

As with other keys in the [project] table, default-optional-dependencies may be listed in dynamic, in which case the build backend is responsible for determining the default extras. Since its value is an array, it may also be specified statically and listed in dynamic simultaneously, following the rules for list-valued keys in the pyproject.toml specification. In all cases the entries must be validated against the final set of extras produced by the build backend, including when optional-dependencies is itself dynamic. Correspondingly, Default-Extra may be listed in the Dynamic field of a source distribution’s core metadata.

Overriding default extras

If extras are explicitly given in a dependency specification, the default extras must be ignored. If a package is specified bare, that is with no extras and no brackets at all, the default extras must be installed.

For example, if a package defines an extra1 default extra as well as a non-default extra2 extra, then if a user were to install the package with:

$ pip install package

the dependencies of the default extra1 extra would be included. If the user instead installs the package with:

$ pip install package[extra2]

then the extra2 extra would be installed but the default extra1 extra would be ignored.

If the same package is specified multiple times in an installation command or dependency tree, the default extras must be installed if any of the instances of the package are specified bare. For instance, consider installing spam, whose dependency tree contains package several times:

spam
├── tomato
│   └── package[extra2]
└── egg
    └── package

then the default extra must be installed because package appears at least once bare.

An empty set of extras, such as package[], must be interpreted as a request to install the package without any default extras. package[] neither triggers nor prevents the installation of default extras: the default extras are installed only if package also appears bare elsewhere in the dependency tree, as described above. For example, a dependency tree containing only package[] and package[extra2] results in extra2 being installed but no default extras, since no instance of package is bare. package[] therefore provides a standardized way of obtaining a minimal installation of a package.

If a package is already installed without its default extras (for example, because it was pulled in as a dependency via package[]), and a user subsequently runs pip install package (or equivalent), the installer must ensure that the default extras’ dependencies are installed. In other words, a bare package specification must always be treated as requesting the package with its default extras, regardless of whether the package is already present in the environment.

Conversely, this PEP does not introduce any mechanism for removing packages. If a package was installed with its default extras, or with an explicitly specified extra, and is subsequently installed with a different set of extras, the dependencies that are no longer requested are not uninstalled. This is consistent with how extras behave today.

When multiple packages in a dependency tree define default extras, the above rules apply independently to each package. For example, consider two packages flask and werkzeug that both define default extras. If flask lists werkzeug[] in its required dependencies and werkzeug (without []) in a default extra, then installing flask would include werkzeug with its own default extras (because the default extra causes werkzeug to appear bare), while installing flask[] would include only werkzeug[] (no default extras for either package). This follows from the rules above and requires no special handling of nested default extras.

We also note that some tools such as pip currently ignore unrecognized extras, and emit a warning to the user to indicate that the extra has not been recognized, e.g.:

$ pip install package[non-existent-extra]
WARNING: package 3.0.0 does not provide the extra 'non-existent-extra'
...

For tools that behave like this (rather than raising an error), if an extra is recognized as invalid in a dependency specification, it should be ignored, and if all specified extras are invalid, then this should be considered equivalent to package[] (rather than package) and not install any default extras.

Finally, we note (as also discussed in Relying on tooling to deselect any default extras) that package installers are allowed to implement their own options to control the above behavior, for example implementing an option that disables default extras for some or all packages regardless of where these packages appear in the dependency tree. If such tool-specific options are implemented, tool developers should make these opt-in, and users should experience the above PEP 771 behavior as the default.

Default extras do not change how dependencies within extras interact with platform compatibility. If a default extra includes a dependency that is only available on certain platforms, the dependency should use environment markers as with any other dependency, e.g.:

Requires-Dist: windows-only-package ; sys_platform == "win32"

The behavior of the installer when a dependency within a default extra cannot be satisfied on the current platform is the same as for any other extra.

Examples

In this section we take a look at the use cases described in the Motivation section and how these can now be addressed by using the specification outlined above.

Packages requiring at least one backend or frontend

As described in Motivation, some packages may support multiple backends and/or frontends, and in some cases it may be desirable to ensure that there is always at least one backend or frontend package installed, as the package would be unusable otherwise. Concrete examples of this might include a GUI application that needs a GUI library to be present to be usable but is able to support different ones, or a package that can rely on different computational backends but needs at least one to be installed.

In this case, package maintainers could make the choice to define an extra for each backend or frontend, and provide a default, e.g.:

[project]
default-optional-dependencies = [
    "backend1"
]

[project.optional-dependencies]
backend1 = [
    "package1",
    "package2"
]
backend2 = [
    "package3"
]

If packages can support e.g. multiple backends at the same time, and some of the backends should always be installed, then the dependencies for these must be given as required dependencies rather than using the default extras mechanism.

To take one of the concrete examples mentioned in Motivation, the napari package can make use of one of PyQt5, PyQt6, PySide2, or PySide6, and users currently need to explicitly specify napari[all] in order to have one of these be installed, or e.g., napari[pyqt5] to explicitly specify one of the frontend packages. Installing napari with no extras results in a non-functional package. With this PEP, napari could define the following configuration:

[project]
default-optional-dependencies = [
    "pyqt5"
]

[project.optional-dependencies]
pyqt5 = [
    "PyQt5",
    "..."
]
pyside2 = [
    "PySide2",
    "..."
]
pyqt6 = [
    "PyQt6",
    "..."
]
pyside6 = [
    "PySide6",
    "..."
]

meaning that:

$ pip install napari

would work out-of-the-box, but there would still be a mechanism for users to explicitly specify a frontend, e.g.:

$ pip install napari[pyside6]

Supporting extras that should not remove default extras

An additional case we consider here is where a package maintainer wants to support the ability for users to opt in to non-default extras, without removing default extras. Essentially, they might want:

  • package[] to give an installation without any extras
  • package to install recommended dependencies (in a recommended extra)
  • package[alternative] to not install default extras, but to install an alternative set of optional dependencies (in an alternative extra)
  • package[additional] to install both recommended and additional dependencies (in an additional extra)

This could be achieved with e.g.:

[project]
default-optional-dependencies = [
    "recommended"
]

[project.optional-dependencies]
recommended = [
    "package1",
    "package2"
]
alternative = [
    "package3"
]
additional = [
    "package[recommended]",
    "package4"
]

The ability for a package to reference itself in the extras is supported by existing Python packaging tools.

Once again considering a concrete example, astropy could with this PEP define a recommended extra (as described in Recommended dependencies and minimal installations). However, it also defines other extras, including for example jupyter, which adds packages that enhance the user experience inside Jupyter-based environments. It is possible that users opting in to this extra would still want recommended dependencies to be installed. In this case, the following configuration would solve this case:

[project]
default-optional-dependencies = [
    "recommended"
]

[project.optional-dependencies]
recommended = [
    "scipy",
    "..."
]
jupyter = [
    "astropy[recommended]",
    "ipywidgets",
    "..."
]

Users installing:

$ pip install astropy[jupyter]

would then get the same as:

$ pip install astropy[recommended, jupyter]

Packages with multiple kinds of defaults

In some cases, it may be that packages need multiple kinds of defaults. As an example, in Packages requiring at least one backend or frontend, we considered the case of packages that have either backends or frontends, but in some cases, packages may have to support backends and frontends, and want to specify one or more default frontend and one or more default backend.

Ideally, one may want the following behavior:

$ pip install package  # installs default backend and frontend
$ pip install package[]  # installs no backends or frontends
$ pip install package[backend1]  # installs backend1 and default frontend
$ pip install package[frontend2]  # installs frontend2 and default backend
$ pip install package[backend1, frontend2]  # installs backend1 and frontend2

However, this PEP chooses not to provide a mechanism for making it so that e.g., if backend1 is specified, the default backend would be disabled, but the default frontend would be enabled, since this adds complexity.

Maintainers should instead for now document that if a backend or frontend is explicitly specified, both backend and frontend need to be specified. Discoverability for users who want to do this should not be an issue, however, since users need to read the documentation in any case to find out what backends or frontends are available, so they can be shown at the same time how to properly use the extras for backends and frontends.

One option to increase user friendliness is that maintainers can create extras called for example defaultbackend and defaultfrontend which do install the default backend and frontend. They can then recommend that users do:

$ pip install package  # installs default backend and frontend
$ pip install package[]  # installs no backends or frontends
$ pip install package[backend1, defaultfrontend]  # installs backend1 and default frontend
$ pip install package[defaultbackend, frontend2]  # installs frontend2 and default backend
$ pip install package[backend1, frontend2]  # installs backend1 and frontend2

This would allow (if desired) users to then get whatever the recommended backend is, even if that default changes in time.

This limitation – the inability to deselect individual defaults while keeping others – is shared by Cargo’s default features in the Rust ecosystem, where the only option is all-or-nothing via default-features = false. If there is a desire to implement a better solution in the future, we believe this PEP should not preclude this. For example, one could imagine in the future adding the ability for an extra to specify which default extras it disables, and if this is not specified then explicitly specified extras would disable all default extras (consistent with the present PEP).

Depending on packages with default extras

The examples above focus on packages that define default extras. It is also useful to illustrate how downstream packages can depend on packages with default extras in their pyproject.toml. The following examples assume that astropy has defined recommended as a default extra.

A downstream package that is happy with the default extras needs no special syntax:

[project]
dependencies = [
    "astropy>=6.0",
]

A downstream library that wants only the minimal dependencies can use package[]:

[project]
dependencies = [
    "astropy[]>=6.0",
]

This syntax is backward-compatible: for versions of astropy that pre-date the introduction of default extras, astropy[] is equivalent to astropy.

A downstream package can also request specific extras, which will override the defaults:

[project]
dependencies = [
    "astropy[jupyter]>=6.0",
]

The same dependency specifiers can be used anywhere PEP 508 strings are accepted, and the rules in Overriding default extras apply identically. For example, in PEP 723 inline script metadata:

# /// script
# dependencies = [
#     "astropy[]>=6.0",
# ]
# ///

or in PEP 735 dependency groups:

[dependency-groups]
docs = [
    "astropy",
]
minimal-tests = [
    "astropy[]",
]

In all of these cases, astropy requests the default extras and astropy[] does not, regardless of whether the specifier appears in a project’s dependencies, a script, a dependency group, a requirements file, or on the command line.

Backward Compatibility

Packages not using default extras

Once support for this PEP is added to tools in the packaging ecosystem, packages that do not make use of default extras will continue to work as-is and there should be no break in compatibility.

Packages using default extras

Once packages start defining default extras, those defaults will only be honored with recent versions of packaging tools which implement this PEP, but those packages will remain installable with older packaging tools – with the main difference being that the default extras will not be installed automatically when older packaging tools are used.

As described in Supporting older versions of package installers, package authors need to carefully evaluate when and how they adopt default extras depending on their user base, since some actions, such as moving a required dependency to a default extra, will result in breakage for users of older installers. Authors are thus responsible for minimizing the impact on their users.

Change in meaning of a bare package name

This PEP changes the meaning of a bare package name (e.g. package without any extras) for packages that define default extras: prior to this PEP, installing package would install only the dependencies listed in dependencies, whereas with this PEP it may additionally install the dependencies associated with default extras.

It is worth noting, however, that in practice the dependencies listed in dependencies have never strictly represented a minimal set. Package authors have always had full discretion over what to include in dependencies, and many packages include dependencies in dependencies that are not strictly required for core functionality but are considered important for a good default experience. There is currently no mechanism for users or downstream packages to opt out of such dependencies. This PEP provides a way for authors to separate truly required dependencies from recommended-but-optional ones, and introduces package[] as a standardized way to obtain a minimal installation – something that does not exist today.

This PEP is also opt-in: no existing package changes behavior until its maintainer explicitly adds default extras, which is no different in nature from other metadata changes authors routinely make, such as adding or removing required dependencies.

Security Implications

As noted in Packaging-related tools, tools that audit dependencies for security purposes will need to take Default-Extra into account to accurately report what will be installed. Beyond this, this PEP introduces no security risks that do not already exist with required dependencies, since package authors can already cause arbitrary additional packages to be installed by listing them as required dependencies.

How to teach this

This section outlines information that should be made available to people in different groups in the community in relation to the implementation of this PEP. Some aspects described below will be relevant even before the PEP is fully implemented in packaging tools as there are some preparations that can be done in advance of this implementation to facilitate any potential transition later on. The groups covered below are:

Package end users

Package users should be provided with clear installation instructions that show what extras are available for packages and how they behave, for example explaining that by default some recommended dependencies or a given frontend or backend will be installed, and how to opt out of this or override defaults, depending on what is available.

Package authors

While the mechanism used to define extras and the associated rule about when to use it are clear, package authors need to carefully consider several points before adopting this capability in their packages, to avoid inadvertently breaking backward-compatibility.

Supporting older versions of package installers

Package installers such as pip or uv will not necessarily implement support for default extras at the same time, and once they do it is likely that package authors will want to keep supporting users who do not have the most recent version of a package installer. In this case, the following recommendations would apply:

  • Moving a package from being a required dependency to a default extra would be a breaking change, because older versions of package installers would not recognize the concept of default extras, and would then install the package with fewer dependencies, which could affect users that would have been relying on these. Therefore, changing dependencies from required to a default extra in an established package should only be done in the future once the developers only want to support users with installers that implement this PEP.
  • Making an existing extra become a default should be safer, such as making recommended in astropy be a default extra, but in order to support users with older versions of package installers, the documentation should still mention the extra explicitly as long as possible (until it is clear that most/all users are using package installers that implement this PEP). There is no downside to keeping the extra explicitly mentioned, but this will ensure that users with modern tooling who do not read documentation (which may be a non-negligible fraction of the user community) will start getting the recommended dependencies by default.
  • Since prior to this PEP, package[] was equivalent to package, authors will be able to document package[] as a backward-compatible way of getting a minimal installation. For packages that define default extras, installing package[] will always give a minimal installation even with older versions of packaging tools such as pip, and releases of this package that pre-date the introduction of default extras for a specific package will also be installable with package[] (although in these cases this will be equivalent to package). For packages that do not define default extras, package[] will continue to be equivalent to package.

Avoiding the addition of many default dependencies

One temptation for authors might be to include many dependencies by default since they can provide a way to opt out from these. We recommend, however, that authors carefully consider what is included by default to avoid unnecessarily bloating installations and complicating dependency trees. Using default extras does not mean that all extras need to be defaults, and there is still scope for users to explicitly opt in to non-default extras.

As a guideline, a default extra should contain dependencies that the majority of users would need or expect when installing the package. Good candidates include dependencies that are required for the primary documented use case of the package (such as a default backend or frontend), or dependencies that are needed for core functionality that most users rely on but that some downstream consumers or advanced users may wish to exclude. Dependencies that serve niche use cases, add significant installation size, or are only relevant to a subset of users should remain as non-default extras.

Default extras should generally be treated with the same “weight” as required dependencies. When a package is widely used, introducing a default extra will result in that extra’s dependencies being transitively included – unless all downstream packages are updated to explicitly opt out using minimal installation specifications.

As an example, the pytest package currently has nearly 1,500 plugins that depend on it. If pytest were to add a default extra and those plugins were not updated accordingly, installing a plugin would include the default extras’ dependencies. This doesn’t preclude the use of default extras, but addition of default extras requires careful evaluation of its downstream effects.

As discussed in Change in meaning of a bare package name, this is analogous to what already happens when a widely-used package adds a new required dependency, with the difference that downstream packages can opt out of default extras using package[].

In this regard, the mechanism proposed in this PEP differs from the Recommends feature in Linux distributions such as Debian/Ubuntu and Fedora. In those systems, a package cannot opt out of the Recommends of its own dependencies, so recommended dependencies accumulate throughout the dependency tree with no way for intermediate packages to limit this. The only opt-out is a global one applied by the end user, such as --no-install-recommends, which disables all recommendations at once. With this PEP, any package in the dependency tree can opt out of specific upstream default extras on a per-package basis using package[], which limits the compounding effect.

Impact on downstream packages

When a dependency adopts default extras, downstream packages that depend on it will receive additional dependencies. Authors of downstream packages should be aware of this possibility and can respond in several ways:

  • If the additional dependencies are acceptable, no action is needed.
  • If a minimal dependency footprint is desired, the downstream package can depend on package[] to receive only the required dependencies (this is backward-compatible, as described in Supporting older versions of package installers).
  • If specific extras are needed, the downstream package can depend on package[specific-extra], which as with any explicitly specified extra will override the default extras.

Downstream packages that define their own extras may also need to consider how those extras interact with upstream default extras. For instance, if a downstream package offers both a full and minimal extra, the author will need to decide whether the minimal extra should depend on upstream-package[] and the full extra on upstream-package, or whether both should use the same form. This is a new consideration introduced by this PEP, but is similar in nature to decisions downstream authors already make when upstream packages add or reorganize their own extras.

Note that if different downstream packages make different choices – for example, one depending on upstream[] and another on upstream – the default extras will be installed if any instance of the package appears without extras in the dependency tree (as described in Overriding default extras). This is consistent with the existing union semantics for extras: upstream[] should be read as “I do not need the defaults” rather than “I forbid the defaults.”

For the same reason, downstream authors should not add [] to every dependency defensively. Depending on upstream[] is a statement that the downstream package does not rely on any functionality provided by the default extras. If upstream later moves functionality that the downstream package uses from its required dependencies into a default extra, a downstream package depending on upstream[] would break, while one depending on upstream would continue to work. [] should therefore be used deliberately, where a minimal footprint is genuinely needed and the downstream package has been verified to work without the default extras.

Default extras are not a substitute for good package design

A concern that has been raised during the discussion of this PEP is that default extras might discourage authors from properly structuring their code into well-scoped packages – for example, by using a default extra to bundle unrelated functionality together rather than distributing it as a separate package.

Default extras are not intended to replace good package design. They are a mechanism for expressing the author’s intent about what a typical installation should include, in cases where the boundary between “required” and “optional” is genuinely ambiguous. A package whose components are truly independent and have distinct user bases would still be better served by being split into separate packages. Default extras are most appropriate when a package is conceptually one thing but has optional components that most users want – such as recommended performance or convenience dependencies, or a default backend for a package that supports multiple backends.

The alternative approach of splitting packages into a core package and a meta-package is discussed in detail in Using a meta-package for recommended installations, along with the reasons it is not considered a compelling alternative.

Inheriting from default extras

If package authors choose to make an extra be installed by default, it is important that they are aware that if users explicitly specify another extra, the default may not be installed, unless they use the approach described in Supporting extras that should not remove default extras.

There are cases, such as the interchangeable backends, where ignoring the default if an extra is explicitly specified is the right thing to do. However, for other cases, such as using default extras to include recommended dependencies while still providing a way to do minimal installs, it may be that many of the other extras should explicitly ‘inherit’ the default extra(s), so package authors should carefully consider in which cases they want the default extras to be installed.

Incompatible extras

In some cases, it may be that packages have extras that are mutually incompatible. In this case, we recommend against using the default extra feature for any extra that contains a dependency that could be incompatible with another.

Consider a package that has extras package[A] and package[B]. Users can already try to install package[A] and package[B] or package[A,B] which would result in a broken installation, however it would at least be explicit that both extras were being installed. Making A into a default extra, however, could lead to unintuitive issues. A user could do:

$ pip install package  # this installs package[A]
$ pip install package[B]

and end up with a broken installation, even though A and B were never explicitly both installed. For this reason, we recommend against using default extras for dependencies where this is likely to be an issue.

Circular dependencies

Authors need to take special care when circular dependencies are present. For instance, consider the following dependency tree:

package1
└── package2
    └── package1

If package1 has a default extra named recommended then:

$ pip install package1[]

will still result in the recommended extra being installed if package2 continues to depend on package1 (with no extras specified). This could be solved by changing the dependency tree to instead be:

package1
└── package2
    └── package1[]

assuming that indeed package2 does not depend on any features provided by the extra dependencies of package1. Authors therefore need to carefully consider a migration plan, coordinating with the authors of package2.

Documenting packages with default extras

Regardless of how default extras are used, package authors should aim to ensure that their package’s documentation makes it clear how extras are to be used. ‘Best practices’ documentation should mention:

  • that installing package will be equivalent to package[<default extras>]
  • that installing package[] will include only minimal/required dependencies, but that this will not guarantee that optional dependencies do not get installed if package appears anywhere else in the dependency tree
  • what other optional extras are available, and whether or not they disable the default extras (since this can be controlled as described in Supporting extras that should not remove default extras)
  • any instructions specific to packages that might have e.g. default backends and frontends (as described in Packages with multiple kinds of defaults)

It is also recommended that packages using default extras include a continuous integration (CI) job that performs a minimal installation (without default extras) and verifies that core functionality still works. This helps ensure that the package does not accidentally rely on default extra dependencies for functionality that should be available in a minimal installation.

Packaging repository maintainers

The impact on individuals who repackage Python libraries for different distributions, such as conda, Homebrew, Linux package installers (such as apt and dnf) and so on, needs to be considered. Not all package distributions have historically had mechanisms that would line up with the approach described, although this is evolving (for example, conda has recently added support for optional dependencies via CEP 44).

There are two cases to consider here:

  • In cases where the repackaging is done by hand, such as for a number of conda-forge recipes, the introduction of default extras should not have a large impact since manual decisions already have to be made as to which dependencies to include (for example, the conda-forge recipe for the astropy package mentioned in the Motivation includes all the recommended dependencies by default).
  • In cases where the repackaging is done in an automated way, distribution maintainers will need to carefully consider how to treat default extras, and this may imply a non-negligible amount of work and discussion.

While repackaging projects that use default extras requires decisions about how to map them, this does not make repackaging impossible. Repackagers already routinely handle concepts that have no direct equivalent in their target system – for example, until recently extras had no equivalent in conda, yet conda-forge has successfully packaged thousands of Python projects (with some recipes including extra dependencies as required, others omitting them, and others splitting packages into e.g. package-base and package). Some distribution systems already have concepts that map naturally to default extras, such as the Recommends field in Debian/Ubuntu and Fedora mentioned in Rationale.

When an upstream project introduces a default extra, the repackager can choose to include those dependencies as required or omit them. In cases where the default extra contains dependencies that were previously required, the repackager may not need to change anything at all – their existing recipe continues to work, and they gain the option to make those dependencies optional if the target distribution supports this.

Default extras provide additional signal about how package authors would like their package to be installed for the majority of users, and this should inform repackaging decisions regardless of the target distribution.

Reference Implementation

The following repository contains a fully functional demo package that makes use of default extras. This makes use of modified branches of several packages:

In addition, this branch contains a modified version of the Flit package.

The implementations above are proofs-of-concept at this time and the existing changes have not yet been reviewed by the relevant maintainers. Nevertheless, they are functional enough to allow for interested maintainers to try these out.

Rejected Ideas

Syntax for deselecting extras

One of the main competing approaches was as follows: instead of having defaults be unselected if any extras were explicitly provided, default extras would need to be explicitly unselected.

In this picture, a new syntax for unselecting extras would be introduced as an extension of the mini-language defined in PEP 508. If a package defined default extras, users could opt out of these defaults by using a minus sign (-) before the extra name. The proposed syntax update would have been as follows:

extras_list   = (-)?identifier (wsp* ',' wsp* (-)?identifier)*

Valid examples of this new syntax would have included, e.g.:

  • package[-recommended]
  • package[-backend1, backend2]
  • package[pdf, -svg]

However, there are two main issues with this approach:

  • One would need to define a number of rules for how to interpret corner cases such as if an extra and its negated version were both present in the same dependency specification (e.g. package[pdf, -pdf]) or if a dependency tree included both package[pdf] and package[-pdf], and the rules would not be intuitive to users.
  • More critically, this would introduce new syntax into dependency specification, which means that if any package defined a dependency using the new syntax, it and any other package depending on it would no longer be installable by existing packaging tools, so this would be a major backward compatibility break.

For these reasons, this alternative was not included in the final proposal.

Adding a special entry in extras_require

A potential solution that has been explored as an alternative to introducing the new Default-Extra metadata field would be to make use of an extra with a ‘special’ name.

One example would be to use an empty string:

Provides-Extra:
Requires-Dist: numpy ; extra == ''

The idea would be that dependencies installed as part of the ‘empty’ extras would only get installed if another extra was not specified. An implementation of this was proposed in https://github.com/pypa/setuptools/pull/1503, but it was found that there would be no way to make this work without breaking compatibility with existing usage. For example, packages using Setuptools via a setup.py file can do:

setup(
    ...
    extras_require={'': ['package_a']},
)

which is valid and equivalent to having package_a being defined in install_requires, so changing the meaning of the empty string would break compatibility.

In addition, no other string (such as 'default') can be used as a special string since all strings that would be a backward-compatible valid extras name may already be used in existing packages.

There have been suggestions of using the special Python value None, but again this is not possible, because even though one can use None in a setup.py file, this is not possible in declarative files such as setup.cfg or pyproject.toml, and furthermore ultimately extras names have to be converted to strings in the package metadata. Having:

Provides-Extra: None

would be indistinguishable from the string ‘None’ which may already be used as an extra name in a Python package. If we were to modify the core metadata syntax to allow non-string ‘special’ extras names, then we would be back to modifying the core metadata specification, which is no better than introducing Default-Extra.

Relying on tooling to deselect any default extras

An alternative to the package[] syntax to unselect extras would be to implement this at the level of packaging tools. For instance, pip could include an option such as:

$ pip install package --no-default-extras

This option could apply to all or specific packages, similar to the --no-binary option, e.g.,:

$ pip install package --no-default-extras :all:

The advantage of this approach is that tools supporting default extras could also support unselecting them. This approach would be similar to the --no-install-recommends option for the apt tool.

However, this solution is not ideal on its own because it would not allow packages to specify themselves that they do not need some of the default extras of a dependency. It would also carry risks for users who might disable all default extras in a big dependency tree, potentially breaking packages in the tree that rely on default extras at any point.

Nevertheless, this PEP does not disallow this approach and it is up to the maintainers of different packaging tools to decide if they want to support this kind of option. It is a flag that could at the very least be useful for package maintainers who want to identify places in dependency trees where default extras are being relied on. However, if it is supported, it should be made clear that using this flag does not guarantee a functional environment.

Standardizing an opt-in extra name

During the discussion of this PEP, an alternative was proposed that would keep the ability for package authors to designate recommended extras, but make it opt-in rather than opt-out: a conventional extra name such as recommended or full would be standardized to mean “the dependencies that a typical user should install”, and users would be taught to install package[recommended]. The meaning of a bare package would then be unchanged, which would be fully backward-compatible.

This approach was not adopted for the following reasons:

  • It does not address the core problem described in Motivation: users who do not know about extras, or do not read the installation instructions, would still type pip install package and get an incomplete installation. This is precisely the situation that packages such as astropy (with astropy[recommended]) and napari (with napari[all]) are in today, so standardizing the name would be at best a marginal improvement over the status quo.
  • As discussed in Adding a special entry in extras_require, any name that is a valid extra name may already be in use by existing packages with a different meaning, so a reserved name cannot be introduced without potential collisions, and any name that is not a valid extra name would require new syntax, with the compatibility problems described in Syntax for deselecting extras.

Requiring one of several extras

For the multiple backends or frontends use case, it was suggested that a more natural solution would be a mechanism to express that a package requires one of several alternatives (a logical OR between dependencies or extras), with the installer selecting an alternative that is already present in the environment, or prompting the user otherwise.

This PEP does not pursue this approach because:

  • A logical OR between dependencies cannot be expressed in the dependency specification mini-language defined in PEP 508 without introducing new syntax, with the backward-compatibility problems described in Syntax for deselecting extras.
  • It would require installers to make choices based on the current state of the environment, which makes the result of an installation depend on context and is difficult to reconcile with lock files and other reproducible resolution workflows.
  • It would address only the backends and frontends use case, and not the recommended dependencies use case which motivates a large fraction of the interest in this PEP.

Default extras address the backends and frontends case by letting authors designate a default. A future PEP could build on the metadata introduced here to allow a set of extras to be declared as interchangeable alternatives, and nothing in this PEP precludes this.