This page documents a few miscellaneous topics at the edges of this guide’s scope: low-level buffers, doctests, and bytecode cache files.
Raw buffer protocol:
- Fixer: None
- Prevalence: Very rare
Python 2 used a buffer interface for sharing blocks of memory between Python objects and other libraries.
In Python 3, the buffer object and the related C API is removed.
Unfortunately, the specifics of low-level interfaces between Python and
non-Python libraries are too different across projects for us to offer
universal advice on porting to the new API.
If your code uses
buffer (or the
PyBuffer C API), you will need to
refer to the Python documentation for details, and combine that with
knowledge about your particular interface.
- Fixer: None
- Prevalence: Common
Doctests are a common practice for quick tests and testing documentation. They work by extracting snippets example code and its output in documentation, running the code, and verifying that its result matches, textually, the example output.
This relies on minute details of textual representation of Python objects, which is generally not under any backwards-compatibility guarantee and may change at any time – even across minor versions of Python (e.g. 2.6 to 2.7 or 3.5 to 3.6).
Some examples of what changed between Python 2 and 3 are:
- String have different
bprefixes depending on if they’re bytes or text.
- Large integers lost the
- The order of items in dictionaries may be different (and unpredictable).
Doctests are a good way to ensure that the documentation is correct (i.e. it doesn’t contain broken examples), but they are not a good way to actually test the code.
If your code uses doctests as the main means of testing, rewrite them as tests
that do not rely on exact textual output.
You can use the built-in
unittest, or the third-party pytest library,
Once your doctests are only testing documentation, we recommend the following strategy:
- Keep running doctests under Python 2
- Port all code to be compatible with (and tested on) both Python 2 and 3
- At one moment, update examples in the docs, and start only using Python 3 to run the doctests.
Since the code is tested elsewhere, it generally does not matter that code examples are tested under only one of the supported Python versions.
Since compiling Python code is a relatively expensive operation, and many modules
do not change often, Python caches compiled bytecode in
In Python 2,
.pyc files were written in the same directory as the
.py source files, with only a
c added to the filename.
The exact mechanism had two major drawbacks:
- Bytecode is not compatible across Python versions.
If the same module was being imported by different versions of Python,
each would overwrite the
.pycfile with its own flavor of bytecode on import. This would invalidate the cache for all other versions.
.pyccache could be used even without a corresponding
.pyfile, which allowed some space saving (by distributing only the compiled file). However, if one deleted a
.pyfile but forgot to also remove the
.pyc, Python would act as if the module was still present. This was quite confusing, especially for beginners.
Python 3 puts
.pyc files in a separate directory called
and adds version information to the filename.
The new mechanism avoids the above two problems: per-version caches are
separated, and if the
.py source is missing, the
.pyc file is not
If your code relies on the location of
.pyc files (for example, if
your build/packaging system doesn’t handle Python 3), you will need to update
to the new location.
If you rely on importing
.pyc files without corresponding source,
you will need to move the
.pyc to the old location, and remove the
version tag. For example, move:
Under this name, the
.pyc will be recognized by Python 3’s import