Style and Idioms
Why Style Matters in Python
Section titled “Why Style Matters in Python”Python’s design philosophy famously states that “there should be one— and preferably only one —obvious way to do it.” This is not merely aesthetic guidance. In a language where indentation is Syntactically significant, where dynamic typing defers error detection, and where the same Abstraction can be expressed through functions, classes, generators, or metaclasses, consistent Style is what makes large codebases maintainable. A Python project without enforced style Conventions will inevitably fracture into incompatible dialects as different contributors make Different implicit choices about naming, structure, and error handling.
The practical consequences are severe: inconsistent code is harder to review, harder to refactor, And harder to onboard new contributors into. The tools and conventions described in this document Exist to eliminate these problems at the mechanical level, freeing developers to focus on logic Rather than formatting.
PEP 8 Essentials
Section titled “PEP 8 Essentials”PEP 8 is the foundational style guide for Python code. It is a Community standard, not a language mandate — the Python interpreter does not enforce any of these Rules. This means enforcement depends entirely on tooling and team discipline.
Naming Conventions
Section titled “Naming Conventions”Naming conventions exist so that a reader can determine the role of a name from its form alone, Without needing to look up its definition.
## Module names: short, lowercase, no underscores for readability if single wordimport collectionsimport json_parser
## Package names: same as modules# mypackage/
# Class names: CapWords (CamelCase)class UserRepository: pass
class HTTPConnectionError(Exception): pass
# Exception names: end with "Error" if they indicate an errorclass ValidationError(ValueError): pass
# Function and variable names: snake_casedef calculate_total(order_items): subtotal = sum(item.price for item in order_items) return subtotal
# Constants: UPPER_SNAKE_CASE at module levelMAX_CONNECTIONS = 100DEFAULT_TIMEOUT = 30.0
# Private attributes: single leading underscore (convention, not enforced)class Database: def __init__(self): self._connection = None self._pool = []
def _ensure_connected(self): if self._connection is None: self._connect()
# Name mangling: double leading underscore (actually enforced by interpreter)class Base: def __init__(self): self.__value = 42 # stored as _Base__value
class Derived(Base): def get_parent_value(self): # self.__value would be _Derived__value -- different attribute return self._Base__valueThe distinction between _private and __mangled is worth understanding precisely. A single Leading underscore is a convention that signals “this is an implementation detail, don’t use it from Outside the class.” It is not enforced by the interpreter — any code can access _private Attributes freely. A double leading underscore triggers name mangling: the interpreter rewrites __name to _ClassName__nameWhich makes it harder (but not impossible) to accidentally access From subclasses.
Line Length and Formatting
Section titled “Line Length and Formatting”PEP 8 recommends a maximum line length of 79 characters for code and 72 for docstrings. These limits Originate from the era of 80-column terminals and side-by-side diff reviews. Many modern teams use 88 or 100 characters instead (88 is the default in Black, chosen because it fits more code without Significantly reducing readability on modern displays).
The key insight is that the specific number matters less than having a consistent limit enforced by Tooling. Arguments over 79 vs 88 vs 120 are less productive than just picking one and running a Formatter.
# Implicit line continuation inside brackets -- the preferred wayresult = some_function( first_argument, second_argument, third_argument, fourth_argument,)
# Dictionary displayconfig = { "database_url": "postgresql://localhost/mydb", "max_connections": 10, "timeout_seconds": 30,}
# Long string -- use parentheses, not backslash continuationquery = ( "SELECT users.id, users.name, orders.total " "FROM users " "JOIN orders ON users.id = orders.user_id " "WHERE orders.created_at > %s")
# Avoid backslash continuation except in rare cases# Bad:if a == 1 and \ b == 2 and \ c == 3: pass
# Good:if ( a == 1 and b == 2 and c == 3): passImports
Section titled “Imports”Imports should be grouped in three sections separated by blank lines, in this order: standard Library, third-party packages, local application modules. Within each group, imports should be Sorted alphabetically. isort and ruff enforce this automatically.
# Standard libraryimport osimport sysfrom pathlib import Pathfrom typing import Optional
# Third-partyimport httpxfrom pydantic import BaseModel, field_validator
# Local applicationfrom myapp.models import Userfrom myapp.services import AuthServicefrom myapp.utils import format_dateSpecific rules to follow:
# Avoid wildcard imports -- they pollute the namespace and make it# impossible to determine where a name came from# Bad:from os.path import *
# Good:from os.path import join, exists, isfile
# Use absolute imports by default -- they are unambiguous# Bad (relative):from ..models import User
# Good (absolute):from myapp.models import User
# Relative imports are acceptable only within a package when the# package structure makes absolute paths unwieldyType Annotations
Section titled “Type Annotations”Type annotations make contracts between code units explicit. In a dynamically-typed language, they Are the most cost-effective tool for preventing a large class of bugs: passing the wrong type to a Function, returning an inconsistent type from a method, or misinterpreting the shape of a data Structure.
Basic Annotations
Section titled “Basic Annotations”from typing import Optional, Union
def greet(name: str) -> str: return f"Hello, {name}"
def divide(numerator: float, denominator: float) -> Optional[float]: if denominator == 0: return None return numerator / denominator
def process(value: Union[int, str]) -> str: return str(value)Modern Syntax (Python 3.10+)
Section titled “Modern Syntax (Python 3.10+)”# Union via pipe operator (3.10+)def process(value: int | str) -> str: return str(value)
# Optional is just a shorthand for X | Nonedef find_user(user_id: int) -> User | None: return db.query(User).filter_by(id=user_id).first()
# Type alias with the `type` statement (3.12+)type JSONValue = dict[str, "JSONValue"] | list["JSONValue"] | str | int | float | bool | None
def parse(data: JSONValue) -> None: ...Generics, Protocols, and TypeVar
Section titled “Generics, Protocols, and TypeVar”from typing import TypeVar, Generic, Protocol, Anyfrom collections.abc import Sequence
T = TypeVar("T")
class Stack(Generic[T]): def __init__(self) -> None: self._items: list[T] = []
def push(self, item: T) -> None: self._items.append(item)
def pop(self) -> T: return self._items.pop()
def __len__(self) -> int: return len(self._items)
# Use Sequence instead of list for function parameters -- it accepts# any iterable sequence type (list, tuple, range, etc.)def first_item(items: Sequence[T]) -> T | None: return items[0] if items else None
# Protocol for structural subtyping (duck typing with static checking)class Drawable(Protocol): def draw(self, surface: Any) -> None: ...
class Circle: def draw(self, surface: Any) -> None: print(f"Drawing circle on {surface}")
class Rectangle: def draw(self, surface: Any) -> None: print(f"Drawing rectangle on {surface}")
def render_all(objects: list[Drawable], surface: Any) -> None: for obj in objects: obj.draw(surface) # type checker verifies draw() existsWhen to Use # type: ignore
Section titled “When to Use # type: ignore”The # type: ignore comment should be a last resort, used only when the type checker is genuinely Wrong and the code is correct. Every # type: ignore should have a brief explanation:
result = complex_third_party_function(data) # type: ignore[return-value] # bug in library stubsIf you find yourself writing # type: ignore frequently, the problem is likely with your type Annotations or the library’s stubs, not the type checker. File issues upstream or contribute fixes To typeshed.
Virtual Environments
Section titled “Virtual Environments”A virtual environment is an isolated Python installation directory that contains its own interpreter Executable and its own site-packages directory. Without virtual environments, every project on a System shares the same global site-packagesWhich means installing a package for one project can Break another project that depends on a different version of the same package.
# Create a virtual environmentpython -m venv .venv
# Activate it# On macOS/Linux:source .venv/bin/activate# On Windows:# .venv\Scripts\activate
# Install packages into the isolated environmentpip install requests
# Verify isolationpython -c "import sys; print(sys.prefix)"# Outputs: /path/to/project/.venv
# Deactivate when donedeactivateWhy Not Use the System Python
Section titled “Why Not Use the System Python”The system Python is managed by the OS package manager (apt, brew, etc.). Installing packages into It with pip can overwrite files that the OS depends on. More importantly, the system Python’s Package versions are pinned to whatever the OS distributor chose, which may conflict with your Project’s requirements. Virtual environments eliminate both problems.
venv vs conda vs pyenv
Section titled “venv vs conda vs pyenv”| Tool | Purpose | Isolation Scope |
|---|---|---|
venv | Creates virtual environments | Python packages only |
conda | Manages environments with multiple languages | Python + non-Python dependencies (C libraries, R, Node) |
pyenv | Installs multiple Python versions | Python interpreters only (no package isolation) |
For most Python projects, venv is sufficient. Use conda when your project depends on non-Python Scientific libraries (NumPy, SciPy) that need specific C/Fortran compilers. Use pyenv when you Need to test against multiple Python interpreter versions.
Dependency Management
Section titled “Dependency Management”requirements.txt
Section titled “requirements.txt”The simplest approach. Lists exact package versions with their hashes for reproducibility.
requests==2.32.3httpx==0.27.2pydantic==2.10.4pip install -r requirements.txtpip freeze > requirements-lock.txt # pin all transitive depsLimitations: requirements.txt does not distinguish between direct and transitive dependencies. It cannot specify metadata like entry points, optional dependency groups, or build system Requirements. It provides no mechanism for managing development vs production dependency groups.
pyproject.toml and Modern Tooling
Section titled “pyproject.toml and Modern Tooling”PEP 517 and PEP 621 established pyproject.toml as the single source of truth for project metadata, Build configuration, dependency specification, and tool configuration. It replaces the fragmented Ecosystem of setup.py``setup.cfg``requirements.txt``tox.ini``.flake8``.mypy.iniAnd .isort.cfg with one file.
[project]name = "myapp"version = "2.1.0"description = "A well-structured Python application"requires-python = ">=3.11"dependencies = [ "httpx>=0.27", "pydantic>=2.0", "sqlalchemy>=2.0",]
[project.optional-dependencies]dev = [ "pytest>=8.0", "pytest-cov>=5.0", "ruff>=0.4", "mypy>=1.10", "hypothesis>=6.100",]docs = [ "mkdocs>=1.5", "mkdocs-material>=9.5",]
[build-system]requires = ["hatchling"]build-backend = "hatchling.build"
[tool.ruff]target-version = "py311"line-length = 88
[tool.ruff.lint]select = ["E", "F", "I", "N", "UP", "B", "SIM", "RUF"]
[tool.mypy]strict = truepython_version = "3.11"
[tool.pytest.ini_options]testpaths = ["tests"]addopts = "-v --tb=short"Poetry vs PDM vs uv
Section titled “Poetry vs PDM vs uv”| Feature | Poetry | PDM | uv |
|---|---|---|---|
| Lock file | poetry.lock | pdm.lock | uv.lock |
| Resolver speed | Slow (Python implementation) | Medium (Rust-based since 2.x) | Fast (Rust-based) |
| PEP 621 compliance | Partial (own metadata format) | Full | Full |
| Package manager | pip (via virtualenv) | pip (via virtualenv) | Own installer (Rust) |
| Workspace support | Yes | Yes | Yes |
| Build backend | poetry-core | pdm-backend | hatchling (default) |
WHY pyproject.toml Over setup.py
Section titled “WHY pyproject.toml Over setup.py”setup.py has several fundamental problems:
Execution at import time.
setup.pyis executed by runningpython setup.pyWhich means arbitrary code runs during package inspection. This makes it impossible to safely determine a package’s metadata without executing potentially malicious code. PEP 517 solved this by defining a standard interface where the build frontend tells the build backend what to do via subprocess calls, rather than executing the project’s code directly.Scattered configuration. Before
pyproject.tomlProject metadata lived insetup.pyorsetup.cfgTool configuration lived in.flake8``.mypy.ini``.isort.cfg``tox.inipytest.ini``MANIFEST.inAndpyproject.toml(for build system). A single project could have ten or more configuration files.pyproject.tomlconsolidates all tool configuration under[tool.*]sections.No standard dependency format.
install_requiresinsetup.pyandrequirements.txtused different formats.pyproject.tomlprovides[project.dependencies]and[project.optional-dependencies]as the standard format.Dynamic vs static metadata.
setup.pyencourages dynamic metadata generation (computing the version from git tags, reading README files at build time), which makes it impossible to determine the package’s metadata without executing code. PEP 621 defines static metadata fields inpyproject.tomlthat can be read without execution.
setup.py is still supported for backward compatibility, but new projects should use pyproject.toml exclusively.
Project Structure
Section titled “Project Structure”Flat Layout
Section titled “Flat Layout”myproject/ pyproject.toml README.md myproject/ __init__.py main.py models.py services.py tests/ test_main.py test_models.pysrc Layout
Section titled “src Layout”myproject/ pyproject.toml README.md src/ myproject/ __init__.py main.py models.py services.py tests/ test_main.py test_models.pyWHY src Layout Over Flat Layout
Section titled “WHY src Layout Over Flat Layout”The src layout places the importable package inside a src/ directory. This seems like Unnecessary nesting, but it solves a subtle and serious problem.
When you run pip install -e . (editable install) with a flat layout, Python adds the project root To sys.path. This means import myproject resolves to the local directory before any Installed version. If you have requests installed globally and also have a file called requests.py in your project root, your local file shadows the real package. This can cause Extremely confusing import errors, especially when the shadowing only happens during testing (because test runners often add the project root to sys.path).
The src layout prevents this entirely. The project root is on sys.pathBut there is no myproject/ directory at the root — it is nested inside src/. The only way to import myproject Is through the installed package entry in site-packagesWhich points to src/myproject/ via a .pth file or symlink. This guarantees that imports are always resolved through the installed Package, exactly as they would be in production.
# With flat layout, this can happen during testing:import myproject.models # resolves to ./myproject/models.py (local)# But if you installed a different version of myproject globally,# the import silently uses the local version instead of the installed one.
# With src layout, this cannot happen:# tests/test_models.pyimport myproject.models # resolves to .venv/lib/.../myproject/models.py (installed)# Always uses the installed version, matching production behavior.A secondary benefit: the src layout makes it obvious which files are part of the package and which Are project configuration. Everything under src/ is package code. Everything at the root is Project infrastructure.
Recommended Directory Structure
Section titled “Recommended Directory Structure”myproject/ pyproject.toml README.md .gitignore .github/ workflows/ ci.yml src/ myproject/ __init__.py __main__.py models.py services.py utils.py tests/ __init__.py conftest.py test_models.py test_services.pyTesting with pytest
Section titled “Testing with pytest”Why pytest Over unittest
Section titled “Why pytest Over unittest”The standard library unittest framework uses class-based test organization with verbose assertion Methods (self.assertEqual``self.assertTrue``self.assertRaises). pytest uses plain functions With native assert statements and provides powerful features like parametrized tests, fixtures, And plugin-based discovery — all without requiring inheritance from a test base class.
# unittest style -- verbose and rigidimport unittest
class TestCalculator(unittest.TestCase): def test_add(self): self.assertEqual(1 + 1, 2)
def test_divide_by_zero(self): with self.assertRaises(ZeroDivisionError): 1 / 0
# pytest style -- concise and Pythonicdef test_add(): assert 1 + 1 == 2
def test_divide_by_zero(): with pytest.raises(ZeroDivisionError): 1 / 0The assert statement in pytest is rewritten at import time to provide detailed failure messages. When assert x == y fails, pytest displays the values of x and y automatically, without any Special assertion methods.
Fixtures
Section titled “Fixtures”Fixtures are pytest’s dependency injection system. They provide a reusable way to set up and tear Down test resources.
import pytestfrom myproject.models import User, Session
@pytest.fixturedef db_session(): """Creates a database session that rolls back after each test.""" session = Session() session.begin_nested() yield session session.rollback() session.close()
@pytest.fixturedef sample_user(db_session): """Creates a user that exists in the database for the test.""" user = User(name="Alice", email="alice@example.com") db_session.add(user) db_session.commit() return user
def test_user_retrieval(db_session, sample_user): found = db_session.query(User).filter_by(name="Alice").first() assert found is not None assert found.email == "alice@example.com"
def test_user_update(db_session, sample_user): sample_user.name = "Bob" db_session.commit() found = db_session.query(User).filter_by(name="Bob").first() assert found is not NoneFixtures have scopes that control when they are created and torn down:
| Scope | Created once per… | Common use |
|---|---|---|
function | test function | Database sessions, mock objects |
class | test class | Shared class-level setup |
module | test module | Expensive resources (Docker containers) |
package | test package | Package-level state |
session | test run | Global services, process-level state |
Parametrize
Section titled “Parametrize”Parametrized tests replace copy-pasted test functions with a data-driven approach. One test function Can cover dozens of input combinations.
import pytest
@pytest.mark.parametrize( "input_value, expected", [ ("hello", "HELLO"), ("World", "WORLD"), ("", ""), ("123", "123"), ("already UPPER", "ALREADY UPPER"), ],)def test_uppercase(input_value, expected): assert input_value.upper() == expected
@pytest.mark.parametrize("x", [0, 1, 2, 3, 10, 100])@pytest.mark.parametrize("y", [0, 1, 5, 10])def test_multiply(x, y): assert x * y == y * x # commutativity propertyMocking
Section titled “Mocking”from unittest.mock import MagicMock, patch, PropertyMock
def test_external_api_call(): with patch("myproject.services.httpx.Client") as MockClient: mock_response = MagicMock() mock_response.status_code = 200 mock_response.json.return_value = {"id": 1, "name": "Alice"}
mock_instance = MockClient.return_value.__enter__.return_value mock_instance.get.return_value = mock_response
from myproject.services import fetch_user result = fetch_user(1)
assert result["name"] == "Alice" mock_instance.get.assert_called_once_with("https://api.example.com/users/1")Mocking has a critical danger: if you mock the wrong path, your test can pass even when the real Code is broken. Always mock at the boundary where the dependency is used, not where it is defined. If services.py imports httpxMock myproject.services.httpx (where it is used), not httpx Directly (where it is defined).
Coverage
Section titled “Coverage”# Run tests with coveragepytest --cov=myproject --cov-report=term-missing --cov-report=html tests/
# Enforce a minimum coverage threshold in CIpytest --cov=myproject --cov-fail-under=80 tests/Coverage is a useful metric, but treat it as a minimum bar, not a quality target. 100% coverage does Not mean bug-free code — it means every line was executed, not that every meaningful input Combination was tested. Conversely, 60% coverage on a well-tested critical path is more valuable Than 95% coverage achieved by testing trivial getters and setters.
Configuration
Section titled “Configuration”[tool.pytest.ini_options]testpaths = ["tests"]addopts = [ "-v", "--tb=short", "--strict-markers",]markers = [ "slow: marks tests as slow (deselect with '-m \"not slow\"')", "integration: marks tests that require external services",]Linting and Type Checking
Section titled “Linting and Type Checking”Comparison of Linting and Formatting Tools
Section titled “Comparison of Linting and Formatting Tools”| Tool | Language | Checks | Formatter | Speed | Notes |
|---|---|---|---|---|---|
| ruff | Rust | Flake8 + isort + pydocstyle + many more | Built-in (Black-compatible) | Very fast (10-100x flake8) | Replaces flake8, isort, Black, pydocstyle, and more |
| flake8 | Python | PEP 8 + pycodestyle + pyflakes | No | Slow | Requires plugins for isort, Black, etc. |
| pylint | Python | Deep analysis, refactoring suggestions | No | Very slow | Many false positives; highly configurable |
| Black | Python | Formatting only | Yes | Medium | Uncompromising formatting; no linting |
| isort | Python | Import sorting only | Yes | Medium | Replaced by ruff’s import sorting |
| mypy | Python | Static type checking | No | Medium | Reference type checker |
| pyright | TypeScript/Python | Static type checking | No | Fast | Microsoft’s type checker; stricter by default |
| basedpyright | TypeScript/Python | Static type checking | No | Fast | Fork of pyright with additional features |
WHY Ruff Over flake8/pylint
Section titled “WHY Ruff Over flake8/pylint”ruff replaces an entire pipeline of separate tools (flake8, isort, Black, pydocstyle, pyupgrade, And dozens of flake8 plugins) with a single Rust binary. The practical advantages are:
Speed.
ruffis 10-100x faster than flake8 because it is written in Rust. On a large codebase,ruffcompletes in milliseconds where flake8 takes seconds. This makes it practical to run on every file save in an editor, providing instant feedback.No plugin management. With flake8, each additional check requires installing and configuring a separate plugin (flake8-bugbear, flake8-comprehensions, flake8-simplify, etc.). Each plugin has its own version constraints and configuration format.
ruffincludes equivalent rules for all of these out of the box, selectable via rule codes.Unified configuration.
ruffis configured entirely throughpyproject.toml. There is no separate.flake8``.isort.cfgOrpyproject.toml[tool.black]section. One tool, one configuration block.Built-in formatting.
ruff formatis compatible with Black’s formatting style (and can be configured to diverge where desired). This eliminates the need to run Black as a separate step.Autofix.
ruffcan automatically fix many issues (ruff check --fix), including import sorting, unused imports, and simple refactoring suggestions. This reduces the cycle of “run linter, see errors, manually fix, re-run linter.”
[tool.ruff]target-version = "py311"line-length = 88
[tool.ruff.lint]select = [ "E", # pycodestyle errors "W", # pycodestyle warnings "F", # pyflakes "I", # isort "N", # pep8-naming "UP", # pyupgrade "B", # flake8-bugbear "SIM", # flake8-simplify "RUF", # ruff-specific rules]ignore = ["E501"] # line too long (handled by formatter)
[tool.ruff.lint.isort]known-first-party = ["myproject"]Type Checking with mypy and pyright
Section titled “Type Checking with mypy and pyright”[tool.mypy]strict = truepython_version = "3.11"warn_return_any = truewarn_unused_configs = truedisallow_untyped_defs = true
[tool.pyright]pythonVersion = "3.11"typeCheckingMode = "strict"Run both in CI. mypy and pyright catch different categories of errors. mypy is more permissive By default (even in strict mode), while pyright is more aggressive about catching potential None dereferences and type narrowing edge cases. When both pass, you have high confidence in your Type annotations.
mypy src/pyright src/Error Handling Patterns
Section titled “Error Handling Patterns”Exceptions Over Return Codes
Section titled “Exceptions Over Return Codes”Python uses exceptions for error handling, not return codes. This is a fundamental design choice: Exceptions propagate automatically through the call stack, while return codes require explicit Checking at every level. A function ten frames deep that encounters an error cannot communicate it To the caller through a return value without every intermediate function explicitly checking and Re-raising.
# Bad: return code pattern (Java/C style, un-Pythonic)def read_config(path): if not os.path.exists(path): return None, "File not found" try: with open(path) as f: return json.load(f), None except json.JSONDecodeError as e: return None, str(e)
config, error = read_config("config.json")if error: print(f"Error: {error}") sys.exit(1)
# Good: exception pattern (Pythonic)def read_config(path: str) -> dict: """Reads configuration from a JSON file.
Raises: FileNotFoundError: If the config file does not exist. json.JSONDecodeError: If the file contains invalid JSON. """ with open(path) as f: return json.load(f)Catch Specific Exceptions
Section titled “Catch Specific Exceptions”# Bad: bare except -- catches KeyboardInterrupt, SystemExit, GeneratorExittry: result = process(data)except: result = None
# Bad: catching Exception is only slightly bettertry: result = process(data)except Exception: result = None
# Good: catch the specific exception you expecttry: result = process(data)except ValueError: logger.warning("Invalid data format, using default") result = get_default()
# Good: catch multiple specific exceptions when they share a handlertry: result = process(data)except (ValueError, TypeError) as e: logger.warning(f"Data error: {e}") result = get_default()Never catch BaseException or bare except in application code. These catch KeyboardInterrupt (Ctrl+C) and SystemExit (sys.exit), which prevents the user from terminating the program. Catch Exception only when you have a specific recovery strategy.
Define Custom Exception Hierarchies
Section titled “Define Custom Exception Hierarchies”class AppError(Exception): """Base exception for all application errors."""
class DatabaseError(AppError): """Errors related to database operations."""
class ConnectionError(DatabaseError): """Failed to connect to the database."""
class QueryError(DatabaseError): """A database query failed."""
class ValidationError(AppError): """Input data failed validation."""
def get_user(user_id: int) -> User: try: return db.query(User).filter_by(id=user_id).one() except NoResultFound: raise ValidationError(f"User {user_id} not found") from NoneUse raise ... from None to suppress the original exception’s traceback when it is not useful (e.g., when translating a low-level exception into a domain exception). Use raise ... from original when the chain of exceptions provides useful debugging context.
Context Managers for Resource Management
Section titled “Context Managers for Resource Management”The Problem Context Managers Solve
Section titled “The Problem Context Managers Solve”When a function acquires a resource (file handle, database connection, network socket, lock), it Must release that resource regardless of whether the function completes normally or raises an Exception. Manual release requires a try/finally block, which is verbose and error-prone — it is Easy to forget the finally clause.
# Manual resource management -- verbose and easy to forgetdef process_file(path): f = open(path) try: data = f.read() return transform(data) finally: f.close()
# Context manager -- concise and correct by constructiondef process_file(path): with open(path) as f: data = f.read() return transform(data)Writing Custom Context Managers
Section titled “Writing Custom Context Managers”from contextlib import contextmanagerimport time
@contextmanagerdef measure_time(label: str): """Measures and logs the time taken by a block of code.""" start = time.perf_counter() yield elapsed = time.perf_counter() - start print(f"[{label}] {elapsed:.4f}s")
# Usagewith measure_time("database query"): results = db.execute("SELECT * FROM large_table")
@contextmanagerdef database_transaction(session): """Provides a database transaction that commits on success, rolls back on failure.""" try: yield session session.commit() except Exception: session.rollback() raise
# Usagewith database_transaction(session): session.add(new_record) session.add(another_record) # If anything raises here, the transaction is rolled back automaticallyFor more complex cases, implement the context manager protocol directly:
class ManagedConnection: def __init__(self, host: str, port: int): self.host = host self.port = port self._socket = None
def __enter__(self): self._socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) self._socket.connect((self.host, self.port)) return self
def __exit__(self, exc_type, exc_val, exc_tb): self._socket.close() self._socket = None return False # do not suppress exceptions
def send(self, data: bytes) -> None: self._socket.sendall(data)contextlib Utilities
Section titled “contextlib Utilities”from contextlib import suppress, redirect_stdout, closing
# suppress replaces try/except pass for expected exceptionswith suppress(FileNotFoundError): os.remove("temp_file.txt")
# redirect_stdout captures print outputimport iobuffer = io.StringIO()with redirect_stdout(buffer): print("This goes to the buffer, not stdout")output = buffer.getvalue()
# closing works with objects that have a close() method but no context managerimport urllib.requestwith closing(urllib.request.urlopen("https://example.com")) as response: data = response.read()Property-Based Testing with Hypothesis
Section titled “Property-Based Testing with Hypothesis”Unit tests verify that a function produces the correct output for specific, hand-picked inputs. Property-based testing verifies that a function satisfies a property for arbitrary inputs Generated automatically. This finds edge cases that a human tester would never think to write.
from hypothesis import given, strategies as st, assume
# Strategy: generate arbitrary lists of integersint_lists = st.lists(st.integers(min_value=-1000, max_value=1000))
@given(int_lists)def test_sort_preserves_length(items): """Sorting should not change the number of elements.""" assert len(sorted(items)) == len(items)
@given(int_lists)def test_sort_preserves_elements(items): """Sorting should not add or remove elements.""" assert sorted(sorted(items)) == sorted(items)
@given(st.lists(st.integers()))def test_sort_is_idempotent(items): """Sorting an already-sorted list returns the same list.""" assert sorted(sorted(items)) == sorted(items)
@given(st.lists(st.integers()))def test_reverse_reverses_sort(items): """Reversing a sorted list gives descending order.""" assert sorted(items, reverse=True) == list(reversed(sorted(items)))
# Use assume to filter generated inputs@given(st.integers(), st.integers())def test_division_properties(numerator, denominator): assume(denominator != 0) result = numerator / denominator assert result * denominator == pytest.approx(numerator)
# Custom strategiesfrom myproject.models import User
user_strategy = st.builds( User, name=st.text(min_size=1, max_size=100, alphabet=st.characters(whitelist_categories=('L', 'N', 's'))), email=st.emails(), age=st.integers(min_value=0, max_value=150),)
@given(user_strategy)def test_user_name_not_empty(user): assert len(user.name) > 0
@given(st.text())def test_encode_decode_roundtrip(text): """Encoding then decoding should return the original text.""" assert text.encode("utf-8").decode("utf-8") == textThe key insight of property-based testing is that you identify properties of your code (invariants, round-trip behaviors, idempotency) rather than specific input/output pairs. The testing Framework then generates thousands of random inputs and verifies that the property holds for all of Them. When a failure is found, Hypothesis automatically minimizes the failing input to the smallest Counterexample, making it easy to understand and fix the bug.
Docstrings
Section titled “Docstrings”Google Style
Section titled “Google Style”def fetch_user(user_id: int, include_orders: bool = False) -> dict: """Fetches a user from the database by ID.
Args: user_id: The unique identifier of the user. Must be a positive integer. include_orders: Whether to include the user's order history in the response. Defaults to False.
Returns: A dictionary containing user data with keys 'id', 'name', 'email'. If include_orders is True, also includes 'orders' with a list of order dictionaries.
Raises: ValueError: If user_id is not a positive integer. NotFoundError: If no user exists with the given ID.
Examples: >>> fetch_user(42) {'id': 42, 'name': "Alice'', "email': "alice@example.com''}
>>> fetch_user(42, include_orders=True) {"id': 42, 'name': "Alice'', "email': "alice@example.com'', "orders': [...]} """ if user_id <= 0: raise ValueError("user_id must be a positive integer") ...NumPy Style
Section titled “NumPy Style”def calculate_moving_average(data, window_size): """Calculate the moving average of a data series.
Parameters ---------- data : array_like Input data series. Must be a 1-dimensional sequence of numeric values. window_size : int The number of data points to include in each average. Must be positive and less than or equal to the length of data.
Returns ------- numpy.ndarray An array containing the moving averages. The output has length ``len(data) - window_size + 1``.
Raises ------ ValueError If window_size is not positive or exceeds the data length.
See Also -------- exponential_moving_average : Weighted moving average with exponential decay.
Notes ----- Uses a convolution-based implementation for numerical stability. """ ...Sphinx/reStructuredText Style
Section titled “Sphinx/reStructuredText Style”def parse_config(path): """Parse a configuration file and return a validated config object.
:param path: Path to the configuration file. :type path: str :returns: A validated configuration object. :rtype: Config :raises ConfigError: If the configuration file is malformed. :raises FileNotFoundError: If the file does not exist. """ ...Which Style to Choose
Section titled “Which Style to Choose”Pick one style and apply it consistently across the entire project. Google style is the most Readable for documentation generated by tools like Sphinx (with the Napoleon extension) or MkDocs (with the mkdocstrings plugin). NumPy style is standard in scientific computing. Sphinx style is the Oldest and most verbose. All three convey the same information; the differences are purely Formatting.
Packaging
Section titled “Packaging”pyproject.toml Build Configuration
Section titled “pyproject.toml Build Configuration”[build-system]requires = ["hatchling"]build-backend = "hatchling.build"
[project]name = "myapp"version = "2.1.0"description = "A well-structured Python application"readme = "README.md"license = "MIT"requires-python = ">=3.11"authors = [ {name = "Jane Doe", email = "jane@example.com"},]classifiers = [ "Development Status :: 4 - Beta", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.11", "Programming Language :: Python :: 3.12", "License :: OSI Approved :: MIT License",]dependencies = [ "httpx>=0.27", "pydantic>=2.0",]
[project.scripts]myapp = "myapp.__main__:main"
[project.entry-points."myapp.plugins"]csv_export = "myapp.plugins.csv:CSVExporter"
[project.urls]Homepage = "https://github.com/janedoe/myapp"Documentation = "https://myapp.readthedocs.io"Repository = "https://github.com/janedoe/myapp"Build Backends
Section titled “Build Backends”| Backend | Maintainer | Notes |
|---|---|---|
hatchling | Hatch | Simple, fast, good default choice |
setuptools | pypa | Legacy; still widely used, supports setup.py fallback |
flit-core | flit | Minimal; for pure-Python packages with no complex build needs |
pdm-backend | PDM | PEP 621 compliant; good for complex packages |
Build and Publish
Section titled “Build and Publish”# Build the packagepip install buildpython -m build
# The build command produces dist/myapp-2.1.0.tar.gz (sdist)# and dist/myapp-2.1.0-py3-none-any.whl (wheel)
# Publish to PyPI (requires twine)pip install twinetwine upload dist/*
# Or use the newer build-based uploadpython -m build --uploadsetup.py Legacy
Section titled “setup.py Legacy”setup.py still exists in many projects and is required by some older tools. If you encounter it, You should understand what it does, but you should not create new projects with it.
# Legacy setup.py -- do not use for new projectsfrom setuptools import setup, find_packages
setup( name="myapp", version="2.1.0", packages=find_packages(), install_requires=[ "httpx>=0.27", "pydantic>=2.0", ],)This file has three problems that pyproject.toml solves: it requires execution to read metadata (security risk), it mixes logic with declaration (the setup() call can contain arbitrary Python Code), and it cannot store tool configuration. Migrate to pyproject.toml when you encounter setup.py in existing projects.
Putting It All Together
Section titled “Putting It All Together”A well-configured Python project uses these tools in concert. The development workflow looks like This:
Editor integration. Your editor runs
ruffon save for instant linting and formatting feedback. It runspyrightormypyin the background for type checking.Pre-commit hooks.
pre-commitrunsruff check --fix``ruff format``mypyAndpytestbefore every commit, preventing violations from entering the repository.CI pipeline. GitHub Actions runs the full suite:
ruff check``ruff format --check``mypypytest --covAnd any integration tests. The pipeline fails if any step fails.Dependency management. Dependencies are declared in
pyproject.tomland locked inuv.lock(orpoetry.lock). The lock file ensures reproducible installs across all environments.
name: CIon: [push, pull_request]jobs: test: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: actions/setup-python@v5 with: python-version: '3.12' - run: pip install uv - run: uv sync --all-extras - run: uv run ruff check - run: uv run ruff format --check - run: uv run mypy src/ - run: uv run pytest --cov=myproject --cov-fail-under=80This layered approach catches problems at the earliest possible stage: the editor catches them as You type, pre-commit catches them before they are committed, and CI catches anything that slips Through both. The result is a codebase where style violations, type errors, and failing tests are Fixed immediately, not discovered weeks later during code review.
Common Pitfalls
Section titled “Common Pitfalls”Confusing authentication (who you are) with authorisation (what you can do) in security contexts.
Mixing up Big O, Big , and Big notation. Big O is an upper bound, not necessarily tight.
Forgetting that average-case for quicksort becomes worst-case on already sorted input.
Writing pseudocode that is too language-specific rather than using standard algorithmic constructs.
Summary
Section titled “Summary”The key principles covered in this topic are linked in the sub-pages above. Focus on understanding the definitions, applying the formulas or frameworks, and evaluating strengths and limitations of each approach.
Worked Examples
Section titled “Worked Examples”Worked examples demonstrating the application of key concepts are covered in the detailed sub-pages linked above.