Essential Modules

The os.path module is a collection of free functions that operate on path strings. It was designed In an era before Python had a coherent object model for paths. pathlib (Python 3.4+) replaces this With an object-oriented API where a Path instance represents a single filesystem path.

The argument for pathlib is not aesthetic preference. It is about composability and correctness:

1. Method chaining. os.path requires you to thread a string through successive function calls: os.path.join(os.path.dirname(os.path.abspath(p)), "config.json'). The equivalent Path expression is Path(p).resolve().parent / 'config.json'. The / operator is overloaded on PurePosixPath and PureWindowsPath to join path components, which reads as natural composition rather than nested function calls.

2. No silent truncation. os.path.join('/etc', '/var') returns /var — an absolute second argument silently discards the first. Path('/etc') / '/var' raises no error but returns PosixPath('/var'). Both are surprising, but pathlib at least provides a single consistent type (PosixPath or WindowsPath) whose semantics are visible in the type.

3. Uniform access to filesystem operations. os.path only handles path manipulation. Actual I/O (reading, writing, stat, mkdir, glob) requires importing os``shutil``glob``statAnd others. Path objects carry methods for all of these: .read_text()``.write_text() .stat()``.mkdir()``.glob()``.rename()``.unlink().

4. Cross-platform correctness. os.path relies on the host operating system to determine separator behavior. pathlib exposes PurePosixPath and PureWindowsPath for explicit control when you need to manipulate paths for a different platform (e.g., generating URLs on a Linux server that target Windows).

from pathlib import Path

config_dir = Path.home() / ".config" / "myapp"
config_dir.mkdir(parents=True, exist_ok=True)

config_file = config_dir / "settings.json"
config_file.write_text('{"theme": "dark"}')

content = config_file.read_text()
print(config_file.exists())       # True
print(config_file.stat().st_size) # byte count
print(config_file.suffix)         # '.json'
print(config_file.stem)           # 'settings'

pathlib does not cover everything in the os module. The following still require os directly:

• os.environ: environment variables (pathlib has no equivalent).
• os.chdir()``os.getcwd(): changing and querying the current working directory.
• os.walk(): recursive directory traversal (though Path.rglob() covers most use cases).
• os.umask()``os.getuid()``os.setsid(): low-level process and permission operations.
• os.path.expandvars()``os.path.expanduser(): shell variable expansion (note: Path does expand ~ in constructors but not $VAR).

import os

## These have no pathlib equivalent
pid = os.getpid()
env_home = os.environ.get("HOME", "/tmp")
os.chmod("/tmp/file.txt", 0o644)

Even in new code, some os.path functions are unavoidable or more convenient than their pathlib Equivalents:

import os.path

os.path.exists(p)         # Path(p).exists()
os.path.isfile(p)         # Path(p).is_file()
os.path.isdir(p)          # Path(p).is_dir()
os.path.getsize(p)        # Path(p).stat().st_size
os.path.abspath(p)        # Path(p).resolve()
os.path.basename(p)       # Path(p).name
os.path.dirname(p)        # Path(p).parent
os.path.join(a, b, c)     # Path(a) / b / c
os.path.splitext(p)       # Path(p).suffix, Path(p).stem
os.path.normpath(p)       # Path(p) (constructor normalizes)

from pathlib import Path, PurePosixPath

p = PurePosixPath("/usr/local/bin/python3.12")

print(p.anchor)   # '/'
print(p.drive)    # ''
print(p.parts)    # ('/', 'usr', 'local', 'bin', 'python3.12')
print(p.parent)   # PurePosixPath('/usr/local/bin')
print(p.name)     # 'python3.12'
print(p.stem)     # 'python3.12'
print(p.suffix)   # '.12'
print(p.suffixes) # ['.12']

PurePosixPath and PureWindowsPath perform only string manipulation — no filesystem access. This Is useful for constructing or parsing paths for remote systems.

The sys module exposes the runtime environment: interpreter configuration, the module search path, Reference counting, and process-level control.

sys.argv is a list of strings. sys.argv[0] is the script name (or '-' for stdin). Everything After is a positional argument. It does not handle options, flags, or defaults — for that, use argparse.

import sys

if len(sys.argv) != 3:
    print(f"Usage: {sys.argv[0]} <input> <output>", file=sys.stderr)
    sys.exit(1)

input_file, output_file = sys.argv[1], sys.argv[2]

When you write import fooPython searches for foo in the directories listed in sys.path. The First match wins. The initial value is populated from:

1. The directory containing the script (or the current directory for interactive mode).
2. PYTHONPATH environment variable.
3. Installation-dependent defaults (site-packages).

import sys

print(sys.path[:3])
## ['/home/user/project', '/usr/lib/python312.zip', '/usr/lib/python3.12']

# Temporarily prepend a directory
sys.path.insert(0, "/opt/custom_libs")
import mymodule  # found in /opt/custom_libs first

Modifying sys.path at runtime is fragile. For reproducible imports, use proper package Installation or PYTHONPATH. Mutating sys.path in library code is particularly dangerous because It affects the global import state of the entire process.

sys.modules is a dictionary mapping module names to loaded module objects. The import system Checks this dictionary first — if a module is already loaded, import returns the cached object Without re-executing the module’s code.

import sys
import json

print(sys.modules["json"])  # <module 'json' from '...'>
sys.modules["json"] = None  # breaks all subsequent json imports

This is occasionally useful for reloading modules during development or for testing, but modifying sys.modules in production code is almost always a mistake.

sys.exit() raises SystemExitWhich the interpreter catches at the top level to terminate the Process with the given exit code. Because it is an exception, it can be caught and handled — finally blocks and context managers still execute.

import sys

try:
    sys.exit(42)
except SystemExit as e:
    print(f"Caught exit with code: {e.code}")  # 42
# Process continues normally

This is why sys.exit() is preferred over os._exit(). os._exit() terminates the process Immediately without cleanup: no finally blocks, no atexit handlers, no buffer flushing.

import json

data = {"users": [{"name": "Alice", "active": True}, {"name": "Bob", "active": False}]}

serialized = json.dumps(data, indent=2, sort_keys=True)
deserialized = json.loads(serialized)

print(type(serialized))     # <class 'str'>
print(type(deserialized))   # <class 'dict'>

json.dumps() returns a string. json.dump() writes directly to a file object. The symmetric pair Is json.loads() (from string) and json.load() (from file object).

The default parameter of json.dumps() is a function called for objects that are not natively Serializable (i.e., not dict``list``str``int``float``boolOr None).

from datetime import datetime, date
import json

def serialize_custom(obj):
    if isinstance(obj, datetime):
        return obj.isoformat()
    if isinstance(obj, date):
        return obj.isoformat()
    if isinstance(obj, set):
        return sorted(obj)
    raise TypeError(f"Object of type {type(obj).__name__} is not JSON serializable")

data = {"created": datetime(2025, 6, 4, 14, 0), "tags": {"python", "stdlib"}}
print(json.dumps(data, default=serialize_custom, indent=2))

For more control, subclass json.JSONEncoder and override default():

class CustomEncoder(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, datetime):
            return {"__type__": "datetime", "value": obj.isoformat()}
        return super().default(obj)

class CustomDecoder(json.JSONDecoder):
    def __init__(self, *args, **kwargs):
        super().__init__(object_hook=self._object_hook, *args, **kwargs)

    def _object_hook(self, dct):
        if dct.get("__type__") == "datetime":
            return datetime.fromisoformat(dct["value"])
        return dct

Pickle can serialize functions, classes, and object graphs with cycles. But pickle.loads() on Untrusted data is equivalent to arbitrary code execution — the pickled byte stream can contain Instructions to call any callable, import any module, and execute arbitrary code. For data Interchange between systems or for storage that must survive Python version upgrades, JSON is the Only safe choice.

import pickle

class Node:
    def __init__(self, value, left=None, right=None):
        self.value = value
        self.left = left
        self.right = right

tree = Node(1, Node(2), Node(3))
data = pickle.dumps(tree)
restored = pickle.loads(data)
print(restored.value)  # 1
print(restored.left.value)  # 2

Python’s re module uses a backtracking NFA engine. Patterns are compiled into bytecode that the Engine interprets. Compilation is the expensive step; matching is fast on the compiled pattern.

import re

pattern = re.compile(r'\b(\w+)@(\w+)\.(\w+)\b')
match = pattern.search("Contact alice@example.com or bob@test.org")
print(match.group(0))  # 'alice@example.com'
print(match.group(1))  # 'alice'
print(match.group(2))  # 'example'
print(match.group(3))  # 'com'
print(match.groups())  # ('alice', 'example', 'com')

Always use raw strings (r'...') for regex patterns. Without the raw prefix, \b is interpreted as A backspace character, and \d``\w``\s are interpreted as escape sequences (some of which are Valid in Python strings, producing the wrong character in the regex).

import re

pattern = re.compile(r'(?P<user>\w+)@(?P<domain>[\w.]+)')
match = pattern.match("alice@example.com")
print(match.group("user"))    # 'alice'
print(match.group("domain"))  # 'example.com'
print(match.groupdict())      # {'user': "alice'', "domain': "example.com''}

Non-capturing groups (?:...) participate in alternation and quantification but do not create a Backreference. This prevents group numbering from shifting when you add groups for structural Purposes.

# Non-capturing group for alternation
pattern = re.compile(r"(?:https?|ftp)://([\w./]+)')

import re

text = "Hello\nWorld"

re.findall(r'^\w+', text)            # ['Hello'] (default: ^ matches start of string)
re.findall(r'^\w+', text, re.MULTILINE)  # ['Hello', 'World']

re.findall(r'hello', "Hello World")   # []
re.findall(r'hello', "Hello World", re.IGNORECASE)  # ['Hello']

# Combining flags with pipe
re.findall(r'^\w+', text, re.MULTILINE | re.IGNORECASE)

• Types and Variables — Standard library modules work with Python’s built-in types including lists, dicts, and strings.
• File I/O — The os and pathlib modules provide file system interaction beyond basic file reading and writing.
• Context Managers — Many standard library resources support the context manager protocol for safe resource management.