Python Regular Expressions

The Four Core Calls

The re module gives four calls for matching. Each call has a different job.

re.search finds the first match anywhere in the string. re.match looks only at the start of the string. re.fullmatch needs the pattern to cover the whole string. re.findall returns a list of all matches.

Use re.search when the match can be anywhere. Use re.match when the text must start with the pattern. Use re.fullmatch when the whole text must match. Use re.findall when you need every match.

import re

text = "cat dog cat"
print(re.search(r"dog", text).group())
print(re.match(r"cat", text).group())
print(re.fullmatch(r"cat dog cat", text).group())
print(re.findall(r"cat", text))

Groups and Named Groups

Parentheses make a group. A group lets you read part of the match. The call match.group(1) returns the first group.

A named group uses the syntax (?P<name>...). You can read it with match.group("name"). Named groups make the code easier to read.

The code below finds a date. The pattern has three named groups for year, month, and day. The print calls show each part.

import re

pattern = r"(?P<year>\d{4})-(?P<month>\d{2})-(?P<day>\d{2})"
match = re.search(pattern, "Date: 2024-05-17")
print(match.group("year"))
print(match.group("month"))
print(match.group("day"))

Flags

Flags change how the pattern works. You pass them as the second argument to the call.

re.IGNORECASE makes the match ignore upper and lower case. re.MULTILINE makes ^ and $ match at the start and end of each line.

The code below uses both flags. The pattern finds lines that start with "item". The match ignores case.

import re

text = "Item one\nitem two\nother"
pattern = r"^item.*"
print(re.findall(pattern, text, re.IGNORECASE | re.MULTILINE))

Replacement and Compiled Patterns

re.sub replaces each match with a new string. You can use a group in the replacement. The syntax \1 refers to group 1.

re.compile builds a pattern object one time. Use it when the same pattern runs in a loop. The pattern object has the same calls, such as search and sub.

The code below replaces a name. It swaps the first and last name. The compiled pattern runs the same job many times.

import re

pattern = re.compile(r"(\w+) (\w+)")
print(pattern.sub(r"\2 \1", "Ada Lovelace"))
for name in ["Alan Turing", "Grace Hopper"]:
    print(pattern.sub(r"\2 \1", name))

Greedy and Non-Greedy Matching

A quantifier such as * or + is greedy. It takes as much text as it can. This is a common mistake. The pattern <.*> on "<a><b>" matches the whole string.

Add a question mark to make the quantifier non-greedy. The pattern <.*?> stops at the first match. On "<a><b>" it finds "<a>" first.

Use the non-greedy form when you want the smallest match. Use the greedy form when you want the largest match.

import re

text = "<a><b>"
print(re.findall(r"<.*>", text))
print(re.findall(r"<.*?>", text))

Why Raw Strings Matter

A normal Python string treats a backslash as an escape. The string "\d" is not a backslash and a d. Python may change it or raise a warning.

A raw string starts with r. It keeps each backslash as a backslash. Write patterns as raw strings, such as r"\d+".

The code below shows the difference. The raw string keeps the backslash. The normal string does not.

print(r"\d")
print(len(r"\d"))
print(len("\d"))

Common questions

What is the difference between search and match?

re.search finds a match anywhere in the string. re.match looks only at the start of the string.

Why does my pattern match too much?

A greedy quantifier such as .* takes as much text as it can. Add a question mark, as in .*?, to get the smallest match.

Do I need re.compile?

No, the module calls work without it. Use re.compile when the same pattern runs many times, such as in a loop.