Beware! A new Python is approaching!
The next big Python release, Python 3.10, should happen in October. Even though that is still a few months away, we already know what to expect. The 3.10 branch went to feature freeze when the first beta version (Python 3.10.0b1) was released on May 3rd, 2021.
Let’s have a peek at the new features and other changes.
Overview
The most fascinating feature in 3.10 is Structural Pattern Matching. It might look like a simple switch statement, but it’s actually so much more than that.
The second important set of changes are improving Python’s interaction with static type checkers. Included is the much-awaited support for using X | Y syntax for Union types and a couple of other typing-related changes.
A bit of a surprise was one change of plans after the latest alpha release: Postponed evaluation of annotations (PEP 563) was supposed to be the default in 3.10, but that change got postponed to Python 3.11.
There is also a bunch of other changes which won’t be covered here. See the whole list of changes in Python 3.10 change log.
Structural Pattern Matching
Structural pattern matching is a quite powerful new feature. It allows detecting the shape and type of a data structure and assigning selected properties of it to variables. Its form is twofold: First, there is a match statement, which states what will be matched. Then follows one or more case statements. Each case statement specifies a pattern and a block of actions to take if the pattern matches. Let’s check a few examples to clarify.
Pattern Matching example 1
Suppose that we’re building an application that reads commands from the user as text and we need to act based on form of the text that the user entered. That kind of logic can be implemented very easily with pattern matching as you can see in the analyze_input_from_user function below.
def analyze_input_from_user():
text = input("Enter some text: ")
match text.split():
case []:
print("Got nothing")
case [("add" | "subtract") as cmd, a, b]:
print(f"Got operation {cmd} with parameters {a} and {b}")
case [x] if x.isdigit():
number = int(x)
print(f"Got a single integer number: {number}")
case [x] if re.match(r"^[+-]?\\d+(.\\d*)?$", x):
number = float(x)
print(f"Got a single floating point number: {number:.3f}")
case [word]: # Must be non-numeric, since numerics are matched above
print(f"Got a single non-numeric word: {word}")
case _:
print("Got something else")To implement similar functionality with Python 3.9 or older, the code gets a few lines longer and somewhat harder to read, because the expected patterns of the user input are no longer as visual.
def analyze_input_from_user_the_old_way():
text = input("Enter some text: ")
words = text.split()
if len(words) == 0:
print("Got nothing")
elif len(words) == 3 and words[0] in ["add", "subtract"]:
(cmd, a, b) = words
print(f"Got operation {cmd} with parameters {a} and {b}")
elif len(words) == 1 and words[0].isdigit():
number = int(words[0])
print(f"Got a single integer number: {number}")
elif len(words) == 1 and re.match(r"^[+-]?\\d+(.\\d*)?$", words[0]):
number = float(words[0])
print(f"Got a single floating point number: {number:.3f}")
elif len(words) == 1:
word = words[0]
print(f"Got a single non-numeric word: {word}")
else:
print("Got something else")Pattern matching example 2
Here’s another use case where pattern matching could be beneficial. Let’s say that we receive a list of items from an API which combines several kind of issues to a single list. We have to process them based on the form of each item. Our data and calling code could look like this:
issues = [
{
'issuer': "Mr Praline",
'date': date(1969, 12, 7),
'type': 'complaint',
'title': "a Dead Parrot",
},
{
'issuer': "A man",
'date': date(1972, 11, 2),
'type': 'request',
'subject': "an argument",
'location': "the Argument Clinic",
},
"Now something completely different",
]def print_info_about_issues():
for issue in issues:
print_info_about_issue(issue)
Processing of each issue could utilize a dictionary kind of pattern. See print_info_about_issue.
def print_info_about_issue(issue):
match issue:
case {'type': 'complaint', 'date': d, 'title': t, 'issuer': i}:
print(f"{i} complained about {t} on {d}.") case {
'type': 'request',
'date': date(year=y),
'subject': s,
'location': l,
'issuer': i,
}:
print(f"{i} requested to have {s} on {y} at {l}.") case str(text):
print(f"Textual issue: {text}")
That is quite readable way to represent what kind of objects we expect to process. Compare this to the old way of doing the same thing in print_info_about_issue_the_old_way.
def print_info_about_issue_the_old_way(issue):
if isinstance(issue, dict) and (
issue.get('type') == 'complaint' and
'date' in issue and 'title' in issue and 'issuer' in issue):
d = issue['date']
t = issue['title']
i = issue['issuer']
print(f"{i} complained about {t} on {d}.")
elif isinstance(issue, dict) and (
issue.get('type') == 'request' and
isinstance(issue.get('date'), date) and
'subject' in issue and 'location' in issue and 'issuer' in issue):
y = issue['date'].year
s = issue['subject']
l = issue['location']
i = issue['issuer']
print(f"{i} requested to have {s} on {y} at {l}.")
elif isinstance(issue, str):
print(f"Textual issue: {issue}")Tool support
The implementation of pattern matching uses so-called soft keywords for the “match” and “case” statements because those words are used as identifiers already. Making them hard keywords would break many old code using “match” as an identifier (e.g. re.match). However, this soft keyword implementation makes things harder for tools that process Python source code, since it is no longer possible to a use a simple LL(1) parser to parse the source code, but a more sophisticated parser is needed.
This means that there’s some work to do for maintainers of the IDEs, linters, code formatters etc. Currently it seems that at least PyCharm, Jedi, Flake8 and Black seem to struggle with the new syntax.
Here’s some links to follow their progress on this issue:
- PyCharm: https://youtrack.jetbrains.com/issue/PY-47446
- Jedi: https://github.com/davidhalter/jedi/issues/1756
- Flake8 (pyflakes): https://github.com/PyCQA/pyflakes/pull/630
- Black: https://github.com/psf/black/issues/2242
More pattern matching
If you want to learn more about the structural pattern matching feature, check out the tutorial from PEP-636. And to get even deeper, there’s also the specification in PEP-634 and the motivation and rationale in PEP-635.
Typing related improvements
Writing Union types as X | Y
Type specifications for Unions are now easier to write, since this new syntax doesn’t need an import and is shorter to type. Compare these two:
from decimal import Decimaldef format_euros(value: float | Decimal | None) -> str | None:
if value is None:
return None
return '{:.2f} €'.format(value)from decimal import Decimal
from typing import Optional, Uniondef format_euros_old(value: Optional[Union[float, Decimal]]) -> Optional[str]:
if value is None:
return None
return '{:.2f} €'.format(value)
Clearly the readability is better in the new version compared to the old one. And as you can see, also the Optional is no longer needed, since Optional[X] can be written as X | None. This might become the preferred spelling for writing type specifiers of values that can be None.
It’s also possible to use the new syntax for isinstance and issubclass:
assert isinstance(3.5, float | None)
assert issubclass(bool, int | str) # since bool is subclass of int
assert not isinstance("3.10", float | int | bytes)Exact details are specified in PEP-604.
Parameter Specification Variables
This new feature is useful for creating type safe decorators or other code that modifies the signature of an existing function. The new tool for this is called ParamSpec. Check this example to see it in action:
from typing import Callable, ParamSpec, TypeVarP = ParamSpec("P")
R = TypeVar("R")measured_times: dict[str, float] = {}def time_measured(f: Callable[P, R]) -> Callable[P, R]:
def inner(*args: P.args, **kwargs: P.kwargs) -> R:
t0 = time.perf_counter()
result = f(*args, **kwargs)
t1 = time.perf_counter()
measured_times[f.__name__] = t1 - t0
return result
return inner@time_measured
def repeated_string(a: str, n: int) -> str:
return a * nrepeated_string("ABC", 123) # This is OK
repeated_string(123, "ABC") # This should be rejected by type checker
However, this feature is not yet very well supported by the type checkers, so it might take a while before it can be utilized. See https://github.com/python/typeshed/issues/4827 for updated status information.
Full specification is in PEP-612.
Explicit Type Aliases
Type aliases are very useful to shorten the type signatures of functions or variables when some commonly used type is very complex or otherwise too long to be used nicely. Sometimes there was just a little problem with them though: If you needed to use a forward reference in the type alias, because some part of the type is not yet defined, then the alias would have to be defined as a string, but such string cannot be distinguished from any other global string variable and so type checker might not allow using it as a type.
For example, check this small snippet:
TreeMap = "dict[str, Tree]"class Tree:
def __init__(self, subtrees: TreeMap) -> None:
self.subtrees = subtrees
The snippet makes Mypy to report the following error:
Variable “typealiases.TreeMap” is not valid as a type
Pyright reports the following two errors from it:
Illegal type annotation: variable not allowed unless it is a type alias (reportGeneralTypeIssues)Expected class type but received Literal[‘dict[str, Tree]’] (reportGeneralTypeIssues)
The issue can be fixed with TypeAlias:
from typing_extensions import TypeAliasTreeMap: TypeAlias = "dict[str, Tree]"class Tree:
def __init__(self, subtrees: TreeMap) -> None:
self.subtrees = subtrees
That won’t yield any errors with Pyright or Pyre, but Mypy doesn’t have PEP-613 support yet so it still complains. Also, the import still has to be done from typing_extensions even though TypeAlias was added to the typing module in Python 3.10. This is probably just a small update to those type checkers that already support the explicit type aliases.
Full specification is in PEP-613. The status of its implementation to the type checkers can be followed from https://github.com/python/typeshed/issues/4913.
Postponed evaluation of annotations
Making postponed evaluation of annotations, the default (PEP 563) was postponed to Python 3.11. So you’ll still need to use the from __future__ import annotations in Python 3.10 for that functionality, but be mindful that there are some other planned changes to this whole annotation evaluation subject in PEP-649 and it might even be that from __future__ import annotations will be deprecated rather than becoming the default. See the the Steering Council’s message for details.
Should I already update?
It’s still an early beta release, so of course, it’s not recommended to use it in production. However, it’s wise to add 3.10 to the CI matrices of your projects already to be able to see pending problems and possibly resolve them before the final release in October. Especially if you’re a maintainer of a popular library, make sure that your library supports 3.10 and states so in its metadata so that it will appear green on pyreadiness.
