PEP 572 – Assignment Expressions

This is a proposal for creating a way to assign to variables within an expression using the notation NAME := expr .

As part of this change, there is also an update to dictionary comprehension evaluation order to ensure key expressions are executed before value expressions (allowing the key to be bound to a name and then re-used as part of calculating the corresponding value).

During discussion of this PEP, the operator became informally known as “the walrus operator”. The construct’s formal name is “Assignment Expressions” (as per the PEP title), but they may also be referred to as “Named Expressions” (e.g. the CPython reference implementation uses that name internally).

Rationale

Naming the result of an expression is an important part of programming, allowing a descriptive name to be used in place of a longer expression, and permitting reuse. Currently, this feature is available only in statement form, making it unavailable in list comprehensions and other expression contexts.

Additionally, naming sub-parts of a large expression can assist an interactive debugger, providing useful display hooks and partial results. Without a way to capture sub-expressions inline, this would require refactoring of the original code; with assignment expressions, this merely requires the insertion of a few name := markers. Removing the need to refactor reduces the likelihood that the code be inadvertently changed as part of debugging (a common cause of Heisenbugs), and is easier to dictate to another programmer.

The importance of real code

During the development of this PEP many people (supporters and critics both) have had a tendency to focus on toy examples on the one hand, and on overly complex examples on the other.

The danger of toy examples is twofold: they are often too abstract to make anyone go “ooh, that’s compelling”, and they are easily refuted with “I would never write it that way anyway”.

The danger of overly complex examples is that they provide a convenient strawman for critics of the proposal to shoot down (“that’s obfuscated”).

Yet there is some use for both extremely simple and extremely complex examples: they are helpful to clarify the intended semantics. Therefore, there will be some of each below.

However, in order to be compelling, examples should be rooted in real code, i.e. code that was written without any thought of this PEP, as part of a useful application, however large or small. Tim Peters has been extremely helpful by going over his own personal code repository and picking examples of code he had written that (in his view) would have been clearer if rewritten with (sparing) use of assignment expressions. His conclusion: the current proposal would have allowed a modest but clear improvement in quite a few bits of code.

Another use of real code is to observe indirectly how much value programmers place on compactness. Guido van Rossum searched through a Dropbox code base and discovered some evidence that programmers value writing fewer lines over shorter lines.

Case in point: Guido found several examples where a programmer repeated a subexpression, slowing down the program, in order to save one line of code, e.g. instead of writing:

match = re.match(data) group = match.group(1) if match else None 

they would write:

group = re.match(data).group(1) if re.match(data) else None 

Another example illustrates that programmers sometimes do more work to save an extra level of indentation:

match1 = pattern1.match(data) match2 = pattern2.match(data) if match1: result = match1.group(1) elif match2: result = match2.group(2) else: result = None 

This code tries to match pattern2 even if pattern1 has a match (in which case the match on pattern2 is never used). The more efficient rewrite would have been:

match1 = pattern1.match(data) if match1: result = match1.group(1) else: match2 = pattern2.match(data) if match2: result = match2.group(2) else: result = None 

Syntax and semantics

In most contexts where arbitrary Python expressions can be used, a named expression can appear. This is of the form NAME := expr where expr is any valid Python expression other than an unparenthesized tuple, and NAME is an identifier.

The value of such a named expression is the same as the incorporated expression, with the additional side-effect that the target is assigned that value:

# Handle a matched regex if (match := pattern.search(data)) is not None: # Do something with match # A loop that can't be trivially rewritten using 2-arg iter() while chunk := file.read(8192): process(chunk) # Reuse a value that's expensive to compute [y := f(x), y**2, y**3] # Share a subexpression between a comprehension filter clause and its output filtered_data = [y for x in data if (y := f(x)) is not None] 

Exceptional cases

There are a few places where assignment expressions are not allowed, in order to avoid ambiguities or user confusion:

y := f(x) # INVALID (y := f(x)) # Valid, though not recommended 
y0 = y1 := f(x) # INVALID y0 = (y1 := f(x)) # Valid, though discouraged 
foo(x = y := f(x)) # INVALID foo(x=(y := f(x))) # Valid, though probably confusing 
def foo(answer = p := 42): # INVALID . def foo(answer=(p := 42)): # Valid, though not great style . 
def foo(answer: p := 42 = 5): # INVALID . def foo(answer: (p := 42) = 5): # Valid, but probably never useful . 
(lambda: x := 1) # INVALID lambda: (x := 1) # Valid, but unlikely to be useful (x := lambda: 1) # Valid lambda line: (m := re.match(pattern, line)) and m.group(1) # Valid 
>>> f'(x:=10)>' # Valid, uses assignment expression '10' >>> x = 10 >>> f'x:=10>' # Valid, passes '=10' to formatter ' 10' 

Scope of the target

An assignment expression does not introduce a new scope. In most cases the scope in which the target will be bound is self-explanatory: it is the current scope. If this scope contains a nonlocal or global declaration for the target, the assignment expression honors that. A lambda (being an explicit, if anonymous, function definition) counts as a scope for this purpose.

There is one special case: an assignment expression occurring in a list, set or dict comprehension or in a generator expression (below collectively referred to as “comprehensions”) binds the target in the containing scope, honoring a nonlocal or global declaration for the target in that scope, if one exists. For the purpose of this rule the containing scope of a nested comprehension is the scope that contains the outermost comprehension. A lambda counts as a containing scope.

The motivation for this special case is twofold. First, it allows us to conveniently capture a “witness” for an any() expression, or a counterexample for all() , for example:

if any((comment := line).startswith('#') for line in lines): print("First comment:", comment) else: print("There are no comments") if all((nonblank := line).strip() == '' for line in lines): print("All lines are blank") else: print("First non-blank line:", nonblank) 

Second, it allows a compact way of updating mutable state from a comprehension, for example:

# Compute partial sums in a list comprehension total = 0 partial_sums = [total := total + v for v in values] print("Total:", total) 

However, an assignment expression target name cannot be the same as a for -target name appearing in any comprehension containing the assignment expression. The latter names are local to the comprehension in which they appear, so it would be contradictory for a contained use of the same name to refer to the scope containing the outermost comprehension instead.

For example, [i := i+1 for i in range(5)] is invalid: the for i part establishes that i is local to the comprehension, but the i := part insists that i is not local to the comprehension. The same reason makes these examples invalid too:

[[(j := j) for i in range(5)] for j in range(5)] # INVALID [i := 0 for i, j in stuff] # INVALID [i+1 for i in (i := stuff)] # INVALID 

While it’s technically possible to assign consistent semantics to these cases, it’s difficult to determine whether those semantics actually make sense in the absence of real use cases. Accordingly, the reference implementation [1] will ensure that such cases raise SyntaxError , rather than executing with implementation defined behaviour.

This restriction applies even if the assignment expression is never executed:

[False and (i := 0) for i, j in stuff] # INVALID [i for i, j in stuff if True or (j := 1)] # INVALID 

For the comprehension body (the part before the first “for” keyword) and the filter expression (the part after “if” and before any nested “for”), this restriction applies solely to target names that are also used as iteration variables in the comprehension. Lambda expressions appearing in these positions introduce a new explicit function scope, and hence may use assignment expressions with no additional restrictions.

Due to design constraints in the reference implementation (the symbol table analyser cannot easily detect when names are re-used between the leftmost comprehension iterable expression and the rest of the comprehension), named expressions are disallowed entirely as part of comprehension iterable expressions (the part after each “in”, and before any subsequent “if” or “for” keyword):

[i+1 for i in (j := stuff)] # INVALID [i+1 for i in range(2) for j in (k := stuff)] # INVALID [i+1 for i in [j for j in (k := stuff)]] # INVALID [i+1 for i in (lambda: (j := stuff))()] # INVALID 

A further exception applies when an assignment expression occurs in a comprehension whose containing scope is a class scope. If the rules above were to result in the target being assigned in that class’s scope, the assignment expression is expressly invalid. This case also raises SyntaxError :

class Example: [(j := i) for i in range(5)] # INVALID 

(The reason for the latter exception is the implicit function scope created for comprehensions – there is currently no runtime mechanism for a function to refer to a variable in the containing class scope, and we do not want to add such a mechanism. If this issue ever gets resolved this special case may be removed from the specification of assignment expressions. Note that the problem already exists for using a variable defined in the class scope from a comprehension.)

See Appendix B for some examples of how the rules for targets in comprehensions translate to equivalent code.

Relative precedence of :=

The := operator groups more tightly than a comma in all syntactic positions where it is legal, but less tightly than all other operators, including or , and , not , and conditional expressions ( A if C else B ). As follows from section “Exceptional cases” above, it is never allowed at the same level as = . In case a different grouping is desired, parentheses should be used.

The := operator may be used directly in a positional function call argument; however it is invalid directly in a keyword argument.

Some examples to clarify what’s technically valid or invalid:

# INVALID x := 0 # Valid alternative (x := 0) # INVALID x = y := 0 # Valid alternative x = (y := 0) # Valid len(lines := f.readlines()) # Valid foo(x := 3, cat='vector') # INVALID foo(cat=category := 'vector') # Valid alternative foo(cat=(category := 'vector')) 

Most of the “valid” examples above are not recommended, since human readers of Python source code who are quickly glancing at some code may miss the distinction. But simple cases are not objectionable:

# Valid if any(len(longline := line) >= 100 for line in lines): print("Extremely long line:", longline) 

This PEP recommends always putting spaces around := , similar to PEP 8’s recommendation for = when used for assignment, whereas the latter disallows spaces around = used for keyword arguments.)

Change to evaluation order

In order to have precisely defined semantics, the proposal requires evaluation order to be well-defined. This is technically not a new requirement, as function calls may already have side effects. Python already has a rule that subexpressions are generally evaluated from left to right. However, assignment expressions make these side effects more visible, and we propose a single change to the current evaluation order:

Differences between assignment expressions and assignment statements

Most importantly, since := is an expression, it can be used in contexts where statements are illegal, including lambda functions and comprehensions.

Conversely, assignment expressions don’t support the advanced features found in assignment statements:

x = y = z = 0 # Equivalent: (z := (y := (x := 0))) 
# No equivalent a[i] = x self.rest = [] 
x = 1, 2 # Sets x to (1, 2) (x := 1, 2) # Sets x to 1 
# Equivalent needs extra parentheses loc = x, y # Use (loc := (x, y)) info = name, phone, *rest # Use (info := (name, phone, *rest)) # No equivalent px, py, pz = position name, phone, email, *other_info = contact 
# Closest equivalent is "p: Optional[int]" as a separate declaration p: Optional[int] = None 
total += tax # Equivalent: (total := total + tax) 

Specification changes during implementation

The following changes have been made based on implementation experience and additional review after the PEP was first accepted and before Python 3.8 was released:

Examples

Examples from the Python standard library

site.py

env_base is only used on these lines, putting its assignment on the if moves it as the “header” of the block.

env_base = os.environ.get("PYTHONUSERBASE", None) if env_base: return env_base 
if env_base := os.environ.get("PYTHONUSERBASE", None): return env_base 

_pydecimal.py

Avoid nested if and remove one indentation level.

if self._is_special: ans = self._check_nans(context=context) if ans: return ans 
if self._is_special and (ans := self._check_nans(context=context)): return ans 

copy.py

Code looks more regular and avoid multiple nested if. (See Appendix A for the origin of this example.)

reductor = dispatch_table.get(cls) if reductor: rv = reductor(x) else: reductor = getattr(x, "__reduce_ex__", None) if reductor: rv = reductor(4) else: reductor = getattr(x, "__reduce__", None) if reductor: rv = reductor() else: raise Error( "un(deep)copyable object of type %s" % cls) 
if reductor := dispatch_table.get(cls): rv = reductor(x) elif reductor := getattr(x, "__reduce_ex__", None): rv = reductor(4) elif reductor := getattr(x, "__reduce__", None): rv = reductor() else: raise Error("un(deep)copyable object of type %s" % cls) 

datetime.py

tz is only used for s += tz , moving its assignment inside the if helps to show its scope.

s = _format_time(self._hour, self._minute, self._second, self._microsecond, timespec) tz = self._tzstr() if tz: s += tz return s 
s = _format_time(self._hour, self._minute, self._second, self._microsecond, timespec) if tz := self._tzstr(): s += tz return s 

sysconfig.py

Calling fp.readline() in the while condition and calling .match() on the if lines make the code more compact without making it harder to understand.

while True: line = fp.readline() if not line: break m = define_rx.match(line) if m: n, v = m.group(1, 2) try: v = int(v) except ValueError: pass vars[n] = v else: m = undef_rx.match(line) if m: vars[m.group(1)] = 0 
while line := fp.readline(): if m := define_rx.match(line): n, v = m.group(1, 2) try: v = int(v) except ValueError: pass vars[n] = v elif m := undef_rx.match(line): vars[m.group(1)] = 0 

Simplifying list comprehensions

A list comprehension can map and filter efficiently by capturing the condition:

results = [(x, y, x/y) for x in input_data if (y := f(x)) > 0] 

Similarly, a subexpression can be reused within the main expression, by giving it a name on first use:

stuff = [[y := f(x), x/y] for x in range(5)] 

Note that in both cases the variable y is bound in the containing scope (i.e. at the same level as results or stuff ).

Capturing condition values

Assignment expressions can be used to good effect in the header of an if or while statement:

# Loop-and-a-half while (command := input("> ")) != "quit": print("You entered:", command) # Capturing regular expression match objects # See, for instance, Lib/pydoc.py, which uses a multiline spelling # of this effect if match := re.search(pat, text): print("Found:", match.group(0)) # The same syntax chains nicely into 'elif' statements, unlike the # equivalent using assignment statements. elif match := re.search(otherpat, text): print("Alternate found:", match.group(0)) elif match := re.search(third, text): print("Fallback found:", match.group(0)) # Reading socket data until an empty string is returned while data := sock.recv(8192): print("Received data:", data) 

Particularly with the while loop, this can remove the need to have an infinite loop, an assignment, and a condition. It also creates a smooth parallel between a loop which simply uses a function call as its condition, and one which uses that as its condition but also uses the actual value.

Fork

An example from the low-level UNIX world:

if pid := os.fork(): # Parent code else: # Child code 

Rejected alternative proposals

Proposals broadly similar to this one have come up frequently on python-ideas. Below are a number of alternative syntaxes, some of them specific to comprehensions, which have been rejected in favour of the one given above.

Changing the scope rules for comprehensions

A previous version of this PEP proposed subtle changes to the scope rules for comprehensions, to make them more usable in class scope and to unify the scope of the “outermost iterable” and the rest of the comprehension. However, this part of the proposal would have caused backwards incompatibilities, and has been withdrawn so the PEP can focus on assignment expressions.

Alternative spellings

Broadly the same semantics as the current proposal, but spelled differently.

stuff = [[f(x) as y, x/y] for x in range(5)]