Skip to main content

Sequence Patterns

seq pipelines run over non-string, non-mapping Python iterables. A sequence can contain plain values, row dictionaries, objects, functions, or other Python values. Polars tables annotated as seq are converted to row dictionaries. The main rule is: use the shorthand that matches what each item is.

Plain Values​

For numbers, strings, and other scalar values, use placeholders or lambdas:

values = seq [1, 2, 3, 4]

result = values
|> filter(_ > 1)
|> map(_ * 10)

_ means the current item. Use _1, _2, and so on for callbacks that receive more than one value:

total = values |> reduce(_1 + _2)

Row Dictionaries​

For JSON-like records, filter(...) and set_fields(...) can read dictionary fields with bare names:

orders = seq [
{"item": "coffee", "qty": 3},
{"item": "pens", "qty": 2},
{"item": "paper"},
]

restock = orders
|> filter(qty > 1)
|> keep_fields(item)
|> set_fields(buy = item == "pens")

Rows without qty are skipped by the filter. Missing fields are boolean-false; item == "pens" is False when item is absent, while item != "pens" is True for rows without item.

Arithmetic still requires present values. This errors if a row is missing qty or price:

orders |> set_fields(total = qty * price)

That distinction keeps filtering forgiving while still surfacing missing data in calculations.

Sparse JSON Rows​

When JSON records do not all have the same keys, use comparisons to make the missing-field behavior explicit:

events = seq [
{"id": 1, "status": "open"},
{"id": 2},
{"id": 3, "status": "closed"},
]

open_events = events |> filter(status == "open")
not_closed_events = events |> filter(status != "closed")
with_flags = events |> set_fields(is_open = status == "open")

open_events contains only rows whose status exists and equals "open". not_closed_events also keeps rows without status, because missing fields do not equal "closed". with_flags sets is_open to False on rows where status is absent.

The same rule applies when a field name is misspelled or the upstream JSON schema changes:

typoed = events |> filter(stauts == "open")
both_missing = events |> filter(nonsense1 == nonsense2)

stauts is treated as a missing field on every row. The pipeline still runs, but the result is wrong for a required-field workflow. If both sides are misspelled or renamed, nonsense1 == nonsense2 still matches no rows, while nonsense1 != nonsense2 matches rows where both fields are absent. When a field must exist, validate the input shape first or use explicit row access in a Python helper that raises on missing keys.

Python Objects​

The same bare-name shorthand works with object attributes in filter(...):

priority = seq orders
|> filter(qty > 1)

If orders contains Python objects with a .qty attribute, PyPlyne reads that attribute. Objects without the attribute do not match comparison filters. Record field verbs such as set_fields(...), drop_fields(...), and keep_fields(...) require row dictionaries, not arbitrary objects.

Use explicit lambdas when the object API is richer than simple attributes:

priority = seq orders
|> filter(order => order.is_priority())
|> map(order => order.to_dict())

Functions And Callables​

Functions are ordinary Python objects, so you can filter by attributes:

chosen = seq checks
|> filter(__name__ == "is_priority")

When you want to call the current item, use _:

passing = seq checks
|> filter(_("sample input"))

If you have a named predicate function and want to use it directly, pass the bare function name:

orders |> filter(is_priority)

PyPlyne preserves a single bare name in filter(...) as a predicate function. Expressions such as qty > 1 are treated as row-field or object-attribute predicates.

When To Use Which Form​

  • Use _ for the current item: filter(_ > 10), map(_["item"]), filter(_(sample)).
  • Use bare fields or attributes in record/object filters: filter(qty > 1).
  • Use bare fields in dictionary record updates: set_fields(net = amount - discount).
  • Use keep_fields(...) and drop_fields(...) for dictionary-shaped rows.
  • Use explicit lambdas when you need method calls, custom names, or richer object behavior.

A single bare name in filter(...) is treated as a predicate function, not as a field truthiness check:

orders |> filter(is_priority)
orders |> filter(active == True)
orders |> filter(_["active"])

map(...) does not infer bare fields. Use a placeholder or lambda there:

items = orders |> map(_["item"])
items = orders |> map(order => order["item"])

Crossing To Tables​

Use to_table() when your sequence is a list of row dictionaries and you want Polars-backed table verbs:

summary = seq orders
|> filter(qty > 1)
|> set_fields(buy = item == "pens")
|> to_table()
|> group_by(buy)
|> summarize(rows = count())

Use to_rows() to move from a table into record-by-record sequence work.