Small programs built from existing commands. Open a recipe in Pipe, change its input or options, then choose Run.
Each recipe includes sample input, command options, a curl request, expected output, and the missing operations it reveals. Opening this page or loading a recipe makes no outgoing network calls and creates no datasets. Network recipes make the listed calls only when run.
Clean and deduplicate a contact CSV
Trim names, normalize email spelling, discard blank email addresses, and retain one row per email.
parse → replace → replace → jq → format
Sample input · no network requests · no saved output
What this exposes: Trim, case conversion, distinct-by-field, and sort could become short named steps. Email normalization here is a chosen policy, not address validation.
Count warnings and errors by service
Keep warning/error lines, replace one known token pattern, and produce a Markdown summary.
grep → replace → parse → jq → format
Sample input · no network requests · no saved output
What this exposes: Group and aggregate would shorten the jq step. Pattern replacement only removes the pattern shown; it is not a general secret scrubber.
Total paid orders by customer
Parse CSV quantities and integer unit prices, ignore cancelled orders, and total each customer in cents.
parse → grep → jq → format
Sample input · no network requests · no saved output
What this exposes: Typed parsing, grouping, and reduction are recurring needs. Integer cents avoid binary-fraction rounding in this small example; production amounts still need range and currency rules.
Compare two inventory snapshots
Full-join records by SKU, calculate quantity changes, and include additions and removals.
join → jq → format
Sample input · no network requests · no saved output
What this exposes: A keyed record-diff helper could expose added/removed/changed directly. Duplicate SKUs create join combinations, so validate uniqueness first. CSV prefixes negative-looking cells for spreadsheet safety; the JSON result before Format retains numeric deltas.
Find added and removed members
Compare two lists as sets and turn the differences into a short change report.
comm → jq → format
Sample input · no network requests · no saved output
| name | change |
| --- | --- |
| Ada | removed |
| Dev | added |
What this exposes: Comm already handles set membership. It does not preserve duplicate counts or compare changed fields on matching IDs; those are different operations.
Prepare review tasks from a CSV
Keep high-priority documents and build newline-delimited JSON task payloads for another program to submit.
parse → grep → jq → format
Sample input · no network requests · no saved output
What this exposes: There is no per-record Queue write in Pipe. A final queue.push enqueues the entire preceding result as one item. A future explicit each-write operation needs per-item keys, budgets, and partial-failure rules.
What this exposes: A URL resolve/parse step would turn relative references into absolute URLs. HTML selection currently accepts a tag, class, or ID, not a complete CSS selector language.
Make a stable DNS snapshot
Look up public A records and compare sorted addresses rather than changing TTL values or answer order.
dig → jq → format
One public DNS lookup. Live addresses vary; the displayed output uses a test fixture.
What this exposes: Append Diff plus conditional Notify to announce changes. This is a fixed resolver view, not authoritative DNS propagation coverage. DNS failures are errors here, not empty snapshots.
Check a small list of public URLs
Use Xargs to issue a HEAD request for each URL and report status, duration, and errors.
xargs → jq → format
Two public HTTP HEAD requests. Live status and timing vary; the displayed output uses test fixtures.
What this exposes: Xargs supplies each URL string to Fetch when url is omitted. It runs sequentially, at most 20 records. General typed binding of earlier outputs to later options is still missing.
Calculate a total, count, and mean
Reuse the same number list for several calculations: the shape behind a J fork.
jq → format
Sample input · no network requests · no saved output
What this exposes: J can express a mean with the fork (+/ % #). Today jq carries this composition. A future Fork could give the same input to named total/count branches and combine their outputs.
Calculate each project’s share of a total
Keep the original rows while deriving a shared total, then add a percentage to each row.
jq → format
Sample input · no network requests · no saved output
What this exposes: This has a hook-like shape: combine original rows with a value derived from them. A Fork with one identity branch could cover it without a separate Hook command. Zero total produces zero percentages by explicit policy.
Calculate a running balance
Carry a total through an ordered list of changes and retain each intermediate balance.
jq → format
Sample input · no network requests · no saved output
What this exposes: Scan is a strong candidate: reduce returns the final accumulator; scan exposes intermediate accumulators. This works within one request; Window accumulates across separate requests.
Choose a parser from the input format
Select a parser branch, then format its result using a shared following step.
choose → format
Sample input · no network requests · no saved output
What this exposes: Choose changes individual records. Xargs preserves their order and accepts at most 20 records. The empty fallback keeps unmatched events unchanged.
Split and summarize three event streams
Read NDJSON, group records by event kind, and process each populated group once.
parse → route
Sample input · no network requests · no saved output
What this exposes: Route processes whole groups, so aggregation and batching are possible. Output is named branch results. Separate live streams and writes to several destinations are not implemented; one final Pipe action can save this combined result.
What J contributes
J composes functions using rules for hooks, forks, and longer trains. The rules describe how inputs reach functions; they are not simply a list of operations executed in order. Bay Farm can borrow these data-flow patterns while keeping explicit JSON and ordinary command names.
Composition
One-input form
Bay Farm interpretation
Pipe
h(g(f(x)))
Each step receives only the preceding output. Implemented.
Choose
Select one branch for the current value
Implemented. Inspired by J’s gerunds and Agenda selection.
Route
Partition records, then process each group
Implemented for finite batches. Each record enters one named group.
J fork
(f g h) y = (f y) g (h y)
Give one input to two branches, then combine their results. Proposed.
J hook
(f g) y = y f (g y)
Combine the original input with a derived value. Proposed as a fork with an identity branch.
J train
A composition interpreted through hook/fork rules
A broader composition concept. There is no need for a third execution service.
For example, J’s mean (+/ % #) divides a sum by a count from the same list. The shared-summary recipe does that today with jq. The normalize-shares recipe keeps rows while deriving their total. J’s hook and fork definitions.
A proposed value fork. Branches could run sequentially; this does not imply process creation or parallel requests.A hook-like calculation can use a fork with an identity branch.
Conditional branches and grouped streams
J gerunds and Agenda offer a model for selecting functions. Bay Farm uses named cases: Choose selects one branch for the current value; Route selects a group for each record and passes each populated group to its branch once. Fork would instead give the same input to every branch.
Route creates disjoint groups. Each downstream branch receives an array, enabling batching and aggregation.
See parser selection, conditional record transformations and three routed event groups. An empty branch passes its input through. Unmatched selectors use an explicit otherwise branch or fail. An empty Route group returns [] without invoking its commands. Selected branch errors stop the run; they do not invoke the fallback.
Choose can operate on any JSON value; Route takes up to 1000 records. All branches share the run’s operation, network, byte and time limits. Dataset writes stay outside branches as an explicit final Pipe action. Route handles a finite batch; it does not create persistent processes, separate live URLs or several independent delivery transactions.
A possible Fork request shape
Design proposal only. The current API does not accept fork, hook, or train commands. Named branches would make intermediate values visible. Empty branches could mean identity. Begin with a small branch count, pure steps, and one shared execution budget; allow no dataset writes inside branches.
This would be a JSON composition operator inspired by J, not an implementation of J syntax, ranks, or its full train grammar. Here a composition hook means combining original and derived values; HTTP webhooks continue to be Notify receivers.
J also provides useful models for reduction, prefix scan, grouping by keys, and sorting by an ordering. The recipes use jq for these jobs today. New names should shorten recurring programs, not require people to learn another symbolic language.
What the recipes show is missing
These are proposed additions or interface improvements, not commands available in the examples above.
Candidate
Why it helps
Current workaround or boundary
Fork with named branches
Reuse one input, inspect intermediate results, and combine analyses.
jq variables and object expressions. A Hook can be represented with an identity branch; defer a separate Train command.
Assert / require
Stop with a clear failure for a bad response or invalid record.
jq error(...). when short-circuits and returns the current value; it does not fail validation or return an empty result.
Scan
Running totals, balances, streaks, and intermediate state within one input list.
A jq reduce that collects accumulators. Window instead keeps state across requests.
Typed option binding
Use a value produced earlier as a later command’s URL, selected field, or threshold.
Options are literal. Xargs has a narrow URL-string → Fetch / name → Dig convention. General binding needs explicit selectors and type checking.
Group and reduce
Express counts, totals, extrema, and grouped reports with shorter steps.
jq already supports these calculations. Add small named operations only where repeated recipes justify them.
Sort / take / distinct
Build rankings and clean lists without repeated jq expressions.
jq sort_by, array slices, and unique_by. Define stable order, missing values, and duplicate selection.
Zip / transpose
Align parallel lists and convert row-oriented data to columns.
jq expressions. Decide how unequal lengths and ragged arrays behave before exposing an API.
URL parse / resolve
Turn extracted relative links into complete addresses.
Resolve in the caller. All subsequent Fetch calls must retain the current public-network checks.
Explicit per-item writes
Create one queue task or record for each result.
Xargs permits transformations and bounded reads only; one final queue.push writes one payload. A write iterator needs per-item idempotency, quotas, and partial-failure reporting.
Choose and Route now cover conditional processing. Next consider Fork and Assert, then Scan and typed binding. Keep Hook as a derived pattern until examples show that a separate command makes programs clearer. Use one shared execution model for immediate pipes, scheduled pipes, and event pipes.
Use saved data and produce events
Replace a recipe’s top-level input with source to read a Store record or UTF-8 Artifact. Public reads can omit access_key. A dataset page is only one page: follow its cursor before treating a collection comparison as complete.
To save a report, append a final Store action. Revision 0 creates a record; supply the current revision for a checked update. Earlier transformation failures prevent this final write. These examples require your own dataset IDs and keys.
To announce changes to a stable DNS snapshot, finish with Diff and a Notify destination. Create both datasets first. Diff stores the new snapshot and conditionally publishes in the same transaction. Its first value establishes a baseline without an alert. Future differing values emit a message; repeated values stay quiet.
Free saved data is public. Private input cannot be written into public datasets. Receiver delivery happens after publication; retries may repeat a delivery event, so consumers should deduplicate its ID.
Use Schedule’s pipe mode to run a recipe on a calendar, or Events to run it when a message or queued task arrives. They use the same command steps. Dataset access · Webhook delivery · Schedule · Events.
Execution rules that matter
Each step gets the preceding output. Its options are literal JSON; there is no general interpolation of earlier values into later options.
A pipe has 1–8 steps, bounded input/output, and one optional final dataset action. Xargs is sequential, at most 20 inputs and four pure steps per input.
Transformations do not retain input or output. The per-record stream routes buffer the full bounded upload before processing; they are not continuous input processors.
jq emits one value directly, multiple values as an array, and no values as an empty array. Use array-producing expressions when you need a stable array shape.
CSV parsing keeps fields as strings. Convert and validate numbers explicitly. CSV output prefixes formula-looking cells, including negative-looking values, for spreadsheet safety; choose JSON when exact typed values matter.
A false when condition returns the current input and skips later steps. The response saved flag reflects an executed final dataset action, not merely one present in the definition.
Public network calls can fail, return non-2xx responses, or be truncated. Inspect status/error/truncated before interpreting a body. Fixed live recipes show fixture output rather than claiming stable external results.
A consistent record envelope, explicit cardinality, typed bindings, and complete-page handling will improve composition more than adding many new names.
