What am I About to Run?מה אני עומד להריץ?

Overview

The layer beneath the pipeline

Every time an autonomous-vehicle team changes its driving software, it needs to know what that change broke. Driving real cars to find out is too slow and too risky, so the answer comes from simulation: thousands of test scenarios, launched automatically whenever a new build appears. That automation has two halves: the pipeline that decides when tests run, and the layer that defines what runs and how. At Foretellix the pipeline belonged to the customer. I designed the other half.

One challenge ran through four generations of the product: a launch is built from a predefined template, often one someone else created, yet every run needs something slightly different.

---

The problem

A template that every run needs to bend

Before the GUI, a test run was a spreadsheet, with every parameter typed by hand.

The first GUI, in 2023, moved it into one modal: pick a runner file, paste the environment settings as raw JSON, run.

"It is too hard for user to setup a test using raw json file... New engineers have a hard time learning how to use the tool."
A customer's V&V organisation

Underneath the JSON sat a harder problem: no launch is quite the same twice. A new simulator image, another map, one changed parameter: without per-run overrides, changing one image tag meant cloning a whole settings record.

---

How I worked

Working from every signal available

Access to customers was limited, but I could still build the picture from every source available, and ran my own validation where none existed:

• 2023: Mapped the terminal workflow and the launch flows in FigJam.
• 2023-2024: Worked from field material: launch cases, an annotated customer configuration, site-visit notes.
• 2024: Ran my own prototype review: about twenty reviewers, roughly ninety logged items.
• 2025-2026: Reviewed the new flows with an application engineer, and benchmarked eight test and CI tools into design rules.
• Throughout: Kept a master Figma file as the team's single source of truth.

The 2024 concept was paused and never built. But one reviewer's request, to decide what a launch may override, returned two years later as the override model.

Two rounds of the composition concept: a suite as a tree, then a library where a suite overrides a scenario's parameters.

Four generations followed, from a raw-JSON modal (2023) to the curated launch drawer (2026). Three design decisions carried across them.

---

Design decision 1

What to run, and how to run it

In the 2023 modal, the tests and their environment were one block of configuration: changing either meant editing both.

What to run is a scenario: actors, map and parameters. How to run it is a preset: setup files, simulator image, timeouts and variables.

Four objects carry the product:

• Tests: concrete or abstract scenarios.
• Test Suites: named groups of tests. What to run.
• Environment Settings: presets for the simulator, image and variables. How to run.
• Flow Definition (earlier, the Test Plan): a flow type, its suites and a preset in one launchable template. Set up once by an administrator, launched by V&V engineers and developers.

---

Design decision 2

One pattern to build, one drawer to launch

Executable assets got their own top-level tab, Test Management: a library of everything users can run, built on one repeating pattern:

• Entity manager: find, filter and share it.
• Template page: create and edit each master.
• Reusable drawer: launch the test.

I first proposed a creation wizard inside the launch, then dropped it: creating objects mid-launch would leave orphaned plans and suites.

---

Design decision 3

Show what matters, keep everything within reach

By 2026 a preset held far more than anyone needed: application engineers estimated that roughly 95% of environment settings were irrelevant to the person launching a test.

The same drawer twice: the curated default, and every setting one click away.

The template stays stable, and each execution changes only what it needs. Platform engineering calls this a golden path; I reached it from the field.

---

Process change

Changing how design reaches engineering

In the last year my workflow changed too. I designed directly in a branch of the product: Figma for early exploration only, then the UX built in live code with mocked data, down to the shared components in Storybook.

That left one gap: development and QA still needed a source of truth, and redrawing finished code in Figma would double the work.

Its first reader turned out to be me. Writing every state down as a test case worked as a second pass on my own design, and it surfaced edge cases I had missed. On the Manage Maps page alone, it exposed three states the design was asked for and never showed, such as a failed import that leaves no trace.

The front-end developer audited one against its commit and found four uncommitted files: the method doing exactly its job.

---

Outcome

A structure the product could grow into

By the time I left, new work plugged into the model without reshaping it:

• One object, a dozen flow types. Each new flow, from ingesting logs to exporting results, rode the Flow Definition without a redesign.
• Overrides the field asked for. The four fields a customer said they change are the four the 2026 whitelist was built around.
• Enablement. Field engineers were trained on it in four sessions across time zones.

What I would do differently

Measure from day one. Nobody tracked usage, and I argued from that absence instead of fixing it. The first data I finally saw pointed the right way: people opened "Show all" again and again. Next time, the instrumentation ships with the design.