Skip to content

Monitor flow execution

This guide explains how to monitor your pgflow flows during and after execution using SQL queries.

To check the status of your flow runs, query the pgflow.runs table:

SELECT * FROM pgflow.runs
WHERE flow_slug = 'your_flow_slug'
ORDER BY started_at DESC
LIMIT 10;

This shows the most recent runs with their status, inputs, outputs, and remaining steps:

run_id | flow_slug | status | input | output | remaining_steps
--------------+--------------+-----------+----------------------+--------------------------+-----------------
<run_id UUID> | your_flow | completed | {"param": "value"} | {"result": "Success!"} | 0
<run_id UUID> | your_flow | started | {"param": "value2"} | null | 3
<run_id UUID> | your_flow | failed | {"param": "value3"} | null | 5

Run statuses include:

  • started: The run has been created and is executing steps
  • completed: All steps have completed successfully
  • failed: One or more steps have failed after max retries

To check the status of individual steps within a run:

SELECT ss.step_slug, ss.status, ss.remaining_deps, ss.remaining_tasks, st.output
FROM pgflow.step_states ss
LEFT JOIN
pgflow.step_tasks st
ON
ss.run_id = st.run_id
AND ss.step_slug = st.step_slug
AND st.status = 'completed'
WHERE ss.run_id = 'your-run-id-here';

This shows the status of each step with its output:

step_slug | status | remaining_deps | remaining_tasks | output
-------------+-----------+----------------+----------------+---------------------------
process_data | completed | 0 | 0 | {"processed": true}
send_email | completed | 0 | 0 | "Email sent successfully"
final_step | created | 2 | 1 | null

Step statuses include:

  • created: The step has been created but may be waiting for dependencies
  • started: The step has started execution (all dependencies are complete)
  • completed: The step has completed successfully
  • failed: The step has failed after max retries

To see all tasks that are currently active (queued or being processed):

SELECT
run_id,
step_slug,
status,
attempts_count,
message_id,
queued_at,
started_at,
last_worker_id
FROM pgflow.step_tasks
WHERE status IN ('queued', 'started')
ORDER BY queued_at ASC;

This shows active tasks at different stages:

run_id | step_slug | status | attempts_count | message_id | queued_at | started_at | last_worker_id
---------+--------------+---------+---------------+------------+--------------------+--------------------+---------------
<run_id> | process_data | queued | 1 | 123456 | 2023-05-01 12:00:00| null | null
<run_id> | send_email | started | 1 | 123457 | 2023-05-01 12:01:00| 2023-05-01 12:02:00| worker-uuid-123

Active task statuses:

  • queued: Task is ready to run, waiting for a worker to claim it
  • started: Task is currently being processed by a worker (with started_at timestamp and worker_id)

To see how long each step took to execute:

SELECT
step_slug,
status,
created_at,
started_at,
completed_at,
failed_at,
EXTRACT(EPOCH FROM (COALESCE(completed_at, failed_at) - started_at)) AS duration_seconds
FROM pgflow.step_states
WHERE run_id = 'your-run-id-here'
ORDER BY created_at ASC;

When a step fails, you can check for error details:

SELECT
ss.step_slug,
st.attempts_count,
st.error_message,
st.queued_at,
st.failed_at
FROM pgflow.step_states ss
JOIN pgflow.step_tasks st ON
ss.run_id = st.run_id AND
ss.step_slug = st.step_slug
WHERE ss.run_id = 'your-run-id-here'
AND ss.status = 'failed';

To understand the structure of your flow and the dependencies between steps:

SELECT
steps.step_slug,
array_agg(deps.dep_slug) as dependencies
FROM pgflow.steps steps
LEFT JOIN pgflow.deps deps ON
steps.flow_slug = deps.flow_slug AND
steps.step_slug = deps.step_slug
WHERE steps.flow_slug = 'your_flow_slug'
GROUP BY steps.step_slug;