workflow_wal.c File Reference

Binary Write-Ahead Log implementation for AI workflows. More...

#include "csilk/app/workflow_wal.h"
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/stat.h>
#include <time.h>
#include <unistd.h>

Include dependency graph for workflow_wal.c:

Functions
int	_wf_wal_append (const char *wal_path, csilk_wf_event_type_t type, const void *payload, size_t len)
	Append an event record to the workflow Write-Ahead Log.

Detailed Description

Binary Write-Ahead Log implementation for AI workflows.

Architecture: Crash-safe durability for workflow execution state. Events (node start, node finish, workflow end) are written as fixed-size headers + variable-length payloads to a binary WAL file. Each record is fsynced before returning to guarantee durability.

The WAL is replayed during csilk_wf_resume() to reconstruct the workflow's execution state after a crash or restart.

Wire format: [magic:4 bytes][type:1 byte][timestamp:4 bytes][payload_len:4 bytes] [payload:payload_len bytes]

Copyright

MIT License

Function Documentation

_wf_wal_append()

int _wf_wal_append	(	const char *	wal_path,
		csilk_wf_event_type_t	type,
		const void *	payload,
		size_t	len
	)

Append an event record to the workflow Write-Ahead Log.

Append a WAL record to the log file.

Algorithm:

1. Open (or create) the WAL file in O_WRONLY | O_APPEND mode.
2. Write a fixed-size csilk_wf_wal_header_t (magic, type, timestamp, payload_len).
3. If payload_len > 0 and payload is non-NULL, write the payload bytes immediately after the header.
4. fdatasync() the file to flush the data to disk before closing.

Parameters

wal_path	Absolute path to the WAL file.
type	Event type (e.g., WF_EV_START, WF_EV_NODE_FINISH).
payload	Opaque payload data to persist (may be NULL if len is 0).
len	Number of payload bytes.

Returns

0 on success, -1 if wal_path is NULL or any write fails.

Note

Not thread-safe for concurrent writes to the same file — callers should serialize access at the workflow context level.