Before you start
Have these ready, or ask your developer to build them:
- A public HTTPS URL on port 443 that accepts POST. It must answer directly: redirects are not followed.
- A place to keep the signing secret (an environment variable such as SEARCHLARK_WEBHOOK_SECRET). You get it once, when you connect.
- Code that checks the X-Searchlark-Signature header on the raw body, and rejects requests that fail.
- Code that creates the post the first time it sees an idempotency_key and updates that same post afterwards.
- An answer within 20 seconds: a 2xx status with JSON like {"id": "123", "url": "https://yoursite.com/blog/the-post"} (an absolute url).
- Ignore article.test (never publish it) and acknowledge event types you don't know with a 2xx.
- A way to save posts: your CMS's API, or a database table with a column for the Searchlark
idempotency_key.
The quickest path: copy the starter receiver for your stack, replace its save step, deploy it, and connect it.
Connect and test
- In Searchlark, open the site, then Integrations, choose Webhook, enter your endpoint's final HTTPS URL and click Save webhook.
- Copy the signing secret (it is shown once) into your server as
SEARCHLARK_WEBHOOK_SECRET, and deploy. - Click Send a ping. Then click Send a sample article: Searchlark shows what your endpoint answered and how to fix anything wrong.
From then on, approved articles go out on their scheduled day (or when you click Publish now), and Recent deliveries on the Integrations page shows each one.
The request
Every request is a POST with these headers:
| Header | Value |
|---|---|
Content-Type | application/json (UTF-8) |
X-Searchlark-Signature | t=<unix seconds>,v1=<hex HMAC-SHA256 of "<t>.<raw body>"> |
Idempotency-Key | <article id>:<connection id>, the same for every delivery and update of one article (not sent with ping) |
X-Searchlark-Delivery | One id per delivery (retries of it reuse it): use it to ignore a repeated request |
X-Searchlark-Test | 1 on Send a sample article: never publish these |
User-Agent | SearchlarkBot/1.0 (+https://searchlark.com/bot) |
Until 1 Apr 2027, requests also carry X-Kestrel-Signature with the same value as X-Searchlark-Signature.
Events
ping: You click Send a ping in Searchlark. Body:{ "event": "ping", "at": "<ISO time>" }. Answer: Any 2xx.article.test: You click Send a sample article. It carries the X-Searchlark-Test: 1 header and data.test = true. Body:{ "event": "article.test", "data": { …same fields as below, "test": true } }. Answer: 2xx with { id, url }, without publishing anything.article.publish: An approved article reaches its scheduled day, or someone clicks Publish now or Update live post. Body:{ "event": "article.publish", "data": { …the fields below } }. Answer: 2xx with { id, url } once the post is saved.
New event types may be added later: answer any event you don't know with a 2xx and ignore it. Searchlark never asks you to delete or unpublish a post.
The article
| Field in data | Type | Meaning |
|---|---|---|
idempotency_key | string | Stable for this article and destination. Create the post the first time you see it; update that post every later time. |
external_id | string | null | Your id from an earlier answer, if you returned one; null on the first publish. |
publication_id | string | Searchlark's id for this article's publication. |
title | string | The article title (use as the H1). |
slug | string | null | Suggested URL slug. May be null or already taken: you decide the final slug. |
excerpt | string | A one or two sentence summary (the meta description). |
html | string | The body as HTML, sanitised to headings h2 to h4, paragraphs, lists, links, images, tables and quotes. Includes the FAQ section. |
markdown | string | The same body as Markdown. Not sanitised: render it with your own Markdown library. |
seo.meta_title | string | The <title> for the page. |
seo.meta_description | string | The meta description. |
seo.focus_keyword | string | The keyword the article targets. |
featured_image | { url, alt } | null | A public image URL. Download and host it yourself if you want it to stay unchanged. |
faq | { question, answer }[] | The FAQ shown at the end of the article (already in html and markdown). |
json_ld | object | null | Structured data (Article, FAQPage) to add in a <script type="application/ld+json"> tag. |
publish_date | string | null | The scheduled date (YYYY-MM-DD). Null for some manual publishes. A retried delivery can arrive later than this date. |
Bodies are UTF-8 JSON and can reach about 2 MB, so allow that size on your server.
{
"event": "article.publish",
"data": {
"idempotency_key": "3b7d5e20-0000-4000-8000-000000000002:9a0e4c11-0000-4000-8000-000000000003",
"publication_id": "6f1c2a9e-0000-4000-8000-000000000001",
"external_id": null,
"title": "How far ahead should you order a wedding cake?",
"slug": "order-wedding-cake-timeline",
"excerpt": "Order your wedding cake 4 to 6 months ahead. Here is the timeline we give couples.",
"html": "<p>Most couples order <strong>4 to 6 months</strong> before the wedding.</p>\n<h2>The timeline</h2>\n<ul>\n<li>6 months: tastings</li>\n<li>3 months: final design</li>\n<li>2 weeks: final numbers</li>\n</ul>\n<h2>Frequently asked questions</h2>\n<h3>Can I order a cake 1 month ahead?</h3>\n<p>Sometimes, if the bakery has space.</p>\n",
"markdown": "Most couples order **4 to 6 months** before the wedding.\n\n## The timeline\n\n- 6 months: tastings\n- 3 months: final design\n- 2 weeks: final numbers\n\n## Frequently asked questions\n\n### Can I order a cake 1 month ahead?\n\nSometimes, if the bakery has space.\n",
"seo": {
"meta_title": "How far ahead to order a wedding cake (timeline)",
"meta_description": "Order your wedding cake 4 to 6 months ahead. Here is the timeline we give couples.",
"focus_keyword": "wedding cake order timeline"
},
"featured_image": null,
"faq": [
{
"question": "Can I order a cake 1 month ahead?",
"answer": "Sometimes, if the bakery has space."
}
],
"json_ld": {
"@context": "https://schema.org",
"@graph": [
{
"@type": "FAQPage",
"mainEntity": [
{
"@type": "Question",
"name": "Can I order a cake 1 month ahead?",
"acceptedAnswer": {
"@type": "Answer",
"text": "Sometimes, if the bakery has space."
}
}
]
}
]
},
"publish_date": "2026-11-02"
}
}Check the signature
Compute HMAC-SHA256 with your secret over <t>.<raw body>, as hex, and compare it with v1 in constant time. Use the raw bytes you received, before any JSON parsing, and reject a t more than 5 minutes from your clock. Every starter receiver below does this.
Create, update, and answer
- The first time you see an
idempotency_key, create the post. Every later request with the same key is an update of that post (after an edit and Update live post, or a retry). Don't cache answers by this key: you would miss updates. To spot a repeated request, useX-Searchlark-Delivery. - Answer within 20 seconds with a 2xx status and small JSON:
{ "id": "123", "url": "https://yoursite.com/blog/the-post" }. Theurlmust be absolute; Searchlark uses it to link the live post and to match it in analytics and site health. Youridcomes back asexternal_idon updates. - Answer directly: redirects (301, 302, 307, 308) are not followed and count as a failure. Answers larger than 1 MB aren't read.
- Want to review posts in your CMS first? Save them as drafts (the Node.js receiver does this with
SEARCHLARK_AS_DRAFT=1) and still answer with the future url. - Use
htmlas is (it is sanitised), or rendermarkdownyourself. Addjson_ldto the page, and downloadfeatured_image.urlif you want to host the image yourself.
When a delivery fails
- Anything other than a 2xx (including redirects, timeouts and network errors) is retried after 1, 2, 4 and up to 60 minutes, for 24 hours. Then the article is marked Failed; publish it again from the editor when your endpoint works.
- After 5 failures in a row Searchlark pauses deliveries to the endpoint for a while (up to an hour), so a broken endpoint isn't flooded.
- After 3 days of failures in a row the webhook is switched off and owners and admins get an email. Fix the endpoint, then save the webhook again to resume: your articles keep their idempotency_key, and you get a new signing secret to put on your server.
- Requests come from changing addresses (no fixed IP list), with the user agent
SearchlarkBot/1.0. Check the signature instead of allowlisting IPs.
Starter receivers
Each one verifies the signature, ignores tests, shows where to create or update by idempotency_key, and answers with id and url. Replace the save step with your CMS or database, and the site address with yours.
Node.js
No dependencies (Node.js 18 or later). Run it behind your HTTPS proxy, with SEARCHLARK_WEBHOOK_SECRET set. It keeps posts in memory: replace savePost with your CMS or database.
// Searchlark webhook receiver for Node.js 18+ (no dependencies).
// Run: SEARCHLARK_WEBHOOK_SECRET=... PORT=8080 node searchlark-webhook.mjs
import { createServer } from "node:http";
import { createHmac, timingSafeEqual } from "node:crypto";
const SECRET = process.env.SEARCHLARK_WEBHOOK_SECRET ?? "";
const SITE = process.env.SITE_URL ?? "https://yoursite.com";
const AS_DRAFT = process.env.SEARCHLARK_AS_DRAFT === "1"; // save drafts instead of live posts
// Check the signature on the raw body, before parsing it.
function verify(raw, header) {
const parts = Object.fromEntries(String(header ?? "").split(",").map((p) => p.trim().split("=", 2)));
const t = Number(parts.t);
if (!SECRET || !parts.v1 || !Number.isInteger(t) || Math.abs(Date.now() / 1000 - t) > 300) return false;
const expected = Buffer.from(createHmac("sha256", SECRET).update(`${t}.${raw}`).digest("hex"));
const given = Buffer.from(parts.v1);
return expected.length === given.length && timingSafeEqual(expected, given);
}
// Replace with your CMS or database. Upsert by idempotency_key: updates reuse it.
const posts = new Map();
async function savePost(a) {
const existing = posts.get(a.idempotency_key);
const id = existing?.id ?? String(posts.size + 1);
const slug = existing?.slug ?? a.slug ?? `post-${id}`;
posts.set(a.idempotency_key, { id, slug, title: a.title, html: a.html, draft: AS_DRAFT });
return { id, url: `${SITE}/blog/${slug}` }; // absolute URL of the live post
}
createServer((req, res) => {
const reply = (status, body) => {
res.writeHead(status, { "content-type": "application/json" });
res.end(JSON.stringify(body ?? {}));
};
if (req.method !== "POST") return reply(405);
const chunks = [];
let size = 0;
req.on("data", (c) => {
size += c.length;
if (size <= 5_000_000) chunks.push(c);
});
req.on("end", async () => {
const raw = Buffer.concat(chunks).toString("utf8");
if (!verify(raw, req.headers["x-searchlark-signature"])) return reply(401, { error: "bad signature" });
const msg = JSON.parse(raw);
if (msg.event === "ping") return reply(200, { ok: true });
// Never publish tests: answer as if you had.
if (msg.event === "article.test") return reply(200, { id: "test", url: `${SITE}/blog/searchlark-test` });
if (msg.event === "article.publish") return reply(200, await savePost(msg.data));
return reply(200, { ok: true }); // new event types may be added: acknowledge them
});
}).listen(Number(process.env.PORT ?? 8080));
Next.js
An App Router route handler. Set SEARCHLARK_WEBHOOK_SECRET in your hosting environment; the endpoint is https://yoursite.com/api/searchlark. Fill in savePost with your data layer.
// app/api/searchlark/route.ts: Searchlark webhook receiver for Next.js (App Router).
import { createHmac, timingSafeEqual } from "node:crypto";
export const runtime = "nodejs";
const SITE = "https://yoursite.com";
function verify(raw: string, header: string | null): boolean {
const secret = process.env.SEARCHLARK_WEBHOOK_SECRET ?? "";
const parts = Object.fromEntries((header ?? "").split(",").map((p) => p.trim().split("=", 2)));
const t = Number(parts.t);
if (!secret || !parts.v1 || !Number.isInteger(t) || Math.abs(Date.now() / 1000 - t) > 300) return false;
const expected = Buffer.from(createHmac("sha256", secret).update(`${t}.${raw}`).digest("hex"));
const given = Buffer.from(parts.v1);
return expected.length === given.length && timingSafeEqual(expected, given);
}
export async function POST(req: Request) {
const raw = await req.text(); // the raw body: verify before parsing
if (!verify(raw, req.headers.get("x-searchlark-signature")))
return Response.json({ error: "bad signature" }, { status: 401 });
const msg = JSON.parse(raw);
if (msg.event === "ping") return Response.json({ ok: true });
if (msg.event === "article.test")
return Response.json({ id: "test", url: `${SITE}/blog/searchlark-test` }); // never publish tests
if (msg.event === "article.publish") {
const a = msg.data;
// Save to your CMS or database: create on the first idempotency_key, update after that.
const post = await savePost(a);
return Response.json({ id: post.id, url: `${SITE}/blog/${post.slug}` });
}
return Response.json({ ok: true });
}
async function savePost(a: { idempotency_key: string; slug: string | null; title: string; html: string }) {
// Replace with your data layer, for example:
// return db.post.upsert({ where: { searchlarkKey: a.idempotency_key }, create: {...}, update: {...} })
return { id: a.idempotency_key, slug: a.slug ?? "post" };
}
PHP
PHP 8 or later. Upload it so it answers at your endpoint URL, and set SEARCHLARK_WEBHOOK_SECRET (or paste the secret in place of getenv). Add your save code where it says so.
<?php
// searchlark-webhook.php: Searchlark webhook receiver for PHP 8+.
$secret = getenv('SEARCHLARK_WEBHOOK_SECRET') ?: '';
$site = 'https://yoursite.com';
$raw = file_get_contents('php://input'); // the raw body: verify before decoding
function searchlark_verify(string $secret, string $raw, string $header): bool {
$parts = [];
foreach (explode(',', $header) as $p) {
[$k, $v] = array_pad(explode('=', trim($p), 2), 2, '');
$parts[$k] = $v;
}
$t = (int) ($parts['t'] ?? 0);
if ($secret === '' || empty($parts['v1']) || abs(time() - $t) > 300) return false;
$expected = hash_hmac('sha256', $t . '.' . $raw, $secret);
return hash_equals($expected, $parts['v1']);
}
header('Content-Type: application/json');
if (!searchlark_verify($secret, $raw, $_SERVER['HTTP_X_SEARCHLARK_SIGNATURE'] ?? '')) {
http_response_code(401);
echo json_encode(['error' => 'bad signature']);
exit;
}
$msg = json_decode($raw, true);
switch ($msg['event'] ?? '') {
case 'ping':
echo json_encode(['ok' => true]);
break;
case 'article.test': // never publish tests
echo json_encode(['id' => 'test', 'url' => $site . '/blog/searchlark-test']);
break;
case 'article.publish':
$a = $msg['data'];
// Save to your CMS: create on the first idempotency_key, update after that.
$id = substr(hash('sha256', $a['idempotency_key']), 0, 12);
$slug = $a['slug'] ?: 'post-' . $id;
echo json_encode(['id' => $id, 'url' => $site . '/blog/' . $slug]);
break;
default:
echo json_encode(['ok' => true]);
}
Python
Python 3.9 or later, standard library only. Run it behind your HTTPS proxy with SEARCHLARK_WEBHOOK_SECRET set. It keeps posts in memory: replace save_post with your CMS or database. In Flask or Django, reuse verify() on request.get_data().
# searchlark_webhook.py: Searchlark webhook receiver (Python 3.9+, standard library only).
# Run: SEARCHLARK_WEBHOOK_SECRET=... PORT=8080 python3 searchlark_webhook.py
import hashlib, hmac, json, os, time
from http.server import BaseHTTPRequestHandler, HTTPServer
SECRET = os.environ.get("SEARCHLARK_WEBHOOK_SECRET", "")
SITE = os.environ.get("SITE_URL", "https://yoursite.com")
POSTS = {} # replace with your CMS or database
def verify(raw: bytes, header: str) -> bool:
parts = dict(p.strip().split("=", 1) for p in header.split(",") if "=" in p)
try:
t = int(parts.get("t", ""))
except ValueError:
return False
if not SECRET or not parts.get("v1") or abs(time.time() - t) > 300:
return False
expected = hmac.new(SECRET.encode(), f"{t}.".encode() + raw, hashlib.sha256).hexdigest()
return hmac.compare_digest(expected, parts["v1"])
def save_post(a: dict) -> dict:
# Create on the first idempotency_key, update after that.
post = POSTS.get(a["idempotency_key"]) or {"id": str(len(POSTS) + 1)}
post.update(title=a["title"], html=a["html"], slug=post.get("slug") or a.get("slug") or "post-" + post["id"])
POSTS[a["idempotency_key"]] = post
return {"id": post["id"], "url": f"{SITE}/blog/{post['slug']}"}
class Handler(BaseHTTPRequestHandler):
def reply(self, status: int, body: dict) -> None:
data = json.dumps(body).encode()
self.send_response(status)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(data)))
self.end_headers()
self.wfile.write(data)
def do_POST(self) -> None:
length = int(self.headers.get("Content-Length") or 0)
if length > 5_000_000: # Searchlark sends at most about 2 MB
return self.reply(413, {"error": "too large"})
raw = self.rfile.read(length)
if not verify(raw, self.headers.get("X-Searchlark-Signature", "")):
return self.reply(401, {"error": "bad signature"})
msg = json.loads(raw)
if msg.get("event") == "ping":
return self.reply(200, {"ok": True})
if msg.get("event") == "article.test": # never publish tests
return self.reply(200, {"id": "test", "url": f"{SITE}/blog/searchlark-test"})
if msg.get("event") == "article.publish":
return self.reply(200, save_post(msg["data"]))
return self.reply(200, {"ok": True})
def log_message(self, *args) -> None:
pass
if __name__ == "__main__":
HTTPServer(("0.0.0.0", int(os.environ.get("PORT", "8080"))), Handler).serve_forever()
Cloudflare Worker
Deploy with Wrangler and add the secret with: wrangler secret put SEARCHLARK_WEBHOOK_SECRET. Add the call to your CMS where it says so.
// worker.js: Searchlark webhook receiver for Cloudflare Workers (Web Crypto).
const SITE = "https://yoursite.com";
async function verify(secret, raw, header) {
const parts = Object.fromEntries((header ?? "").split(",").map((p) => p.trim().split("=", 2)));
const t = Number(parts.t);
if (!secret || !parts.v1 || !Number.isInteger(t) || Math.abs(Date.now() / 1000 - t) > 300) return false;
const key = await crypto.subtle.importKey("raw", new TextEncoder().encode(secret), { name: "HMAC", hash: "SHA-256" }, false, ["sign"]);
const mac = await crypto.subtle.sign("HMAC", key, new TextEncoder().encode(`${t}.${raw}`));
const hex = [...new Uint8Array(mac)].map((b) => b.toString(16).padStart(2, "0")).join("");
if (hex.length !== parts.v1.length) return false;
let diff = 0;
for (let i = 0; i < hex.length; i++) diff |= hex.charCodeAt(i) ^ parts.v1.charCodeAt(i);
return diff === 0;
}
export default {
async fetch(req, env) {
if (req.method !== "POST") return new Response(null, { status: 405 });
const raw = await req.text(); // the raw body: verify before parsing
if (!(await verify(env.SEARCHLARK_WEBHOOK_SECRET, raw, req.headers.get("x-searchlark-signature"))))
return Response.json({ error: "bad signature" }, { status: 401 });
const msg = JSON.parse(raw);
if (msg.event === "ping") return Response.json({ ok: true });
if (msg.event === "article.test") return Response.json({ id: "test", url: `${SITE}/blog/searchlark-test` });
if (msg.event === "article.publish") {
const a = msg.data;
// Save to your CMS (for example its REST API): create on the first idempotency_key, then update.
return Response.json({ id: a.idempotency_key, url: `${SITE}/blog/${a.slug ?? "post"}` });
}
return Response.json({ ok: true });
},
};
No-code tools
Tools such as n8n or Make can receive the webhook if they can run a code step: verify the signature there with the same HMAC check, then call your CMS. A plain catch-hook URL without that check accepts requests from anyone, so we don't recommend it.
Troubleshooting
- 401 or 403: the secret doesn't match, the body was parsed before verifying, or the server clock is off by more than 5 minutes. Lost the secret? Save the webhook again to get a new one (article keys don't change).
- Redirect: save the final URL (https, www and the trailing slash matter).
- Timeout: answer as soon as the post is saved and do slow work afterwards.
- Delivered without a live link: answer with an absolute
url. - Posts appear twice: you created a new post for a key you had seen; look it up by
idempotency_keyfirst.
More: Publishing and the API reference.