The Front Door Your Best Dataset Never Had

Your data is valuable, but nobody can find it behind a login. What if every dataset had its own searchable landing page — and buying it just unlocked an API key? That’s the VBWD dataset vertical.

Rewritten for redrobot.

Picture the last useful dataset you built. Not the messy one, the good one: clean columns, a clear meaning, something a specific stranger would happily pay for. Now answer honestly. If that stranger typed the exact thing they need into a search engine tonight, would they ever arrive at a page where they could buy it? For most people sitting on good data, the answer is no. The data exists. The path to it does not.

That missing path is a surprisingly specific piece of engineering, and it is worth reverse-engineering. Work backwards from the moment money changes hands and you find a short chain of things that all have to be true at once: a page a crawler can read, a checkout that means something, a record of who bought what, and an endpoint that actually enforces it. Miss any link and the whole thing collapses into “email me and I’ll send you a CSV.” VBWD’s dataset plugin exists to make that chain hold on a self-hosted, source-available stack (Python/Flask, Postgres, Vue 3, Docker). Let me trace it from the buyer’s click back to your raw data.

Start at the click, not the code

Someone searching for “regional EV charging stations, monthly, with connector types” is not searching for your marketplace. They have never heard of your marketplace. They are searching for the thing. If your entire catalogue lives behind one JavaScript route that renders the word “Marketplace” in the browser tab, that search never touches you. The dataset could be perfect and it would still be invisible.

So the first link in the chain is a page per dataset that a search engine can actually index. In VBWD this comes from per-entity SEO: a plugin can attach a page_seo record to a single dataset entity, giving it its own title, description, and social share image, served at its own URL. Ten datasets means ten distinct front doors. Two hundred datasets means two hundred, none of them hand-written. The platform features generate the page from the entity plus its metadata rather than from a landing page you build by hand.

There is a catch that bites every single-page app. A Vue frontend renders in the browser, so a naive crawler fetches the URL and sees an empty shell, no title, no description, nothing to rank. VBWD runs an SEO renderer sidecar that produces a real, fully rendered version of the page for bots, so the per-dataset metadata is present in the HTML a crawler receives. The behaviour, and how it hooks into the entity metadata, is written up in the documentation. Worth being precise about what this buys you: it removes the technical reason a page can’t be indexed. It does not conjure a ranking. That distinction comes back later.

The checkout that means something

Say the page did its job and the buyer clicked “buy.” Here the design gets interesting, because the dataset plugin deliberately knows almost nothing about the sale.

A dataset is registered as a Priceable, a first-class sellable object. It carries a price through the platform’s unified price model, so tax, currency, and discounts are computed exactly the way they are for every other product. You write no bespoke pricing math for data. It lists in the catalogue through the same filter and pagination contract as any other product grid. Payment then runs through swappable payment plugins, and the dataset plugin does not care whether the buyer paid by card, by bank transfer, or by crypto.

What it cares about is a single fact: a line was paid. The mechanism connecting the two is an invoice-line link registry, a small contract that says “when an invoice line of this type is paid, run this handler.” The dataset plugin registers a handler that turns a paid line into an entitlement. That indirection is the whole point. The platform core never learns what a dataset is; the same seam that grants a data license could grant a subscription or a booking without the core knowing the difference. Swap the payment provider and the dataset code never moves. If you want to see the money side on its own terms, the billing model is documented separately from the products that ride on top of it.

The record, and the gate

When that handler fires, it writes an entitlement: a durable row recording who bought what, and under what terms. Not a flag in someone’s browser, not a session token, a row in Postgres that survives reloads, token refreshes, and support tickets six months later.

That entitlement is the gate on the data itself. A request to a dataset endpoint checks for a live entitlement before it returns a single row. Here is the honest boundary: the plugin hands you the gate, and you build the door. What the endpoints look like, which fields you expose, how you paginate, what rate limits apply, how you handle a buyer who tries to drain their entire allocation in an hour, all of that is your design work. The gate answers “is this caller allowed in.” It does not answer “should this specific request be served right now.” That second question is engineering you own, and it is where the real product lives. You can see how the dataset piece sits alongside the other sellable verticals on the plugins page.

Where the data comes from

All of this assumes fresh data to sell, and hand-exporting CSVs stops scaling around your third product. That is the job of datavibes, a config-driven companion toolkit that emits versioned CSV snapshots of a dataset on a schedule. You describe the source and the shape once; datavibes runs on cron or a GitHub Action and produces a new versioned snapshot each cycle.

The division of labour stays clean: datavibes makes the snapshots, the dataset plugin sells them and gates them. Because snapshots are versioned, a dataset is not one frozen file but a series, and your entitlement terms decide whether a buyer gets the current version, one specific version, or an ongoing feed. “Monthly refreshed” becomes a config line and a cron entry instead of a recurring chore you’ll eventually forget to do.

The parts nobody can automate for you

Three honest limits, because ignoring them is expensive.

You host the data, so you own its licensing. The plugin gates access; it does not vet your right to sell what you’re selling. “Public source” is not the same as “free to resell.” Scraping a public site or aggregating an open portal does not automatically grant redistribution rights, and terms vary source to source. That is your legal exposure. And a dataset you sell is not in the public domain by default; do not label it as freely reusable unless you genuinely mean to give it away.

A crawlable page is not a ranking. Per-entity SEO removes the technical blocker. It cannot make a thin, auto-generated description on a dataset nobody searches for climb the results. Ranking still depends on the description being genuinely useful and the data being something people actually want.

The gate is not the API. Abuse prevention, response shape, rate limits, and fair-use enforcement are yours to build. Budget engineering time for the endpoint, not just the entitlement check behind it.

Why self-hosting matters here

Trace the chain one more time and notice what you never had to touch: the platform core. You wrote config, a price, some SEO text, and your own API. Everything else was seams the platform already exposed. Because the stack is self-hosted and source-available under BSL 1.1 (free for commercial use while annual VBWD-attributable sales stay under the value of 6.7 BTC per year), you own the data and the customer relationship rather than renting a marketplace that owns both. The undifferentiated plumbing between “I have a file” and “a stranger paid me for access while I slept” is handled. The parts that decide whether the product is worth buying, the rights, the quality, the API, the writing good enough to rank, stay exactly where they belong: with you.