Research and original data

This section is where original research published by this site will live. It currently contains no findings, and that is the accurate and intended state of it rather than an unfinished one. The method is published first. The studies have not been run.

Everything else on this site rests on figures gathered by somebody else. The demand data was bought from a search-data provider and is dated and filed. The legal material is a reading of published instruments and regulator guidance. The pricing bands come from delivered projects and are governed by their own publication guard. Those are all legitimate, and how each was gathered is set out on how this site measures things and what the dataset does and does not cover. None of it is original research. A tool’s dataset read carefully is still a tool’s dataset.

Why this web design research section has to exist

A site that spends several pages pointing out that a competitor’s cost figure has no method behind it has taken on an obligation by doing so. If the objection is that a number arrived without a sample, a date or a collection method, then eventually the objection has to be paid for. Otherwise the criticism is just a way of clearing the field of everyone else’s figures while publishing none of your own.

So this section exists to produce numbers about Australian web design that a reader can interrogate: where the sample came from, when it was collected, what the instrument was, and what the raw material looks like underneath the summary.

The rules this section is bound by

These apply to anything published here, and they are the reason the section is being built before it has anything in it.

The method is published before the findings. A study gets its page, its question and its full method written and dated first. The result is added later, to a page that already exists. A method written after the result is known is a description of the result.

Every figure carries its sample size, its collection dates and its collection method. Not in a footnote and not on a separate page. Beside the figure.

Raw data is released where it can be released. Where it cannot — because it would identify an individual business, or because a data provider’s terms do not permit redistribution — the reason is stated on the page, and the instrument is released even when the responses are not.

Every study that runs gets a page, whether or not the result was interesting. A register of studies where only the good ones surfaced is a publication bias with a public face on it.

Null results and boring results are published rather than buried. A study that finds no difference, or finds the obvious, is published in the same form as one that finds something. This is the rule that is easiest to state and hardest to keep, so it is written down where it can be checked against what actually appears here.

What studies and research are here at the moment

Nothing has been measured. There are no findings, no partial findings, no preliminary numbers and no early reads. No figure has been gathered and withheld.

One study is fully defined and has not been run: an accessibility audit of Australian small-business websites. Its question, sampling frame, instrument and limitations are written out in full, and its status is stated on the page as not started.

The standing rules for how anything here would be measured are on the research method. Everything currently under consideration, with its status, is on planned studies. Listing a study there is not a commitment to run it.

What web design research usually studies: usability, navigation and visitor behaviour

Published web design research typically covers a narrow, recurring set of topics: usability testing of specific page types, how visitors actually navigate a site’s structure, the cognitive load a page’s visual and graphical elements impose, and survey-based studies of what visitors say they want against what they are observed doing. Google and other search engines have published their own studies on navigation and its effect on visitor behaviour, and the web design and website design literature draws heavily on them. This site’s own studies, once run, sit inside that same tradition: each study is a narrower, dated, Australian-specific instance of it, not a new category of research.

Why the method goes first, and what that is worth

The idea is borrowed loosely from pre-registration in clinical research, where a trial’s endpoints are lodged publicly before the data is collected so that the endpoints cannot be shifted afterwards to whichever one happened to come out well. The point is not ceremony. It is that a method fixed in advance can be broken by a result, and a method written afterwards never can be.

It is worth being clear about how much weaker this version is. There is no registry here and no third party holding a copy. The protections are a published page carrying its own publication date, and a version history in the site’s repository. Both are held by the same person who would benefit from changing them. A dated page on your own website is a commitment, not a proof, and it should be read as the first and not the last of them.

The conflict of interest, which is not small

The shape of the problem is specific enough to name. A study finding that Australian small-business websites are widely inaccessible produces demand for accessibility work, which the author’s agency sells. A study finding that quotes in this market are vague produces demand for a supplier who quotes in detail. The direction in which a convenient result points is knowable in advance for every study listed in this section, which is exactly the condition under which method drifts without anyone deciding to drift it.

The structural protections are the ones above: the method is fixed and dated before collection, the sample size and dates travel with every figure, the raw instrument is released, and studies are registered before they run so that an inconvenient one cannot quietly disappear. Those constraints make certain manoeuvres visible. Changing an instrument after seeing the data leaves a trace. Dropping a study leaves a gap in a register that says abandoned rather than showing nothing at all.

They reduce the problem. They do not remove it. Sampling frames, category boundaries and the choice of what to measure at all still involve judgement, and judgement made by an interested party tends in a direction whether or not anyone intends it. The honest position is that this research should be read as coming from a competitor in the market it describes, and that the appropriate response to any finding published here is to check it against a source with no stake in the answer.

The full commercial position is on commercial relationships.

How anything here would be measured

This section has no findings yet, so the method matters more than usual: publishing it now means it cannot be adjusted afterwards to suit whatever a study turns out to show.

Any study published here will state, before its results, how its sample was drawn and how large it is, what tool took each measurement and under what settings, the date collection started and finished, and what the method cannot establish. Where a figure comes from a tool rather than direct observation, the page will say which tool and what that tool is actually counting — a number a tool produced is not the same as the thing the tool was summarising.

The site-wide measurement method, covering the search-demand figures used throughout, is on how this site measures things. This section adds nothing to it; it commits to it in advance. For what web design covers as a discipline, separate from the research about it, start with web design in Australia.