UK BID Crime Map: methodology and data dictionary
Purpose and scope
The UK BID Crime Map compares recorded shoplifting inside each Business Improvement District (BID) with the area around it, over the same two years. It asks one question: did shoplifting in the BID change faster or slower than in its surroundings?
It does not measure whether a BID caused any change. A BID area that did better than its surroundings may have done so because of the BID, a new store, a closed store, a police operation, or a change in how a force records crime. The map shows where the numbers diverge; local knowledge explains why.
Coverage:
- 319 zones in total: 281 BID zones with a shoplifting verdict, 8 industrial BIDs (not assessed), and controls (town centres with no BID, a BID not yet operating, or a former BID).
- 43 police forces across England, Wales and Northern Ireland.
- Cross-checked against the British BIDs index of 372 BIDs: 274 covered.
Out of scope:
- Scotland (45 BIDs): Police Scotland does not publish on police.uk.
- Greater Manchester (8 BIDs): Greater Manchester Police does not publish street-level data on police.uk, and its own published counts exclude shoplifting and have no locations. A Freedom of Information request for shoplifting by small area was refused on cost grounds on 27 October 2026 because of a separate part about robbery records; a narrower request for shoplifting counts only is being submitted. Until then, official tables give district-level shoplifting for Greater Manchester, comparing the year ending June 2026 with the year before: Bolton −35%, Rochdale −29%, Salford −15%, Oldham −14%, Bury −6%, Tameside 0%, Manchester +3%, Stockport +3%, Wigan +6% and Trafford +16%. Force-wide the change is −6%, against −2% for England and Wales. These are whole-district trends, not BID results.
- Republic of Ireland (5 BIDs).
- Industrial, business park, tourism, accommodation and destination BIDs (28): shoplifting is not a meaningful measure for them.
- Not yet run: North Acton (two BIDs, starting June 2026) and North Notts (an area-wide BID that needs a boundary definition).
The primary measure is shoplifting. All crime and anti-social behaviour (ASB) are reported as context, not verdicts.
Data sources
Every crime figure in the zone results comes from open police.uk data. Nothing is estimated or modelled.
| Source | What it provides | Notes |
|---|---|---|
| police.uk street-level crime files | Each recorded crime: month, type, approximate location (a map "snap point"), area code | September 2023 to August 2026, 36 months per force |
| police.uk outcomes files | The latest outcome for each crime | Used for charge and no-suspect rates; Northern Ireland publishes none |
| police.uk stop-and-search files | Searches with location and outcome | Context only; coverage varies by force |
| ONS Police Force Area data tables | Official police recorded crime by force and local authority | Used to check police.uk completeness (see below) |
| GLA London BID boundary file | Official boundaries for London BIDs | Grade A |
| Council and BID business plans, ballot papers, maps | Street lists, maps, term dates, services | Grade B where an official street list exists |
| British BIDs index and BID Data register | Which BIDs exist and their type | British BIDs used as the reference |
Police.uk locations are deliberately approximate. Each crime is moved to the nearest of a fixed set of snap points (a street, a car park, a supermarket, a shopping area) to protect victims. The analysis works with those points, not exact addresses.
The comparison
Each zone gets a gap: its percentage change in shoplifting minus its comparator's percentage change.
Gap = (Y3zone ÷ Y1zone − 1) × 100 − (Y3comparator ÷ Y1comparator − 1) × 100
- Year 1 (Y1): September 2023 to August 2024. Year 3 (Y3): September 2025 to August 2026. The middle year is left out of the comparison.
- Comparator: the rest of the local authority area outside the zone (for example "Rest of Wiltshire"). London BIDs compare with the rest of their borough. Northern Ireland, which has no area codes in its files, uses everything within 8 km (Belfast) or 15 km (towns).
- Verdict: a gap of −10 or lower = fell against trend; +10 or higher = rose against trend; in between = in line.
- The gap is in percentage points of change, not a crime rate. A zone going from 10 to 20 offences shows +100%, so small zones produce large gaps.
Window adjustments where a force's data has a break:
| Force | Months used |
|---|---|
| Devon and Cornwall | December to August (Sep–Nov 2023 under-recorded) |
| Hampshire | September to June |
| Northumbria | January–February excluded |
| Gloucestershire | September to January (data ends January 2026) |
| Dyfed-Powys | January–April excluded (shoplifting missing Jan–Apr 2024) |
The same months are used in both years, so the comparison stays like for like.
Boundaries and grades
Every zone carries a boundary grade.
| Grade | Meaning | Typical source |
|---|---|---|
| A | Official GIS boundary | GLA London file; council GIS layers |
| B | Official street list or official map | BID business plan appendix, ballot proposal, current BID map |
| C | Traced or own definition | Map traced by hand, core shopping streets chosen by the analyst, or a street list from an unverified source |
How a zone is built from a street list: crimes on listed streets are matched to their snap points; a convex hull is drawn around those points and extended by 40 metres. Where named points are too sparse for a hull, a radius around the town centre is used instead, and the row says so.
Core-cluster extension (grade C only): where a hull missed a town's main shopping point, a 150 m circle around the densest town-centre shoplifting cluster is added. That cluster is located using the middle year (September 2024 to August 2025), which favours neither side of the comparison. See the corrections log.
Grade C results are provisional. The open call to councils and BIDs (last section) is designed to move them to A or B.
Robustness tests
A verdict is marked robust only if it survives every test below. If any test changes the verdict, it is marked not robust. The rule is deliberately conservative: 107 of 240 directional BID verdicts are robust (62 fell, 45 rose).
| Test | What it does | Why |
|---|---|---|
| Without supermarkets | Recomputes the gap with all supermarket snap points removed, in zone and comparator | One store opening, closing or changing its reporting can swing a whole zone |
| Without the top location | Removes the single snap point with the largest change | Catches results resting on one spot |
| Boundary variants | Recomputes with a strict hull (no buffer) and with street-name matching | Shows whether the verdict depends on where the line is drawn |
| Re-pinning check | Finds a point that drops sharply while a point within 400 m on the other side of the boundary rises by a similar amount in the same months; merges the pair both ways | police.uk sometimes moves crimes from one snap point to another |
| Comparator distortion | Flags comparators dominated by out-of-centre retail or a force-wide surge | Applied to Cleveland and noted for Ipswich |
Small numbers: zones with fewer than 100 shoplifting offences in Y1 are flagged small_numbers = Yes and marked with an asterisk on the dashboard. Their gaps can be very large on a handful of offences, so read them as indicative only.
Check against official statistics
Police.uk is a public extract of the same police records that feed the official statistics. To test whether the extract is complete, police.uk shoplifting counts were compared with the ONS Police Force Area data tables (Home Office police recorded crime, year ending June 2026 edition) for July 2024 to June 2025 and July 2025 to June 2026, for 41 forces, the City of London and 292 local authority or community safety partnership areas.
- Forces: police.uk counts are within 5% of the official totals for 36 of 41 forces. The year-on-year change agrees within one percentage point for 27.
- Local areas: the change agrees within 2 points for 153 of 292 areas and within 5 points for 224.
- Explained gaps: police.uk months are missing in Gloucestershire (files end January 2026), Northumbria (January 2026) and Hampshire (August 2025), all already excluded from the comparison windows. In five Kent areas (Maidstone, Medway, Tonbridge and Malling, Tunbridge Wells, Dartford and Gravesham), ONS notes a force IT issue, so the official tables are the weaker source there.
- Flags: in Surrey, the Metropolitan Police area and Essex, police.uk was short of the official count in the earlier year and close to it in the later one, so police.uk changes run about 3 to 12 points higher than the official change in some boroughs and districts. In the City of London, police.uk counts run about 20% above the official figure, probably because points near the boundary are placed inside City areas.
A second check compared the year ending September 2024 (close to our first year) with the year ending June 2026 (close to our third). It confirmed the same pattern and picked up the known Dyfed-Powys gap in early 2024. Thirty areas diverged by more than 5 points in the same direction in both checks, after removing the explained gaps.
Every zone is compared with the rest of its own local authority from the same source, so a shortfall spread evenly across an area cancels out of the gap; one concentrated in town centres would not. Annual totals cannot show which, so the 45 zones whose comparator lies in a flagged area carry a caution in their caveat rather than a correction. The full check is published as a CSV. It will be rerun on the year ending September 2026 edition (due 28 January 2027).
Known data problems by police force
Seventeen forces have a recording break, a data gap or a publishing problem. Each is handled as shown.
| Force | Problem | Handling |
|---|---|---|
| Devon and Cornwall | Sep–Nov 2023 under-recorded; ASB break from Dec 2025 | Dec–Aug window; ASB not reported |
| West Midlands | ASB records roughly halve from Sep 2025 | ASB not reported for any West Midlands zone |
| Hampshire | July–August data unreliable | Sep–Jun window |
| Northumbria | Jan–Feb 2026 short | Jan–Feb excluded |
| Gloucestershire | Data ends January 2026 | Sep–Jan window |
| Dyfed-Powys | Shoplifting missing Jan–Apr 2024 | Jan–Apr excluded |
| Cleveland | Force-wide shoplifting surge; comparators dominated by out-of-centre retail | Teesside zones marked not robust |
| Humberside | Force-wide shoplifting jump from Mar 2026 | Absorbed by comparator; noted |
| Lancashire | ASB halves from Jan 2026 | ASB treated with caution |
| West Mercia | ASB break Apr–May 2026 | ASB treated with caution |
| Northern Ireland (PSNI) | No outcomes data; no area codes | Charge rates blank; distance-based comparator |
| Bedfordshire, Norfolk, Nottinghamshire, South Yorkshire, Gwent | Stop-and-search incomplete or missing | Stop-and-search not used for these forces |
| Greater Manchester | Not published on police.uk | Out of scope |
Corrections log
Every change that moved a published or circulated figure is logged here, newest first.
| Date | Version | Change | Effect |
|---|---|---|---|
| 28 Oct 2026 | v52 | Internal shorthand removed from 25 caveats | Wording only; no figures changed |
| 25 Oct 2026 | v51 | Check against ONS official statistics | Caution added to 45 zones whose comparator area diverged from the official count; no verdicts changed |
| 25 Oct 2026 | v50 | Core-cluster extension re-located on middle-year data. Previously the densest cluster was found using Y1 data, which biases results towards falls (regression to the mean). | 49 zones redrawn; average gap moved about +7 points towards rises; Melton fell → rose, Okehampton in line → rose; Newcastle-under-Lyme −78 → −38 |
| 25 Oct 2026 | v49 | Cross-check against the British BIDs index | Buxton, Morley, Trowbridge, Wednesbury reclassified as controls (BID still developing); Torquay, Paignton as former-BID controls; Bracknell as a control (its BID covers a business park); nine missing BIDs added |
| 24 Oct 2026 | v30 | Re-pinning check run on all zones | Six London verdicts changed; West Fitzrovia and New West End Company lost robust status; Bedminster control changed fell → in line |
| 24 Oct 2026 | v21–v29 | Comparator and boundary corrections during the national build (e.g. Woking found to have no BID and moved to controls; Worcester boundary widened) | Logged in each row's caveat |
How to read a result
- Verdict and gap. "Fell against trend, −31" means shoplifting in the zone changed 31 points less than in the comparator. It may still have risen in absolute terms.
- Robust. If not robust, the caveat names the test it failed, usually one supermarket or one location. Treat the verdict as unproven.
- Grade. A grade C boundary may not match the BID's real area. The result can change once an official boundary arrives.
- Caveat and small numbers. A zone with 20 offences in Y1 can show a gap of hundreds of points from a few incidents.
What the national picture shows: of 281 BID zones, 126 fell against trend, 114 rose and 41 were in line. Of the 107 robust results, 62 fell and 45 rose. Controls (town centres with no BID) show a similar mix.
What not to conclude:
- That BIDs reduce shoplifting, or that they do not. The comparison is not designed to show cause, and the split is close.
- That one zone's result is a verdict on that BID's management. Stores, police operations and recording changes move the numbers too.
- That a fall means less shoplifting happened. Recorded crime depends on reporting. Active patrols or better reporting schemes can raise recorded shoplifting while deterring it.
- Anything from a percentage alone. Always check the Y1 and Y3 counts.
Data dictionary
The dataset has one row per zone and 33 columns. Percentages are whole numbers; counts are offences.
| Column | Meaning |
|---|---|
id | Zone name, unique |
region | London, Outside London or Northern Ireland |
area | Town or city |
zone | What the zone covers |
role | BID, new BID (with start date), industrial BID, or control type (no BID, pre-BID, former BID, developing) |
force | Police force supplying the data |
comparator | The surrounding area the zone is compared with |
window | Months compared |
all_A, all_B | All recorded crime in the zone, Y1 and Y3 |
all_pct, comp_all_pct | % change in all crime, zone and comparator |
shop_A, shop_B | Shoplifting offences in the zone, Y1 and Y3 |
shop_pct, comp_shop_pct | % change in shoplifting, zone and comparator |
shop_gap_pts | shop_pct minus comp_shop_pct |
shop_verdict | Fell against trend (≤ −10), In line, Rose against trend (≥ +10), Not assessed |
robust | Yes / No / blank (in line, or not assessed) |
verdict_ex_supermarket | Verdict recomputed without supermarket points |
small_numbers | Yes if fewer than 100 shoplifting offences in Y1 |
asb_pct, comp_asb_pct | % change in ASB, zone and comparator; blank where the force has an ASB break |
shop_charge_rate_centre | Share of zone shoplifting charged v comparator |
nosusp | Share of zone shoplifting closed with no suspect v comparator |
boundary | Grade (A/B/C) and how the zone was drawn |
operating_model | BID term, ballot and services, from published sources |
caveat | Robustness results and anything that limits the verdict |
wave | London, Pilot or National build |
crime_tech | Crime-reporting systems named in BID documents (e.g. DISC, radio) |
staffing_model | Patrol or street staff named in BID documents |
police_funding | BID-funded police time, where documented |
british_bids_status | Result of the cross-check against the British BIDs index |
Blank operating model, staffing or crime-tech cells mean not yet documented, not absent.
Corrections, contributions and next steps
This is a first pass, published so that BIDs, councils and others can correct it. Every correction with a source will be applied and logged.
- Write to councils. Councils bill the BID levy, so they hold each BID's property list and boundary. We are writing to the councils for every grade C BID, starting with those with the most recorded shoplifting.
- Open call to BIDs. Any BID can send its boundary map or street list, current term dates and services (patrols, rangers, reporting systems, funded police time). Grade C zones will be redrawn on what they send.
- Second pass, BID by BID. Close out data gaps, confirm term status for BIDs not in the British BIDs index, and add ballot results.
- Independent re-derivation. A reviewer who has not seen the analysis will recompute a random sample of zones from the raw police.uk files.
- Quarterly refresh as police.uk publishes new months.
How to send a correction: name the zone, say what is wrong, and attach or link the source (business plan, map, ballot notice, council record), to dan@oxcam.online.