Revolução Industrial Data

Working with Industrial Revolution datasets

I spent three weeks last year trying to clean together production output figures from British textile mills between 1780 and 1840. The data exists, it just arrives in fragmented forms. Parish records, factory inquiry reports, customs ledgers, and occasional private accounts that someone digitized with poor OCR quality. Merging them required more patience than skill.

revolução industrial data sources and what they actually contain

The core repositories are scattered across institutions. The UK National Archives holds the factory commission reports. Historians frequently cite the work of historians like David Landes and Robert Allen, whose datasets on wages and energy prices form a baseline. Then there are regional sources: the Lancashire cotton trade directories, Welsh mining returns, and the Scottish weaver accounts collected by the Board of Trustees for Manufactures. I encountered a specific problem with the 1833 Factory Inquiry reports. The digitized text used inconsistent date formats. Some entries read "15th March 1833," others "3/15/1833," and a third batch used the old style where the year started at Easter. My workaround was to parse everything through a date-normalization script that first identified the source collection, then applied the correct calendar rules. This usually cuts the process down from two hours to about fifteen minutes, depending on your setup.

The data problems you will face

Missing values are not the main issue. The bigger problem is definitional drift. "Output" meant different things in different contexts. A mill owner reporting "spindles running" might include idle machines during breakdown periods. Customs records track pounds sterling but rarely quantity. Population figures come from ecclesiastical censuses before 1801, then parliamentary surveys after. Another pitfall beginners miss: regional price variations matter enormously. A wage of one shilling per day in Manchester does not equal one shilling in Cornwall when you adjust for food prices. The substitution pattern between coal and water power also shifted geographically. Running water mills dominated the early period in the Pennines, but steam engines concentrated later near coalfields. Simply averaging regional data obscures this transition.

Where to actually find usable datasets

The Historical Statistics of the United States contains some comparative material but focuses later. For Britain, the Oxford Economic History dataset series provides wage and price series. The British Household Panel Survey has reconstructed analogs going back through the Millennium Archive. The Census of Production from 1907 onward is well digitized but misses the core period. For textile output, consult the annual reports of the Royal Commission on the Hand Loom Weavers. They published quantity estimates by county, though the coverage is patchy for the 1790s. The iron industry data appears in the Annals of Irontrade and the Mining Journal archives, though you need to filter out promotional content from actual production figures.

Building a workable dataset

Start with the Allen dataset on relative prices and wages. It covers 1250 to 2010 with consistent methodology. Cross-reference with the Broadberry, Campbell, Keller, and Van Leeuven work on British growth. Then layer in sector-specific sources: the Textile History bibliography for mill records, the Iron and Steel Institute archives for metallurgical data. The merge usually requires creating a key field from composite attributes. Source name, date range, and geographic code form a stable junction. I recommend using SQLite for the intermediate work rather than pandas, because the join operations handle irregular date ranges better. The final export to CSV or Parquet takes about five minutes on a standard laptop.

Limitations and when to stop

This approach fails for pre-1750 estimates because the documentation gaps become too large. Agricultural census material exists but does not connect reliably to industrial output. The gap between rural cottage industry and factory production blurs, and no dataset resolves it cleanly. If you need figures before 1750, switch to proxy methods like tax records or estate papers, but accept higher uncertainty bands. Regional datasets also become unreliable after 1914 due to wartime recording changes and subsequent administrative reorganizations. The shift from imperial to metric units created conversion errors in mid-century publications. Verify your source dates against the Weights and Measures Act timeline before trusting any pre-1824 figure.

Practical workflow for revolução industrial data projects

The sequence matters. First establish your temporal and geographic boundaries. Second, collect the baseline series from Allen and Broadberry. Third, add sector-specific sources one at a time, validating each against the baseline. Fourth, document every missing value and assumption. Fifth, publish your methodology alongside the dataset. Anyone replicating your work should be able to follow the same path without guessing which version of a report you used. The whole process usually takes two to three weeks for a well-defined project covering a single sector and fifty years. Expect longer if you need cross-regional comparisons or pre-1800 material. Factor in extra time for handling inconsistent date formats and verifying source provenance.