Big data in a historical context


Excellent stuff from Alan Patrick on his Broadstuff blog, talking about the 70s, 80s and 90s versions of big data – or “data”, they were calling it back then…

And you know what – you just cannot simulate the minute operation laden details of a shop floor or logistics network reliably. No matter how big your dataset, or your computers, or your machine tool onboard intelligence, there is just too much variability. Which is why the Just In Time/Lean movement came about as the better approach – the aim was to simplify the problem, rather than hit it with huge algorithm models and simulations so complex no one fully understood what they were doing anymore (just ask the banks what happens going down that route) – the aim of JiT/Lean was to actually reduce the problem variability, to get back to Small Data if you like.

Alan discusses the way that despite fascination with new technology and algorithms, the drumbeat that industry marches to is that of economics – in this case the pendulum swing of offshoring and onshoring, powered by the temporary advantage of emerging economies’ lower labour costs.

[….] It’s back to the future….I suspect they are now using bigger and bigger number crunching to eke the last 20% of improvements from the various kaizen projects ongoing, trying to keep the factories in situ as the Big Economics shift yet again

The rate of change today often feels bewildering at ground level, but keeping one eye on the forces of history and economics, we see ourselves in the context of slower moving, but more significant trends. In The Second Machine Age – which I’ve been fixated with over the last week (I even look dangerously close to finishing it) – the authors point out that

  • productivity gains from electric motors took about 30 years to emerge in manufacturing.
  • steam engines unlocked 100 years of productivity gains (and an exponential growth in human population).
  • microprocessors and the IT revolution unlocked meagre productivity gains until the late 1990s

What drove productivity in these instances was innovation that used the technology better – innovation in products, processes, organisation and management. When we’re looking at new technologies in our lives and workplaces like social computing, big data etc. it could be decades before their actual potential is felt by all bar the early adopters that are able to see their potential and change their mindsets and ways of working fastest.

More new and old media pairing

ZZ03FF51C3More from Writing on the Wall, by Tom Standage, on how old media is often – for a while – enhanced by the new,  rather than being replaced. 

In England in the 1600s, newsletters were distributed about parliamentary and Royal news by mansucript subscription “news letters”. They literally began as letters, which were copied by teams of scribes and sent out – often to be shared in groups, read aloud or copied and passed on again.

Printed newsletters (called “Corantos“)  were largely, at first, about foreign news – partly as a consequence of strict censorship laws. However, some bright sparks in the manuscript trade started included the printed foregin news – the first newspaper supplements, apparently:

But rather than competing, the two forms proved complementary. Corantos could be enclosed within manuscript news letters as they circulated, providing printed foreign news alongside the handwritten domestic sort. Letters from this period contain abundant references to printed material […] entire transcribed copies of them and, on several occasions, the printed corantos themselves. Coratnos were printed versions of what were originally manuscript documents, and the information they contained was in turn recycled into manuscript news networks.

There have always been news networks – and there have always been social networks bound up in them.

Image credit: (cc) Wikipedia 

New technology boosts the old


As cheerleaders for incumbent media often point out, the old is rarely replaced by the new. Newspapers weren’t killed by radio, radio wasn’t killed by TV, TV wasn’t killed by online video – etc., etc.

Sometimes new technology boosts the old.

Dipping in again to the excellent Writing on the Wall: Social Media, the First 2,000 Years, I read:

Printing pushed up demand for paper throughout Europe, encouraging production and making it cheaper (its price fell by 40 percent during the fifteenth century) and more widely available. Printed books promoted literacy and writing manuals could be produced in quantity.

We can see a similar effect today with writing and books. Earlier in Writing on the Wall, Tom Standage notes that book writing was a serious undertaking in Roman times. You had to be literate, rich enough to have a dedicated cohort of slaves for scribing and couriering purposes during the research, notable enough to throw a top-notch launch party and – by some advice of the time – spend about nine years perfecting your manuscript before releasing it into the network of copyists (all reproduction was by hand, of course).

Now writing – and publishing – books is within the grasp of anyone. A cynic would say that you don’t even need that high a degree of literacy.

In the US, 391,000 books were self-published, only about a third of these were e-book only titles. In fact, an article in the Guardian notes, this figure is conservative:

The exclusion of hundreds of thousands of titles published without an ISBN, including many titles on Amazon’s Kindle store, means that the increase of 422% since 2007 this represents is likely to be an underestimate of the size of the self-publishing sector.

Rather than reach for the pessimist’s fall-back of the monkey-typewriter paradigm, recognise this for what it is – a golden age of reading and – even more – writing. New forms of media are making old forms easier for everyone to access and work with, once again.

Change: we’ve been here before

In his book about the start of the First World War, Max Hastings discusses the incredible rate of change – new technologies, ideas, social forces – that were in play in the opening decades of the last century. Reflecting, in 1930, on how dramatic the changes in the world were Winston Churchill said:

Scarcely anything material or established which I was brought up to believe was permanent or vital has lasted. Everything I was sure – or taught to be sure – was impossible, has happened.

Whether for useful perspective or rhetorical symmetry, many are drawing parallels between 2014 and 1914 – rising superpowers, faltering hegemonies, world order in flux, communications technology muddling old certainties about the relationships between creaking elites and restless populaces.

Regardless of how similar today is to that terrible year, it is clear that there’s nothing very new about rapid, disruptive, global change. We need to be looking back as well as forward as we face our own challenges and opportunities.

London’s riverside in 1849

Old books. History. London.

Yes, these are things that make me curious and slightly giddy with delight when they all come along at once…

So I love Sydney Padua's post about her latest antique book purchase – a fold out panorama of London's riverside in 1849. Call it the Google Streetview of its time.

This is a section showing river traffic in Greenwich….

Have a look at the whole thing. It's gorgeous…


Always look on the brightside of the downside…

Image: Grin-and-bear-it optimism...
Image: Grin-and-bear-it optimism...

Being utterly besotted with the web, and especially the social web, as I am, I tend dislike nay-saying about its significance, and the manifold benefits this thing will bring to society, the world etc. You know the sort of Daily Fail nonsense: Facebook gives you cancer, Twitter rots your brain, bloggers never meet real people.

But there’s a difference between reactionary nonsense and thoughtful critiques. Over at the O’Reilly Radar blog, Joshua-Michéle Ross has been poking at some of the more troublesome prospects that social technologies bring. Like how much of our identity and personal data are we surrendering for analysis by corporations and governments (since analysis of that data is a big part of my business, but I also value personal freedom that’s a particularly interesting issue for me).

He takes through a series of four posts that I highly recommend reading:

  1. The Evangelist Fallacy, Social Media and The New Age of Enlightenment: In which we are reminded that the Enlightenment with which we draw so many parallels to today brought not just progressive new ideas about equality and rights, but new (very effective) thinking about how to control the massses.
  2. The Captivity of the Commons: With the whole world connected and people living their lives in public we need to re-think privacy and how corporations work (so that they are less amoral).
  3. The Digital Panopticon: How the nightmare of the Panopticon is effectively at hand if corporations are able to see every detail of our livs in plain sight.
  4. Social Science Moves from Academia to the Corporation: Funding for social sciences will increasingly come from corporations as they try to understand how to manipulate mass social media.

As Alan Patrick says on Broadstuff:

hat makes this post extremely fascinating is that it comes from the O’Reilly Radar, which – in my experience anyway – have tended to be on the “cup overfloweth” side of the New New Social Thing, never mind a Glass Half Full – so this Glass Half Empty article – the first, it seems, of a series, is a rather fascinating shift of tenor, methinks.

He senses the beginning of a backlash, good and proper, perhaps coming from businesses (that aren’t managing to figure out how to get value out of networks as fast as Joshua-Michéle fears) as well as individuals wanting to rein in how much web shadow they are comfortable casting.

Meanwhile, Ian Delaney has a melancholy reflection on this subject that makes for good further reading and thinking matter, about how his early hopes that social media would bring socialist values to the fore are fading. He picks up the Panopticon analogy and extends it to society.

philosopher Michel Foucault back in the 70s picked up and ran with the idea of the Panopticon, especially in his best-known work Discipline and Punish. His idea was that Bentham’s model wasn’t just an idea for a prison; but for a society.

He argued that prisons are a really new idea. Back in the past, we simply thrashed/burned/drowned/stabbed transgressors. That all changed in the C18th with the Enlightenment . The idea of law-enforcement was ‘enlightened’ with the  understanding that resources [people] didn’t need to be wasted and that better social control is exercised through freely-given compliance, rather than co-option.

People could be turned into machines, a consequence of political thinking in the emergence of industrial society and the rush to efficiency and cost-allocation. Once properly mechanised, they could be ‘trusted’ – the scare quotes, because the trusted prisoner is no longer human. A big part of that process is surveillance: once people know that they are always (potentially) watched, they’re a bit more compliant to the rules, and a bit more like machines.

Actually, Ian turns from melancholy to fighting talk. Where is the transgression, he asks? What passes for subversion online is often just prnaksterism, often funded to, in small feats of legerdemain to slip in a flash of brand in front of the viewer.

The echo chamber is another danger in all of this, Ian says. Where are the racists in his network?:

Racists are poised to take Stoke in the next by-election. They don’t appear on my spectrum because I have deliberately blinded myself to their existence on a day-to-day basis. Diversity of opinion is purely opt-in (with strong incentives to opt-out) in socialmediaworld.

Add some racists to your feed list? I don’t know about racists, but I enjoy having different views on hand in my inbox. I detest a great deal of what some political bloggers say, but I like to try and understand. Sometimes I have had my mind changed too. I understand people on the right (OK, mainly the centre right) much better than I did when I was a pre-web student. Then I used to sneer at people for reading the Telegraph for goodness sake. Now I’ll read it’s leaders and blog posts alongside Comment is Free and the Guardian.

I’ll unsubscribe because people are boring, not because I disagree. Maybe that’s just me. And maybe I need to listen more to some Green voices, some far right voices, some Socialist Workers Party voices.

All is not lost, I say. Fight on…This world is still ours to shape, perhaps as never before. We’re right to identify these pitfalls and blind alleys, but nothing is inevitable in all of this. There’s still a revolution to be had.

After we’ve read these warnings, go and read some Umair Haque manifesto. Then think about what you will do this year to change the world. Seriously.

Point is: there’s a lot at stake.

Typograhic history of the City of London

I’m posting a lot of videos recently. Maybe I should be setting up a Tumblr blog for this sort of thing. Maybe not.

Anyway, this is a gem: combining three of my interests: typography, London and history, it’s a short film by Michael Bokkowski at Linefeed:

A Typographic Survey of the City of London from Michael Bojkowski on Vimeo.

Definitely worth 15 minutes…

Via Peter Parkes

Welcome to Mars: electronica/documentary hybrid

Image: Cover of the Welcome to Mars soundtrack CD
Image: Cover of the Welcome to Mars soundtrack CD

One of my favourite things that I have – at least digitally – is a podcast series in twelve parts called Welcome to Mars. A work of pure genius, it combines a series of unscripted monologues by Ken Hollings with some giddying, insanely inspired synth music by Simon James.

It says it is about “the fantasy of science at the beginning of the American Century”. That doesn’t tell the half of it…

Let me list the reasons I love it:

  • American history: First off the history is facinating, and I’m a sucker for 20th century history. It’s a stroll, and then a scamper and lurk in the shadows of the post-World War II period, the beginning of the “Amercian Century” as some call it. We hear about the industrial-military complex’s evolution, the growth of suburbia, paranoia about Sputnik. These are all things I’ve heard about them before, studied at university in fact, but Hollings presents them afresh, in a new, very odd context. He brings alive the uncertainty and fear that charcterised those times, when it is tempting to see the advance of the United States as a confident march into the space age future that never came…
  • Weird coincidences and neo-conspiracies: Hollings throws you off-guard by connecting dates, people, organisations in ways you weren’t expecting. OK, we’re familiar with Levittowns and consumer culture, by what’s Alastair Crowley doing in the mix trying to score acid in the 50s? He never wanders in loonyville conspiracy theorising, but you feel spooked nonetheless. It reminds me of James Ellroy’s last two books American Tabloid and The Cold Six Thousand, which deal with, respectively, the period before and after the Kennedy assassination. Even though there may be some artistic licence at work, the most unsettling facts, and the overall mood, are utterly authentic.
  • Retro Sci Fi: There’s always something delicious – and insightful – about the past’s visions of the future. This is about my Father’s – even my Grandfathers’, though I’m not sure wither of them had a taste for it –  science fiction, the 40s, 50s. The times when, even over here in the UK, there was such a strong confidence in the momentum of the space technology that a Navy officer would predict that the children he was speaking to might one day command star-ships in Her Majesty’s Galactic Fleet.
  • It’s unlike anything else I’ve ever heard: It’s not a history exactly, but it’s far from fiction. This a compelling piece of musing, provocation, art. I have had it on my iPod since last Autumn and I keep going back to it when I’m in the right mood. I listen to it on my own, often when I’m travelling, as it has a great distracting, cocooning quality to it. It feels like this is a new, hybrid art-form that works peculiarly well as a podcast. And, er, did I mention it was free?
  • Insanely good electronic music: Yeah, Simon James is a genius. The music, evocative of Sci Fi films of the period adds so, so much to the experience. As well as playing as background to Hollings’ monologue, there are interludes that are amazing to listen to in their own right. And, ad fans out there will appreciate the spooky re-mixes of 40s and 50s ads for American consumer products.

So there you go. If you want to try it yourself, it’s free to get the Welcome to Mars podcast. You can also check out The Simonsound Simon James’s website, buy the soundtrack and Ken Hollings‘s blog for more related wonderfulness.

Image: The Simonsound's album cover makes use of retro space age images
Image: The Simonsound's album cover makes use of retro space age images