Thursday 16 September 2010

Will you be ready for the data tsunami?

Horribly early the other Saturday morning, I blearily Boris-biked through a completely deserted Fitzrovia to attend Opentech 2010.  It's a refreshingly low-budget (think £5 entry, write-your-own namebadge, one-microphone-between-seven ) low hype conference, which aims to connect various types of geeks interested in how to change the world through data and technology.

This year a big catalyst for discussion was the Coalition Agreement's pledge to create a public "right to data" .  This should be a massive step forward in transparency from the previous government's FOI legislation.  All kinds of public sector data: ranging from procurement contract details to service usage and performance data - should become available to the public for the first time over the next six months.  What's more, the government has promised that our data will be published in a way that will make it clean, comparable, linkable, and above all, useful.  At which point, an army of geeks, entrepreneurs and journalists will move in to analyse the data and ultimately, make money educating, entertaining and alarming the nation with the results.

That's the theory, anyway.

If we lived in France, possibly the most bureaucratic, systematic, hydromatic nation on the planet, this might not be such a challenge.  The French Government has been collecting metric tonnes of data in centrally prescribed, standardised format for years - although there is an equally consistent complete absence of any political will to disclose it.    But  - fortunately or unfortunately, depending on your perspective - our public sector in the UK is a devolved, highly fragmented collection of organisations.    Just on the procurement front alone there are over 44,000 public sector organisations in England that buy things.  All of them record their spend differently, using different software, different formats, different categorisations, different levels of detail: even different names for the same supplier.

"If you build it, he will come"


The task of addressing the mind-bogglingly enormous problem of how to link all this data together has fallen to an admirable group of digital pioneers.  Presenting at Opentech on Saturday, John Sheridan and Jen Tennison of the Cabinet Office data.gov.uk team explained their technical approach to cleaning up submitted data.   The team's hope, as with many of the other presentations at Opentech, is to encourage and inspire geeks to go forth and develop the APIs and applications that will help us all make sense of the gigantic data tsunami soon due to hit the UK.

And they will succeed.  That is to say, spend analysis is a multi million pound industry in the UK.  The same people who have historically made good livings from being given a council's own spend data and selling it back to them in a unique proprietary format have a very strong incentive to find new ways of making money from the right to data.  There are also true transparency champions like Chris Taggart (@countculture) whose site Openly Local already contains partial data from 158 local authorities, and  Hadley Beeman (@hadleybeeman) of LinkedGov.Org


What is of far greater concern - which I sincerely hope is being addressed very seriously by Cabinet Office and DCLG  - is collecting the data in the first place. Yes, data submission deadlines have been set by the Government  - autumn this year for most central government spend data, and January 2011 for local government.   But at present, these deadlines look  hopelessly optimistic.  Many PSOs don't even know themselves what they are buying, who they are buying it from, and how much they are paying - let alone be in a position to report it all to Francis Maude in correct standardised format by Thursday next please.  Small PSOs have never had the technical expertise, and post-Spending Review in November, no PSO will have the time or the resources to prioritise the difficult exercise of identifying, extracting and cleaning up data.


Most importantly - having worked with many local authorities, and regularly witnessed the trials of colleagues in OGC attempting to gather spend data,  I know how difficult it is to impose commands from the center onto the wider public sector.  Even an Eric Pickles shaped transparency stick cannot beat complete clean datasets out of a council if there is no serious will locally.  While PR and marketing has spectacularly fallen out of vogue under the Coalition, there's a very pressing case to direct whatever comms resources are available to selling this to devolved PSOs.  Councils need evidence that extra-organisational linked data can deliver local hard cash savings in order to commit fully - just as they are now beginning to recognise that extra-organisational collaborative procurement will also deliver cash savings.  In both cases, they want to see the gain for their pain.

Thursday 9 September 2010

CIPS awards

An innovative project I've been working on since 2007 for the London Fire Brigade and Emergency Planning Authority and Capital Ambition (aka the 32 London councils) was recently nominated for a prestigious CIPS award.    Naturally, the taxpayer can't be expected to fund our attendance at "lavish champagne-sipping" awards dinners at "expensive Mayfair hotels", but by an incredible stroke of Cinderella-like luck a CIPS fairy godmother arranged for us to be invited to last night's ceremony as the guests of key sponsors Bravo Solutions.   The Bravo guys there on the night clearly couldn't work out who we were or why we had been invited but proved most jovial and generous hosts - both at the ceremony and later at their Cuckoo Club after party.

Christine Morton (Capital Ambition)

Other people (such as the eminent Peter Smith) have blogged elsewhere about the winners and losers   - suffice to say we were beaten to the title of Best Supplier Relationship project by Premier Foods/British Sugar who went on to take the overall prize - but it was clear we'd done very well to be the only public sector project to win a nomination in our category.  In fact, there were surprisingly few public sector projects nominated in any open category:  it will be interesting to see if that's still the case next year, when spending cuts will have forced previously change-resistant public sector organisations to become much more radical in  their approach to procurement.

Having said that, it was inspiring to meet a nominee in the public sector category, Mike Melville, programme manager of xchangeWales, who has achieved millions of pounds of savings by moving all 22 local authorities of Wales onto e-procurement and more closely integrated purchasing  - and who frankly deserved to have won his category hands down (judges, are you listening?)    Mike proved his determination and stamina  is not just confined to driving through procurement change, outlasting virtually every other Bravo client and guest at the after party....

Mike Melville
  
Rory Bremner had the dubious honour of hosting the awards. He did a good job keeping things moving, but it was interesting to notice that his act is getting quite dated: he produced excellent Michael Howard, Ken Clark, Blunkett, Hague, Blair, Prescott and Bush - and had a good shot at Vince Cable - but he isn't doing Cameron, Clegg,  or any of the Milibonks or Eds.   As a 40 something ex-public-schoolboy himself - is he just too similiar to Cameron and Clegg to be able to do a decent impression of them?

Rory Bremner - can't remember who he was doing at this point!



Gavin White, Business process lead at Pfizer
Powered By Blogger