TfGM reinforced its position with regard to open data at the meeting. There has been a number of rumours over the past twelve months as to what the organisation was trying to release to DataGM – Greater Manchester’s open data portal . TfGM are currently releasing data with regard to bus schedules, NaPTAN stop locations, fixed and mobile speed camera locations and monthly Road Traffic Collision updates. There had been mooted some realtime data would be released.

Greater Manchester has been crying out for an intelligent integrated ticketing system. To many a lack of such system has made travel by public transport around Greater Manchester more difficult than it should be. To this end TfGM are developing a specification that will go to tender in the 1st half of 2012. The system will initially cover Metrolink and then encompass Greater Manchester buses. The system will use contactless technologies in a similar vein to TfL’s Oyster Card but with the added functionality of being able to use contactless bankcards and NFC phones. It was interesting to note the certainty that NFC will be adopted, by most handset companies within the next year. Paying by Google Wallet was also mentioned as a possibility. The ticketing system will also have fare rules that will calculate the best price for journeys undertaken.

Although getting Integrated ticketing to work with Metrolink would be a relatively easy task and a useful test bed to prove the utility of the system, getting Greater Manchester’s 40+ independent commercial bus operators to adopt the system maybe more challenging and may need a certain amount of political will. Anonymised journey data from the system or personal access to journey history wasn’t discussed in detail, although the later seems to be fairly standard in smart ticketing systems, access to anonymised data could offer huge potential for applications and services that look at gate loading on routes, passenger density etc.

The advent of the oft mooted, realtime data from TfGM looks closer – although there was no specific timescale mentioned. There will be access to the Metrolink Passenger Information Displays data, although how this will manifest itself is uncertain. Developers present at the meeting suggested that JSON would be preferable. The main challenge with accessing real-time Metrolink location data is that the Tram Management System currently being implemented isn’t currently functioning throughout the network. The initial release of data will cover the South Manchester line and Eccles lines.

Although it doesn’t look like there will be any real-time bus data soon, TfGM would like to release the location information of the free Centreline buses that are being operated on TfGM’s behalf. This data will be location data that won’t identify the actual service the bus is running. It was suggested that as there are only three distinct Centreline routes it wouldn’t be that complicated to identify, even where the routes overlap. There is also an Informed Personal Traveller pilot that is being run in Bury by Logica, ACIS and First Bus. It uses a number of technologies including an AVL system that has been fitted to approximately 100 of their buses. The IPT application hasn’t been released yet and there are indications that the system is closed.

TfGM recently submitted a bid to the Local Sustainable Transport Fund and written into it is the provision of open data and the development of an intelligent multi-modal Journey Planner pulling all relevant data that TfGM has at it’s disposal, how developers could access the Journey Planner was discussed and whether it would exclude the provision of other types of journey data.

There is a move to make other data available through the LSTF, these include Car Park updates, real-time disruption data, journey down roads data and feeds off TfGM’s SCOOT adaptive traffic control system. SCOOT controls half of the approximately 2000 traffic control signals in Greater Manchester.

The lack of transparency with regard to bus fare structures within Greater Manchester has been a subject that has come up many times, especially regarding anecdotal evidence that dependant communities are charged more per mile than others having viable transport alternatives. TfGM stated that Greater Manchester is one of the few places where bus travel is generally more expensive than rail. To this end TfGM are interested in developing a project similar to one that Open Data Manchester was developing over a year ago that encouraged travelers to submit the details of their journey and price.

At the close of the discussion TfGM were encouraged to use the Open Data Manchester Google Group as a resource to ask questions and to highlight initiatives and challenges.

These responses will be used to define the governments approach to Open Data and will hopefully bring about a meaningful push from both central and local government.

The following Greater Manchester based organisations responded:

So far play to the above for the above for engaging.

Summaries of the consultation can be read here:

• Making Open Data Real consultation – summary of responses [pdf]
• Making Open Data Real consultation – summary of responses [odt]
• Making Open Data Real consultation – summary of responses [Word]

Full responses can be downloaded here:

• Making Open Data Real consultation – full responses [zip]

The SOPA and PIPA legislation currently going through the American congress has impact and ramifications far beyond the shores of the US and piracy, and is a threat to the Free and Open internet that we have today. It iS a blunt, badly constructed piece of legislation that seeks to prop up an industry based on old business models by clamping down on new and innovative ways of producing and disseminating content. As with most bad pieces of legislation it will be open to creep and opportunism, where arbitrary take down and closure of sites pointing to or serving content would prevail. It offers a world where the winners will be the established players with large legal teams and vested interests and the losers will be everyone else. It is an outcome which we, as part of the Open movement, are fighting against.

What could it do?

Would it effect Open Data? We don’t know, as open data is made available by data owners under open licensing. It might constrain the publishing of data that has been scraped, but the legal arguments as to whether data is protected by copyright and IP law are complex.

It could constrain search, if you fall foul of its broad and ill-defined terms. You may have a site but it might as well not exist if it doesn’t show up in search results.

It can effect free expression – just imagine the internet awash with copyright bots who can automatically close down or de link sites with out any recourse to the law. You can get a flavour of this on on some sites that have content sniffing technology.

Who decides what is or isn’t copyrighted or IP protected material? In the SOPA world you will be found guilty and then have to fight to prove your innocence. How will this effect the ability to expose wrongdoing if the evidence is subject to SOPA and PIPA?

Many people have written about why these laws are a bad thing and some links are below. From the UK we can only watch and hope.

Technical overview

Why SOPA is bad for business

Guardian article on effect

Added 19/1/12
Excellent article on Al-Jazeera by Jonathan Zitrain et al.

Update 19/1/12

Last night several members of congress, who originally supported the SOPA/PIPA bill withdrew their support due to the massive international campaign that saw many thousands of websites, including Open Data Manchester’s, go dark and people ‘block the switchboards’ of the American congress. It was a victory in the battle to protect our free and open internet but there is too much money at stake and we are sure we will see a revised, more subtle form of SOPA/PIPA in the not too distant future.

Paul Gallagher – Head of Online for Manchester Evening News gave a presentation regarding the role of the MEN during the Manchester Riots. He described how the Manchester Evening News had used Social Media during the riots and how his team had started to collect data regarding the riots and the subsequent court cases to give insight into some of the possible causes of the riots.

Most interesting was the resources that the MEN had put into reporting on the court cases following the riots and by having court reporters sitting in on each of the trials they created a schema and dataset show the areas that people lived in, mitigating circumstances, age, type of offence, sentence etc. This is data that can only be created if you attend the trail. This allowed them to map offences against depravation indices and changes in the way that sentencing was delivered over the course of the trials.

The discussion also touched on news organisations becoming huge archives of sentencing data and how this can effect people’s lives even after their convictions have been struck off. MEN does have a policy where certain details are redacted from the historical archive but this is done on a case by case basis.

There was also an update as to the preparations for the International Open Data Hackday and the responses to the Governments Open Data and Public Data Corporation consultations.

Open Data Cities was initiated in May 2009, premised on the simple question of how cities would evolve if all data were made open. Would the same inequalities and asymmetries persist for example? Moreover, what would need to happen within the city to bring about the adoption of more open and transparent practices?

Greater Manchester is a region in the North West of England with a population of 2.8 million people. It comprises of 10 boroughs containing two cities and many large towns. Open Data Cities approached the city as a functioning organism comprising of these 10 boroughs. For the project to have a genuine impact with its inhabitants, we proposed that the project would need to align with how people used the city rather than the ways in which the city was administered. The reality within the city is that although people access services across authorities and whilst there are a number of pan-Greater Manchester public bodies, local authorities still deliver services to tight geographical boundaries.

Addressing the whole Greater Manchester region in this way, created an environment that allowed the project to evolve in a particular way. As the region was adopting City Region status this would require a certain alignment in terms of data and information. The granting of City Region status also opened up the possibility of bringing about an elected mayor, enabling, theoretically, a coherent region-wide strategy to be implemented.

Working across the ten boroughs – all with their own democratically elected councils is not without its challenges. Each public body has its own administrative and data structure and specific set of difficulties. It was therefore necessary to adopt a pragmatic, non-threatening approach as part of our project. Conversations therefore centered around the idea of allowing citizens to look ‘under the hood’ of public service so to speak, of creating better understanding of what councils do. Most importantly we were interested in rebalancing the relationship between public service and citizen and the possibility for services to be delivered with citizens rather than simply to citizens.

Communicating The Benefits

We were often challenged as to how the release of data would benefit the person on the street and who would create the applications and interpretations to allow this to happen. At the start of the Open Data Cities project the Open Data Manchester community was formed to provide evidence that there was indeed a ‘demand’ for the release of open data within the region. We argued that by giving people the tools to understand and act within communities, open data would have broader benefits too. Moreover, there was a growing acceptance that enabling people to access the data and information relevant to their locality was important. This in part has been born out by the emergence of hyperlocal blogging as a means of disseminating news and information at a community level.

Open Data Cities also strongly emphasised the innovation and economic benefits such open data could bring to the region. Opening up the ‘undiscovered country’ of open data, could kick start an economy based on the creation of data services. We had seen examples where companies such as Elbatrop software in London had created best selling applications for San Francisco based on released tree data. If Greater Manchester released data this could present an opportunity for developers to create applications that could have relevance beyond the Greater Manchester region. Research had identified that open data could add £6 billion of added value to the UK economy, how much of that value could be injected into the regional economy?

High value, ‘quality of life’ datasets were identified. Greater Manchester Passenger Transport Executive now TfGM, made the decision to release large and regularly updated datasets. This sparked a number of good applications but most of them were ‘proof of concept’ with little that could really be considered ready for market. This wasn’t the ‘release the data and people will build cool stuff’ future that we had been promoting, and even though the transport authority had now committed to making data open as a default position, they were very aware that not much was being built.

Acknowledging the Barriers

By talking to people who were involved in Open Data Manchester and the wider Greater Manchester digital community, it became apparent that although open data offered opportunities, there were a number of significant barriers that were inhibiting the development of services. These could be seen as return on investment, risk and liability.

The return on investment argument was quite apparent from early on. People have to make a living and generally want to see their efforts rewarded. By Open Data Cities embracing the 2.8 million people of Greater Manchester it was hoped that there would be enough people to sustain a market in Open Data application development. In order to kickstart this market it was proposed that a number of innovation challenges with sizeable incentives should take place.

It was obvious that there were no large digital businesses in the open data space and we had long held the view that their presence would be an indicator of the health of the open data innovation ecosystem. A suggested reason for the scarcity being that open data licensing was transferring all the risk on to the developer, whereas previously data would be generally released with some sort of service level agreement, none of these guarantees exist with open licensing. The idea of spending large amounts of development time on applications built on data that could then be turned off was deemed too risky.

Liability was also an issue. Who would be liable if someone had bought an application where the data was suddenly turned off or were inaccurate? There were also concerns as to the robustness of supplied data and the sometimes, archaic formats data were supplied in. The liability argument was also been put forward as a ‘supply side’ reason for non-disclosure both from a robustness of data and command and control perspective.

Collaboration

When FutureEverything and Trafford Council began working together on DataGM – The Greater Manchester Datastore, many of the local authorities were in a state of panic through having to negotiate the drastic shortfalls in budget. It was becoming apparent that innovation and citizen empowerment, although appealing were the least of concerns. Public bodies are still in a time of fiscal stress and it has been stated that few, if any, public bodies innovate out of a crisis.

All Greater Manchester local authorities and most pan-Greater Manchester public bodies are represented on the datastore steering group – The benefit of having a local authority leading the project, is their ability to get people around the table. Whilst some members of the group understood the logic of having a datastore and shared intelligence, there was a lot of resistance. Members stated despair at being involved with a project where they didn’t know if they would still be in post in three months time, with others not seeing the point of spending time and resource on something that didn’t have concrete output. There was also a very tangible silo mentality where the idea of shared intelligence across authorities was seen as attractive but not essential.

Evidence and Evolution

As the DataGM project gathered momentum more evidence started to emerge as to the inefficiencies of maintaining a siloed and closed data culture. The servicing of Freedom of information requests costs Greater Manchester public bodies over £4 million a year, over 600 public officials a day are unable to find or use data that they require in order to carry out their jobs – costing authorities over £8.5 million a year. The annoying tendency – for public bodies – of citizens using services outside their borough boundaries also creates difficulties. With no pan – Greater Manchester data initiative it is difficult for public bodies to create and deliver on coherent regional strategies. Open data offers a solution.

Now DataGM is becoming established the economic logic of using a centralised data catalogue, where the data that local authorities use themselves is openly available, is starting to make sense. Open data needs to be transformational. For public bodies enhanced engagement and the creation of innovative services are not enough. We are at a stage where we are saying if you spend A you will get savings of B and with open data you will also gain benefits of C, D, E…

DataGM is starting to develop data release schedules so that local authorities can release similar data in a coordinated way. With developers such as Swrrl – one of the recent winners of the EU Open Data Challenge, some of that data is being expressed as Linked Data. The Open Data Manchester community continues to grow. Although there is still a long way to go with open data in Manchester it feels like more people within public service are starting to see the benefits, and the possibility of Greater Manchester becoming an Open Data City gets closer.

I’m no expert but I really need to be – Licensing

Licensing is a subject that comes up a lot with Open Data. The licence is a key component of the dataset. It defines the use and liability and it shapes how or what innovation will come from data release.

As mentioned in the title I am no expert in this area and I would appreciate any correction or amendments to my understanding.

Traditionally public data has been closed so that the only way you could get access to data to build products was by buying a licence to use. In many cases these licences were expensive and restrictive. The to mitigate this cost often, te licence would also have some level of service agreement built in. You paid for the licence for the data and the data provider would provide you with a level of continuity and support. This helps to limit risk and encourage investment into a product.

The closed ‘paid licence’ system generally has a high barrier to entry ‘price of licence’ limiting the amount of innovative products developed. If innovation ecosystems are ideas that live with most failing. The price of failure being too high could have a chilling effect on the whole system.

One of the first licenses used for the release of Open Data was Creative Commons CC-BY-SA. This licence allowed people to create services and products off the back of the data as long as they attribute where the data came from and share back any data that was created off the back off the originally released dataset (value-added data). The original Creative Commons licenses were devised as an answer to restrictive copyright laws relating to ‘works’ – articles, text, images, music etc., as these were deemed increasingly anachronistic in the digital age. It is up for discussion if data can be deemed as a ‘work’ in the context of this licence.

The Open Database Licence (ODbL) developed by Open Data Commons, was created to address the doubt that data could be seen as a ‘work’. It carries the same attribution and share alike clauses and is used by many datastores including the newly opened Paris Datastore.

Anyone can develop products and services that use datasets with these licences but intellectual property doesn’t extend to the value-added datasets created in the process of developing these products. Releasing value-added datasets back to the community allows further innovative products to be released off the back of these datasets, so potentially the pace of innovation could be increased – It is analogous to the ‘standing on the shoulders of giants’ idea.

By imposing further use of value-added data by other organisations might chill the development of products that create value-added data.

With the above licences there is generally no liability or guarantee of service from data providers. This creates a greater risk scenario. If you were investing in product development this potentially is a source of concern and may be an inhibiting factor

In the UK we have the recently released Open Government Data Licence. That was developed specifically for government data. It borrows from some aspects of the CC-BY-SA licence and ODbL. Unlike the those licences there is no need to share back value-added data.

Would this have any impact on products and services that are developed from Open Data? Again in the licence there is no liability or guarantee of service from the data provider but the developing organisation gets to keep all the rights on the products and services they develop – including value-added datasets.
The advantage of this could be that by allowing people to keep hold of the rights to the products that they develop might be mitigate against the exposed risk posed by the lack of liability and guarantee. The main disadvantage could be that the pace of innovation could be curtailed due to people having to replicate process and value-added datasets.

At the time there were few cities who had embarked down the route of fully opening up their datasets although some cities in North America had started a process that would eventually, as in the case of Vancouver, lead to an adoption of open source, open standards and open data principles.

It was through seeing this emergence of open systems that the Open Data City project began to evolve. Data is is the lifeblood of our modern technologised society. It tracks, evidences and creates mechanisms for decisions. Much of this data doesn’t exist outside the confines of City Hall but we see evidence of the impact of this data everyday. Speed humps suddenly appear on your road or your bus doesn’t turn up when you thought it would. Bins only get emptied every two weeks or your local school closes down. This is the physical manifestation of the publicly held data that few have access to.

The inability to connect action taken by a public body with the evidence on which the decisions are made can have an insidious and corrosive effect on the relationship between the citizenry and government. Just as Louis Brandeis said ‘Sunlight is the best disinfectant’ with regards to transparency and corruption, the opposite is also true. In a closed system even though the decisions might be taken with the most honourable of intentions, the lack of evidence for the decision creates doubt, rumour and misrepresentation. In a closed system the power of the media increases as the distrust of the political sphere decreases. The media becomes the interlocutor and which can interfere with the relationship between citizen and government. This all presumes that those that govern have nothing to hide. The lack of transparency in government creates the opportunity for the media to expose the bad apples using a system of clandestine briefings and investigative reporting. This process of exposé undermines the trust the public has in the system of government because there is no evidence to the contrary or that the evidence that people can see has been derived from a seemingly arbitrary decision making process.

The opportunity has arisen for public bodies to create a new relationship with the people who they serve. A more transparent and open system can lead to a more equitable environment, where the citizen is not a customer or passive consumer of service and information, but an engaged citizen who is able to make decisions based upon facts, not rumour and can hold to account public servants with less than honest intentions.

The Sunlight Foundation www.sunlightfoundation.com, named after the Louis Brandeis quote, are an American lobby group advocating transparency in government. They have produced this graphic which they call the Cycle of Transparency which aptly illustrates the benefits of transparency in government. As each element of the Cycle of Transparency moves forward concurrently, bringing about the changes needed to create a more transparent government whilst identifying new needs.

The Cycle Of Transparency highlights the use of technology to make information open and accessible. It can be argued that transparency and openness has been enabled by digital technology. People are now able to access, interpret and distribute information easily. Until quite recently, the channels for making information open and accessible where limited and to a certain extent controlled.

The landscape is changing. The opening up of data will have a seismic effect on the way we access and share information. New services will be created, as citizens and institutions demand the ability to interpret and navigate through data in the way they want. It will create a more efficient data environment where information is shared rather than duplicated, and it will highlight errors in the system with anomalies being addressed rather than hidden.