ADD2323 Horizon 2020 programme — London City DataStore Development

Type of decision: 
Assistant Director's decision
Date signed: 
04 March 2019
Decision by: 
Jeremy Skinner, Assistant Director of Strategy, Insight & Intelligence

Executive summary

MD1574 approved the GLA’s acceptance of a grant award of €24,988,759 from the European Commission and expenditure of the same, over for the period January 2016 to January 2020, for the coordination and delivery of the ‘Sharing Cities’ programme. €2.4m of funding was awarded for specific GLA activities, including €50,000 (approximately £44,500) for hosting of the ‘Urban Sharing’ Data Platform in London.

This ADD asks for approval to spend up to £25,000 from this £44,500 for development and hosting of the London City DataStore.


That the Interim Assistant Director of City Intelligence Unit approves expenditure of up to £25,000 on the services of DataPress to provide secure data sharing services.

Part 1: Non-confidential facts and advice

Introduction and background

In September 2015 the European Commission (EC) awarded the GLA and a consortium of 34 European partners a grant award to deliver the Horizon 2020 (H2020) Smart Cities and Communities “Lighthouse Project” programme. This five-year programme supports the implementation and integration of near-to-market smart city technologies and systems, to maximise their benefit in demonstrator areas and prove their replicability across wider city environments.

The project aims to develop, deploy and integrate replicable solutions in the energy, transport, data and community sectors in each of the three lead cities of London, Milan, and Lisbon. A further three ‘fellow’ cities - Burgas, Bordeaux and Warsaw are actively involved with the three lead cities to validate city service designs and implement a number of specific solutions

A total of €24,988,759 has been awarded by the European Commission to the consortium with €2.4m of funding awarded for specific GLA activities, €2.7m to the Royal Borough of Greenwich (RBG) and €1.3m to other London partners.

The GLA role in Work Package 4 of Sharing Cities is concerned with the development of an ‘Urban Sharing Platform’ (USP) in each of the lead cities (Milan, Lisbon and Greenwich). These will allow the secure sharing of urban data for analysis by Cities across different systems and between different organisations. Analytics will be used to inform the development of policy and also to improve service delivery. Crucially, these platforms will be able to access data from the emerging networks of sensors (Internet of Things) and also support data marketplaces through Application Programming Interfaces (API)s.

In London, where public services are delivered by a wide range of organisations, much of the challenge is to connect securely between different public-sector systems and the GLA is leading on the development of an Urban Sharing Platform (City DataStore). The City DataStore is a cloud-based system to store data (and information about it), allow data owners to share specific datasets with chosen individuals (or groups) and allow secure connections with other data stores (through APIs). This builds on our previous experience of sharing open data through the London DataStore. We are working with Greenwich as a pilot but are also working with other public organisations across London, including LLDC, LEB and TfL — and other Boroughs through the Borough Data Partnership.

Under Work Package 4 of Sharing Cities, €50,000 (approximately £44,500) is available for hosting of the ‘Urban Sharing’ Data Platform in London. Data Press was recruited to deliver this service, following an invitation to tender conducted in August 2018. This ADD asks for approval to spend up to £25,000 for development and hosting activity from this £44,500.

The GLA has been developing a longer-term specification for the Urban Sharing Platform, with ambitious features. This will require significant development work over the remaining 2 years of Sharing Cities and will be delivered by the recruitment of two full time developers, funded through Sharing Cities.

However, we also have a short / medium term requirement for ‘front end’ features to specifically support the Greenwich pilot and scale up this work. Engaging with the Data Analytics Programme within the London Office of Technology & Innovation will enable the GLA to deliver these commitments.

Objectives and expected outcomes

The City DataStore will be delivered by developing additional features for the existing London DataStore, in particular, the following front-end features will be available to users.

Upload files - Users need to be able to upload files from their local drives (and ideal groups of files) without needing to install extra software. Users need to be able to manage their own files including:

- Permanently deleting, including all backups and copies (with the appropriate warnings); and

- Replacing whole files with an updated version.

Users also need to be able to link to files that are physically stored elsewhere. It is recognised that there may be performance trade-offs depending on the systems being connected to. The link may take the form of a URL or a call to an API on the other system.

Metadata - One of the key concepts of the City DataStore is that metadata is available for all datasets:

- This reassures potential publishers that their data will not be inadvertently misused;

- It ensures that users know the limitations and licencing details for the data;

- The metadata fields are also crucial to data discovery, including search and filter functions described below; and

- Automatic population of basic fields, such as file type, update date, etc.

NOTE: there should be the facility to create metadata regardless of whether the actual files are stored in the DataStore or not. The reason for this is to be able to build up a searchable catalogue of the key public sector datasets even if they are physically held by numerous organisations. The default option will be to use the standard fields that have been developed for the London DataStore.

Sharing – The default for uploaded data should be that it is only visible to the owner. They should then be able to share either just the metadata or the data & metadata at any time (or unshare). Sharing can be with individual users or groups of users (for instance everyone in a department) and the data owner should be able to see an overview of group members. The data owner should be able to access summary stats about who has accessed each of their datasets, frequency and route to it (i.e. what was the search term used).

This is an online system and access will be available immediately on payment.

Equality comments

The programme’s aims of introducing innovations in technology and improving the use of data are presenting new ways of meeting peoples' needs in urban landscapes. These technologies have the potential to transform the efficiency of city operations; city-level decision making and the relationships with people and businesses to improve sustainability, while at the same time addressing resource pressures faced by communities and authorities.

The programme supports the Mayor’s Equality, Diversity and Inclusion (EDI) Strategy by contributing towards an enriched quality of life for the socially disadvantaged – the sector of society with a greater representation of people with protected characteristics. Smart-parking, self-driving cars and e-bike services can specifically support people with certain disabilities, while access to shared vehicles can increase mobility for those on low incomes. An increased emphasis on technological solutions has positive impacts on people with protected characteristics enhancing the community experience. Green buildings and improved infrastructure will cut carbon emissions, and alongside information about NOx emissions will improve health outcomes for Londoners currently dwelling in the most polluted parts of the city. The programme has dedicated work packages for communication and engagement to ensure that people from all age groups and ethnic backgrounds are encouraged to participate in these opportunities.

Other considerations

a) Key Risks:

Compatibility with other data stores — The City DataStore has been designed with open APIs itself and will be able to share with other data stores that use one of the established web standards.

London development times, stability of the platform, ongoing support — The City DataStore is being developed with the agile methodology to ensure that the most important functions are delivered first the platform will be hosted on AWS cloud servers which have extremely high ‘up times’, the platform can be transferred to other hosting arrangements to ensure its ongoing availability

The secure transfer of data also underpins a number of important GLA projects including:

- London Office of Technology and Innovation; and

- London Office for Data Analytics (exchange of data with Boroughs and GLA family).

The lack of a private twin to the open data store has affected the operation of a number of projects and not completing this procurement will have a significant impact on the Intelligence Unit’s capacity to work in partnership with other London public organisations

Consultation on functional requirements has taken place across the Intelligence Unit and more widely across the GLA using structured workshops with key users.

Financial comments

The expenditure of up to £25,000 will be funded from the 2018-19 H2020 Programme budget within the Intelligence Unit, where a full reimbursement of costs will be submitted and reclaimed from the European Commission (EC).

Activity table



Procurement of contract

Sept 2018

Delivery Start Date

December 2018

Delivery End Date

January 2019

Project Closure

February 2019

Share this page