Recommendations from the Productivity Commission


Building a reformed data system.
Recommendation Description
5.1

Consumer data must be provided on request to consumers or directly to a designated third party in order to exercise a number of rights, summarised as the Comprehensive Right to access and use digital data. This Comprehensive Right would enable consumers to:

  • share in perpetuity joint access to and use of their consumer data with the data holder
  • receive a copy of their consumer data
  • request edits or corrections to it for reasons of accuracy
  • be informed of the trade or other disclosure of consumer data to third parties
  • direct data holders to transfer data in machine-readable form, either to the individual or to a nominated third party.

Where a transfer is requested outside of an industry (such as from a medical service provider to an insurance provider) and the agreed scope of consumer data is different in the source industry and the destination industry, the scope that applies would be that of the data sender.

5.2

The Australian Government should introduce an outcome-based definition of consumer data that is, as an overarching objective, data that is sufficient to enable the provision of a competing or complementary service or product for a consumer.

In the relevant service or product context, consumer data is digital data, provided in machine-readable format, that is:

  • held by a product or service provider
  • identified with a consumer
  • associated with a product or service provided to that consumer.

Participants in an industry should determine the scope of consumer data relevant to their industry (where an industry in this context would be determined by a broad description of the service). This should be in the form of a data-specification agreement.

Data-specification agreements should also articulate: transfer mechanisms, and security of data, to ensure that data use is practical and robust to technology updates; and the requirements necessary to authenticate a consumer request prior to any transfer.

These agreements should be registered with the ACCC, which may offer interim approval where an agreement has been reached but other industry agreements have been prioritised for approval.

In the absence of such agreement, consumer data must be in machine-readable form and include all of:

  • personal information, as defined in the Privacy Act 1988 (Cth), that is in digital form
  • information posted online by the consumer
  • data created from consumers’ online transactions, Internet-connected activity, or digital devices
  • data purchased or obtained from a third party that is about the identified consumer
  • other data associated with transactions or activity that is relevant to the transfer of data to a nominated third party.

Data that is solely imputed by a data holder to be about a consumer may only be included with industry-negotiated agreement. Data that is collected for security purposes or is subject to intellectual property rights would be excluded from consumer data.

A consumer for the purposes of consumer data should include a natural person and an ABN holder with a turnover of less than $3m pa in the most recent financial year.

Data that is not able to be re-identified to a consumer in the normal course of business within a data holder should not be considered consumer data.

The definition should be included in a new Act for data sharing and release (recommendation 8.1). Given the need for consumer data to have broad applicability, the outer boundary definition and reference to ACCC registered industry-specific definitions should also be included within the Acts Interpretation Act 1901 (Cth). Consequential amendments to other legislation in the future would ensure harmonisation across federal laws.

5.3

All holders of consumer data should include in their privacy policies, terms and conditions, or on their websites a list of parties to whom consumer data has been traded or otherwise disclosed over the past 12 months.

On the windup of an entity that holds consumer data, consumers should be informed if data to which they hold a joint right has been traded or transferred to another entity. For businesses entering formal insolvency processes, insolvency practitioners should ensure consumers have been informed. For businesses closing but not in insolvency proceedings, the entity acquiring consumer data should inform consumers of this fact and give them the opportunity for data collection to cease.

5.4

The Australian Government should provide for broad oversight and complaints handling functions relating to the use of the Comprehensive Right. Accordingly, the Australian Competition and Consumer Commission (ACCC) should be resourced to undertake the following additional responsibilities:

  • Approving and registering industry data-specification agreements and standards.
  • Handling complaints in relation to a data holder’s failure to meet the terms of the Comprehensive Right, including in regard to the scope of consumer data.
  • Educating consumers (in conjunction with State and Territory fair trading offices) on their rights and responsibilities under the Comprehensive Right.
  • Assessing the validity, when requested or at their discretion, of charges levied by data holders for application of the Comprehensive Right.

The Office of the Australian Information Commissioner and industry ombudsmen should, in order to ensure a ‘no wrong door’ approach to handling consumer engagement, coordinate with the ACCC on the receipt and handling of consumer complaints on data access and use.

5.5

The Australian Government should adopt a minimum target for voluntary participation in Comprehensive Credit Reporting of 40% of all active credit accounts, provided by Australian Securities and Investments Commission (ASIC)-licensed credit providers, for which comprehensive data is supplied to the credit bureaux in public mode.

If this target is not achieved by 30 June 2017, the Government should circulate draft legislation by 31 December 2017, to impose mandatory participation in Comprehensive Credit Reporting (including the reporting of repayment history) by ASIC-licensed credit providers in 2018.

The Office of the Australian Information Commissioner and ASIC should consult with other regulators, industry groups and consumer advocates to collaboratively consider whether there is a need for a hardship flag in credit reporting.

The Department of the Treasury should be given responsibility for monitoring and publicly reporting on a regular basis on participation in Comprehensive Credit Reporting.

6.1

As an immediate objective, all Australian governments should direct the early release of all non-sensitive publicly funded datasets — whether held by a government agency or other body receiving public funding for data collection activities.

A realistic assessment of the risks attached to public release of identifiable information that is already public (in a less accessible form) should be undertaken by all governments, with the intention of releasing low risk data, and mitigating risks where possible to enable far greater public release of data, including that which could be used for program or agency performance management purposes.

Agencies should report annually on the proportions of their datasets made publicly available, shared, and not available for release.

6.2

Additional qualified entities should be accredited to undertake data linkage.

State-based data linkage units should be able to apply for accreditation by the National Data Custodian (recommendation 6.6) to allow them to link Australian Government data.

6.3

All Australian governments entering into contracts with the private sector that involve the creation of datasets in the course of delivering public services should assess the strategic significance and public interest value of the data as part of the contracting process.

Where data is assessed to be valuable, governments should retain the right to access or purchase that data in machine-readable form and to subsequently apply any analysis and release strategy that is in the public interest.

The Australian Government Department of Finance should modify template contracts to, by default, vest access and purchase rights in governments, and avoid the need for negotiating separate rights in each contract. State and Territory governments should adopt a similar approach.

6.4

Publicly funded entities, including all Australian Government agencies, should create comprehensive, easy to access registers of data, including metadata and linked datasets, that they fund or hold. These registers should be published on data.gov.au. Where datasets are held or funded but are not available for access or release, the register should indicate this and the reasons why this is so.

States and Territories should create an equivalent model for their agencies where such registers do not exist. These should, in turn, be linked to data.gov.au.

A reasonable timeframe in which to achieve this is within one year (by March 2018).

6.5

In determining datasets for public release, a central government agency in each jurisdiction with overarching policy responsibility for data should offer a public process whereby datasets or combinations of datasets can be nominated, with a public interest case made, for release.

A list of requested datasets, and decisions regarding dataset release or otherwise, should be transparent and published online — in the Commonwealth’s case, on data.gov.au.

6.6

The Australian Government should establish an Office of the National Data Custodian (NDC) to take overall responsibility for the implementation of data management policy, in consultation with all levels of Government.

The Office of the NDC should have responsibility for:

  • Broad oversight and ongoing monitoring of and public reporting on Australia’s national data system and the operation of the new Data Sharing and Release Act (recommendation 8.1).
  • Preliminary assessments for, and recommending designation of, National Interest Datasets (recommendation 7.1).
  • Accrediting release authorities, be party to determining a funding agreement for Accredited Release Authority (ARA) activities, and promoting cooperation between ARAs.
  • Managing complaints about ARA processes.
  • Providing practical guidance material for ARAs and data custodians on matters such as risk management, data curation and metadata, data security, data de- identification and trusted user models.
  • Advising on ethics and emerging risks and opportunities in data use.

The Office of the NDC should include a small advisory board, comprising members with technical skills related to the NDC’s activities, and a dedicated ethics adviser.

The NDC role should be filled administratively by the end of 2017 to be operational by the time that new draft legislation for data access is completed for public consultation (recommendation 10.2).

6.7

The National Data Custodian should streamline approval processes for access to data by:

  • Issuing clear guidance to all Australian Government data custodians on their rights and responsibilities, ensuring that requests for access to data they hold are dealt with in a timely and efficient manner and are consistent with the risk management approach to be adopted by Accredited Release Authorities (ARAs).
  • Requiring that these data custodians report annually on their handling of requests for data access, including requests from ARAs.

State and Territory governments may opt in to these approaches to enable use of data for jurisdictional comparisons and cross-jurisdictional research.

6.8

Selected public sector and public interest entities should be accredited as release authorities. Accreditation should be determined based on sectoral expertise, capability, governance structures, and include consultation throughout the relevant sector.

Accredited Release Authorities (ARAs) would be responsible for:

  • deciding (in consultation with original data custodians) whether a dataset is available for public release or limited sharing with trusted users
  • collating, curating, linking and ensuring the timely updating of National Interest Datasets and other datasets
  • offering advice, services and assistance on matters such as dataset curation, de-identification and linking
  • providing risk-based access to trusted users.

ARAs should be fully operational from the beginning of 2019.

6.9

All Accredited Release Authorities must have and publish formal risk management processes to effectively assess and manage the risks associated with sharing and release of data under their control.

Standardised, access-friendly Data Sharing Agreements should be implemented with external data providers and users to formalise the activities that can take place with identifiable and de-identified data.

Risk management processes should be regularly reviewed and revised to account for new and emerging risks.

6.10

Funding of Accredited Release Authorities (ARAs), for the purposes of data management, curation, storage and access should be set via a funding agreement with the National Data Custodian.

ARAs should have the power to charge fees sufficient to recoup costs where ARAs undertake requested work beyond that envisaged in their funding arrangement with the National Data Custodian.

In assessing the scope to undertake such activities, ARAs must ensure they do not detract from their primary focus on the public benefits of enabling greater access to, and use of, data (which is the basis for their accreditation and funding).

6.11

The Office of the National Data Custodian should be afforded the power to require an audit of a data custodian’s de-identification processes and issue assurance of de-identification practices used.

6.12

Accredited Release Authorities (ARAs) should be given responsibility to grant, on a continuing program-wide basis, data access to trusted users from a range of potential entities that:

  • Have the necessary governance structures and processes in place to address the risks of inappropriate data use associated with particular datasets, including access to secure computing infrastructure.
  • Have a signed legal undertaking that sets out safeguards for data use and recognises relevant privacy requirements.

In assessing trusted user access, the ARAs should accept existing current approvals of the trusted user’s work environment.

Trusted user status for use of identifiable data would cease for that user when they leave the approved environment, when a program is completed, or if a data breach or mishandling occurs in that same environment and/or program.

6.13

Accredited Release Authorities (ARAs) and data custodians should be required to refer suspected and actual violations of data use conditions that have system-wide implications to the National Data Custodian.

Clarification should be issued detailing how this process would interact with the Privacy Amendment (Notifiable Data Breaches) Act 2017 (Cth).

6.14

Progress by individual research institutions receiving Australian Government funding in making their unique research data and metadata widely available to others should be openly published by those institutions, with reference to past performance.

All bodies channeling public funds for research, such as the National Health and Medical Research Council and Australian Research Council, should similarly require in future funding agreements with research applicants that data and metadata is to be publicly available, and publish the results of progress on this for their funded projects.

On completion of projects, research institutions should include in their reports details of when and how other researchers can access the project’s data and metadata.

6.15

Processes for obtaining approval from human research ethics committees (HRECs) should be streamlined.

To achieve this in the health sector:

  • All HRECs should be required to register with the National Health and Medical Research Council (NHMRC). The NHMRC should receive funding to expand its current registration process, to include audits of registered HRECs.
  • To maintain their registration, HRECs must implement efficient and timely approval processes, which ensure projects are not unduly delayed. The time taken to consider and review projects should be reported to the NHMRC, and included in the annual report on HREC activity.
  • As a condition for registration, all HRECs and the institutions they operate in would be required to accept approvals issued by certified HRECs for multi-site projects, without additional reviews. The Australian Health Ethics Committee should develop uniform review processes to be used by certified HRECs.
  • The Council of Australian Governments’ Health Council should sign an intergovernmental agreement that extends the existing National Mutual Acceptance Scheme to all jurisdictions, including the Commonwealth, and all types of projects. As part of this agreement, all jurisdictions should also implement streamlined governance approvals.
6.16

The Privacy Act 1988 (Cth) exceptions that allow access to identifiable information for the purposes of health and medical research without seeking individuals’ agreement, should be expanded in the legislative package that implements these reforms to apply to all research that is determined by the National Data Custodian to be in the public interest.

6.17

The Australian Government should abolish its requirement to destroy linked datasets and statistical linkage keys at the completion of researchers’ data integration projects. Where an Accredited Release Authority is undertaking multiple linkage projects, it should work towards creating enduring linkage systems to increase the efficiency of linkage processes.

Data custodians should be advised as part of early implementation of this reform package to use a risk-based approach to determine how to enable ongoing use of linked datasets. The value added to original datasets by researchers should be retained and made available to other dataset users.

7.1

The Australian Government, in consultation with State and Territory governments, should establish a process whereby public (and in some exceptional cases, private) datasets are nominated and designated as National Interest Datasets (NIDs).

This process should be public, driven by the National Data Custodian, and involve:

  • The National Data Custodian accepting nominations for NIDs, assessing their public interest merits and, after consideration by the Government, referring selected nominations to a public scrutiny process. Designation would occur via a disallowable instrument on the recommendation of the National Data Custodian.
  • The establishment of a parliamentary committee, or addition of such a role to the work of an existing parliamentary committee, to conduct public scrutiny of nominations for NIDs.

The process of nomination should be open to the States and Territories in order to cover linked datasets.

This process should be in place by the end of 2018, as part of the legislative package to implement these reforms.

7.2

In considering nominations for National Interest Datasets (NIDs), the National Data Custodian’s public interest test should establish that through sharing or release, the designation of a dataset would be likely to generate significant additional community- wide net benefits beyond those obtained by the original data holder.

Once designated, NIDs that contain non-sensitive data should be made available for immediate release.

NIDs that include data on individuals would be available to trusted users only in a manner that reflects the accreditation processes of the relevant Accredited Release Authority, as established and updated by the National Data Custodian, to respect privacy and confidentiality.

Where data from the private and/or not-for-profit sectors is recommended to be included in a NID, the analysis prior to designation should specifically note the ways the designation addresses genuine commercial sensitivity associated with the information and costs (including those related to ongoing dataset maintenance).

7.3

Trusted users should be accredited by the relevant Accredited Release Authority (ARA) for access to those National Interest Datasets (NIDs) that are not publicly released, under processes accredited and updated as needed by the National Data Custodian.

Trusted users should be personnel from a range of potential entities that:

  • Have the necessary governance structures and processes in place to address the risks of inappropriate data use associated with particular datasets, including access to secure computing infrastructure.
  • Have a signed legal undertaking that sets out safeguards for data use and recognises relevant privacy requirements.

The default position should be that after applicants and their institution establish capability to respect the processes and obligations of the ARA’s accredited standard, an individual researcher from one of these organisations would be readily approved for access.

For trusted users of NIDs, this status should provide an ongoing access arrangement to specified unreleased datasets that would only cease on completion of a researcher’s engagement with their relevant institution, or a loss of trust in the user or their organisation (via processes also established in accreditation of the ARA by the National Data Custodian).

7.4

The Australian Government should make provision, in select circumstances as approved by the funding Minister, for the National Data Custodian to pay for access or linkage to private sector datasets (recommendation 9.4).

Equally, the National Data Custodian may consider applying charges for access to National Interest Datasets where this would not be inconsistent with the public interest purpose of the National Interest Dataset.

It is expected this would not be a common occurrence, in either case.

8.1

New Commonwealth legislation — the Data Sharing and Release Act — should be passed drawing on the full range of Commonwealth powers to regulate digital data, in order to authorise the better sharing and release of data.

The new Act should also establish the Comprehensive Right of consumers to access their data from government and private data holders alike, for the purposes of improving the services that are offered to them by alternative providers.

8.2

The Data Sharing and Release Act should establish the risk-based approach to data sharing and release and accompanying institutional frameworks.

  • All non-sensitive data held by agencies and Accredited Release Authorities (ARAs) should be explicitly presumed to be made public, consistent with the Australian Government’s Public Data Policy Statement.
  • Data custodians and ARAs would be authorised to provide sensitive data to trusted users in a secure environment, with de-identification where necessary for risk management of the data.
  • The National Data Custodian should have the authority to issue guidance on how the risks of all sharing of identifiable data between entities should be managed. This guidance should be updated where it judges the risks have shifted.
8.3

The Data Sharing and Release Act (DSR Act) would, where possible, override secrecy provisions or restrictions on use that prevent original custodians actively providing access to data to other public sector data custodians and Accredited Release Authorities (ARAs).

Access should be governed by Data Sharing Agreements that embed the trusted user principles, actively assist data sharing and create clarity of understanding amongst the parties. The National Data Custodian (NDC) should issue a model Data Sharing Agreement early in its life, and update it from time to time.

The DSR Act should establish modern, clear and supportive standards — the new ‘rules of the game’ — for data sharing and release. The Commonwealth Privacy Act would continue to apply, as well as any residual obligations emanating from the original data custodian’s legislation.

Existing protections would remain on datasets that do not utilise the DSR Act, in order to ensure there is no gap between the accountability obligations on original public sector data custodians and the ARA.

In limited exceptional circumstances as the DSR Act transitions to becoming nationally effective, it may be necessary to provide access to data shared under the new Act to a party that has yet to adopt its provisions. The NDC should be provided with the power to use a disallowable instrument to allow access or sharing for such transitional purposes.

8.4

The Australian Government’s Protective Security Policy Framework (and equivalent State and Territory policies) should be amended to recognise that the risk and therefore the classification needed for data can be reduced by:

  • Transforming a dataset, for example through de-identification, such that the risks of misuse on dataset release are reduced.
  • Only making the transformed data available to trusted researchers in a secure computing environment, with usage monitored and output checked for disclosiveness.

This would align the Protective Security Policy Framework with the current legal environment.

The Australian Government should consider doing this as part of its response to the Belcher Review.

8.5

Legislative reform to implement the Commission’s recommendations would need to be undertaken in two parts, moving forward together:

  • The first part is the passage of the Data Sharing and Release Act (DSR Act) itself, that authorises to the greatest extent practical in a single statute, the sharing and release of data for the purposes of the Act and removes existing Commonwealth and State restrictions on integrating, linking and research uses of datasets by Accredited Release Authorities.
  • The second part is a further legislative amendment process that may be necessary, depending on the particular characteristics of, for example, National Interest Datasets, in order to address residual restrictions on the use of specific datasets that were not able to be effected by the DSR Act itself.

The National Data Custodian should be asked to identify residual legislative restrictions that need removal in its consideration of National Interest Datasets.

8.6

The Data Sharing and Release Act (DSR Act) should have national reach — to create a simplified and transparent one-stop location for a national framework for data volunteered, declared or acquired for inclusion under the DSR Act.

The Act should allow for the acquisition of private datasets via disallowable instruments as part of the process of creating National Interest Datasets (NIDs). Acquisition should only occur on just terms after parliamentary scrutiny determines the benefits are demonstrable.

An initial set of NIDs should be identified by the National Data Custodian to accompany the DSR Bill, following processes to establish additionality and public interest.

The DSR Act should apply Commonwealth privacy legislation to datasets managed by Accredited Release Authorities where feasible. It should be drafted with reference to (and with the intention of being consistent with) the Data Sharing (Government Sector) Act 2015 (NSW) and the Public Sector (Data Sharing) Act 2016 (SA) to the extent possible.

8.7

The Australian Competition and Consumer Commission (ACCC) and the Office of the Australian Information Commissioner should enter into working arrangements with each other, industry ombudsmen and other relevant bodies at all levels of government to support a ‘no wrong door’ approach to how individuals (including small businesses) pursue complaints or queries regarding their rights as consumers to data held on them.

Where an industry data-specification agreement (recommendation 5.2) seeks to use a recognised industry ombudsman to address consumer complaints, this should be considered by the ACCC as part of its acceptance or rejection of a proposed industry agreement.

9.1

The emphasis for government agencies in handling data should be on making data available at a ‘fit for release’ standard in a timely manner. Beyond this, agencies should only transform data beyond the basic level if there is a clearly identified public interest purpose or legislative requirement for the agency to undertake additional transformation, or:

  • the agency can perform the transformation more efficiently than either any private sector entities or end users of the data
  • users have a demonstrable willingness to pay for the value added product
  • the agency has the capability and capacity in-house or under existing contract
  • the information technology upgrade risk is assessed and found to be small.
9.2

The pricing of public sector datasets for public interest research purposes should be the subject of an independent review.

9.3

Minimally processed public sector datasets should be made freely available or priced at marginal cost of release.

Where data has been transformed, the transformed dataset may be priced above the marginal cost of release. Data custodians should experiment with low prices initially to gauge the price sensitivity of demand, with a view to sustaining lower prices if demand proves to be reasonably price sensitive.

9.4

Funding should be provided to agencies for the curation and release of those datasets determined through the central data agency’s public request process (recommendation 6.5) to be of high value with a strong public interest case for their release. This funding should be limited and supplemental in nature, payable only in the event that agencies make the datasets available through public release.

Funding would also be required for the Office of the National Data Custodian, for functions undertaken by Accredited Release Authorities and, in some cases, for the purchase and ongoing maintenance of National Interest Datasets. Additional responsibilities required of the Australian Competition and Consumer Commission in regard to the Comprehensive Right should also be resourced.

Aside from these purposes, no additional supplementary funding appears warranted for agencies’ activities related to their data holdings as a consequence of this report.

10.1

The Australian Government should engage actively with the community on matters related to data availability and use.

At a minimum, the National Data Custodian should regularly convene forums for consultation, to ensure community concerns about increased use of data are addressed.

10.2

The Australian Government should set an ambitious — but realistic — timeline for implementation of the Commission’s recommended reforms.

A set of actions in this Report can be completed in 2017, to ensure they deliver benefits to the community in the short term.

Passage of the Data Sharing and Release Act and supporting Part 2 amendments for an initial suite of National Interest Datasets should be in place by the end of 2018.

A central agency with data responsibility should actively support the progress made against the implementation plan until the Office of the National Data Custodian is legislatively established.

Once established, the National Data Custodian should assume responsibility for monitoring and evaluating the effects of the new data Framework, reporting annually on progress and with a formal evaluation after three years’ experience of the Framework’s reforms.

10.3

Government agencies should adopt and implement data management standards to support increased data availability and use as part of their implementation of the Australian Government’s Public Data Policy Statement.

These standards should:

  • be published on agency websites
  • be adopted in consultation with data users and draw on existing standards where feasible
  • deal effectively with sector-specific differences in data collection and use
  • support the sharing of data across Australian governments and agencies
  • enable all digitally collected data and metadata to be available in commonly used machine-readable formats (that are relevant to the function or field in which the data was collected or would likely be most commonly used), including where relevant and authorised, for machine-to-machine interaction.

Policy documents outlining the standards and how they would be implemented should be available in draft form for consultation by the end of 2017, with standards implemented by the end of 2020.

Agencies that do not adopt agreed sector-specific standards would be noted as not fully implementing the Australian Government’s Public Data Policy and would be required to work under a nominated Accredited Release Authority to improve the quality of their data holdings.

10.4

The private sector is likely to be best placed to determine sector-specific standards for data sharing between firms, where required by reforms recommended under the new data Framework.

In the event that cooperative approaches to determining standards and data quality do not emerge or adequately enable data access and transfer (including where sought by consumers), governments should facilitate this.