Artificial Intelligence, Automated Decision-Making and Privacy Laws – the opportunities, the risks and the reforms

Gone are the days of hand-writing notes, having to search through physical filing cabinets to find records or awaiting seven days for correspondence to be sent by post.

The rapid evolution of technology, particularly generative artificial intelligence (AI), can provide a number of opportunities for business efficiency but it is crucially important to be aware of the professional and legal obligations and risks of using technology in your workplace.

This is particularly so given the increased focus on privacy breaches and reforms to the privacy legislation over the last year which have increased penalties and obligations for businesses and may shortly further expand the scope of the privacy laws to small businesses and other information.

Marissa Mackie, Principal in our Commercial and Tax Team, recently presented to the Australian Dental Association, on the integration of AI scribes in dental practices and privacy obligations. This technology effectively captures audio data during a consultation and then converts the data into a summary of a consultation, using various algorithms and predictive models to filter information.

The administrative ease and time-saving from the use of this technology is self-evident. It can provide practitioners the ability to have more time to treat patients, could capture more information and could result in improving the quality of care and outcomes for patients and reduce the administrative burden for practitioners.

But like with all technologies, businesses need to be mindful of the accuracy of any output that technologies generate and the risks of over-reliance on technology. Importantly, across all uses of AI, one only need read recent news headlines to be aware of the ‘hallucinations’ that AI can generate and the consequences it can have. Specifically for AI scribes, it is important to remember that it only picks up what is heard and may not have the benefit of other records, documents or non-verbal cues in generating its output.

Privacy Laws

Businesses also need to ensure that their use of technology and information generally complies with various laws and regulations including under the Privacy Act 1988 (Privacy Act) which has been the subject of recent reforms together with other surveillance[1] and cyber-security legislation[2]. The Privacy Act and the Australian Privacy Principles (APPs) regulate the collection, use, quality, disclosure, storage, retention and cross-border transfer of certain information and sets out what entities must do to ensure compliance with the law.

Businesses need to ensure that confidential and sensitive information is collected, stored and utilised in accordance with those laws and is not being disclosed or utilised without authorisation and that they take reasonable steps to protect information a business holds. It is crucial to ensure employees are aware of their responsibilities and obligations to ensure that technology is utilised safely and responsibly and that information is not unintentionally disclosed. This is particularly critical having regard to increasing reports of data breaches.

Recent reforms to the Privacy Act have expanded the powers and penalties that the Australian Information Commissioner may seek to impose for contraventions of the privacy law and the APPs.

Automated Decision-Making Tools

New rules have also been introduced in relation to the use of automated decision-making tools which will take effect on 10 December 2026. These reforms relate to disclosures required where an entity covered under the Privacy Act (including government) uses an individual’s personal information in programs which fully automate decision-making or substantially assist the decision-making process where those decisions could reasonably be expected to significant affect the individual’s rights or interests.

Whilst some guidance has been provided in respect of the types of decisions that may affect the rights or interests of an individual, there is great potential for the scope of these reforms to capture a number of decision-making tools often utilised by government and private entities in applications for various government or financial benefits, contractual decision making, pricing, and recruitment.

Entities will have to update their privacy policies to provide greater transparency on:

• the kinds of personal information used in the operation of computer programs

• the kinds of decisions made solely by the operation of computer programs, and

• the kinds of decisions for which a thing, that is substantially and directly related to making the decision, is done by the operation of such computer programs.

Where are we at and what is to come?

There are further proposed reforms to the Privacy Act which seek to broaden the scope of the laws to more small business entities and information that may fall under the laws, including employee information. It is also intended that further limitations will be placed on the collection and use of information even if consent has been obtained and reforms will address matters surrounding the use of AI.

The increased scrutiny by the Information Commissioner on privacy breaches and the expanded powers and penalties under the Privacy Act reiterate the importance of ensuring that businesses understand their obligations under privacy laws and the interaction of these laws with the increasing use of technologies (particularly when it comes to the collection, storage and use of information). Having and implementing well drafted privacy and workplace policies are critical to ensuring businesses do not fall foul of these laws.

Before integrating any technology in your business, be it AI or a cloud storage service, it is important to consider a number of matters including:

• How the technology actually works to understand its limitations, accuracy and reliability.
• What data is being collected and how it is being stored, how long it is being stored for, and where it is being stored (especially if overseas).
• Who has access to the information and how it is being utilised.
• Whether your privacy and data collection policies and contractual arrangements permit and appropriately disclose your use of technology and information including consent from your clients.
• Whether your use of technology complies with privacy/surveillance laws.
• Whether there are other professional obligations/laws related to your business which may regulate the use of technologies in the work-place.
• Implementing policies to ensure the responsible use and supervision of technology in your business.
• Your obligations in respect of any data breaches and how your use of technology may impact on your business insurance including cyber-security.

Given AI is rapidly evolving, so are the laws and regulations surrounding its use and there are likely still other risks and considerations that have not been identified yet. Norman Waterhouse is keeping a close eye on this ever-evolving landscape to continue to safeguard and keep its clients up to date on the appropriate, professional and ethical use of AI.

Do not hesitate to contact our team for further information and assistance in implementing and reviewing policies regarding the use of technologies and your privacy obligations.

[1] Surveillance Devices Act 2016

[2] Cyber-Security Act 2024.