Since the publication of my first article on artificial intelligence in the 2018 summer edition of Bar News,¹ both interest, and advancement, in AI technology have increased at a prodigious rate. The latest technology to make headlines, ChatGPT, has been touted as having passed the famed ‘Turing test’ of artificial intelligence – a test posited by Alan Turing in 1950, whereby a machine attempts to convince a human interrogator it really is human through a series of written responses to various questions. While ChatGPT’s purported triumph over the Turing test is debatable, what is clear is that ChatGPT is a powerful technology – which presently has the capacity to assist barristers in their day-to-day practice and the potential to perhaps even disrupt some areas of legal practice in general.

What is it?

Fundamentally, ChatGPT is an autoregressive² language model, which processes input text and generates human-like conversational responses. The expression is a portmanteau of ‘chatbot’ (software which simulates human conversation) and the initialism GPT, meaning ‘generative pre-trained transformer.’ A ‘transformer’ in this context is a species of a deep learning model typically used in natural language processing. Users interact with the technology as they do with any chatbot using natural language. OpenAI, a San Francisco-based technology company whose modest ambition is to create ‘safe artificial intelligence that benefits all of humanity,’ claims that, in developing ChatGPT, it utilised Reinforcement Learning from Human Feedback, whereby human AI trainers were provided with conversations in which they played both sides – the user and an AI assistant. The model uses text databases taken from approximately 570GB of data sourced from books, Wikipedia, articles and other text from the internet – although, the dataset is restricted to September 2021. At present, the freely available version of ChatGPT is purely text-based. On 15 March 2023, OpenAI released GPT-4 which it claims ‘can solve difficult problems with greater accuracy’ and ‘surpasses ChatGPT in its advanced reasoning capabilities.’ Unlike ChatGPT, GPT-4 can accept images as inputs as well as text inputs. At the time of writing, I have not yet had the opportunity to explore GPT-4’s capabilities.

How do I access and use ChatGPT?

Access and use are easy and presently free although, there is now a ‘ChatGPT Plus’ version available for USD $20 per month, which gives added benefits such as availability of the technology even when demand is high and access to GPT-4. Firsttime users need to create an account with OpenAI. Once logged in, users are presented with a minimalist web-page containing a dialogue box at the bottom of the screen, a few examples of its capabilities and a brief disclaimer as to its limitations (more on that below). Users then interact with ChatGPT by inputting text into the dialogue box.

What can it do?

Leaving aside its limitations, of which there are many, ChatGPT produces some surprisingly informed responses. To demonstrate, below are some random questions I posed when writing this article, and their respective responses:

Q. Explain the first law of thermodynamics in simple terms.

A. The first law of thermodynamics is also known as the law of conservation of energy. It states that energy cannot be created or destroyed, only transformed from one form to another. This means that the total amount of energy in a closed system remains constant, and any energy that appears to be lost or gained in one part of the system must be accounted for in another part. In simpler terms, it means that energy is always conserved and never disappears or magically appears out of nowhere.

Q. Explain the concept of responsible government in Australia to an eighth grader.

A. Responsible government in Australia is a system where the elected representatives, like the Prime Minister and members of parliament, are accountable to the people who voted for them. It means that the government is responsible for the decisions it makes and the actions it takes, and must answer to the people if they do something wrong. [Extract only, ChatGPT then went on to explain, accurately, the concepts of the executive and the legislature and expand upon the mechanics of responsible government].

When prompted ‘what can I use ChatGPT for?,’ the chatbot boasted that ChatGPT can be used ‘for a variety of purposes,’ such as language translation, content generation (articles, summaries), as a personal assistant (scheduling and reminders), education, customer service and research (generating hypotheses and analysing data). The internet abounds with examples of novel uses of the technology. These include writing songs, poetry, scenes to fit into well-known sitcoms, such as The Office and Seinfeld, and even how to remove peanut butter from a VCR. Programmers and software developers have been particularly receptive to its potential uses allowing them to write code more efficiently, and quickly identify problems with existing code. One notable example in the legal realm attracted attention when earlier this year, a team of University of Minnesota law professors published a white paper setting out their results of an experiment, whereby they tasked ChatGPT with responding to exams in constitutional law, labour law, taxation and torts. The professors found that ChatGPT passed all four classes on its final exam achieving an average of C+ across all exams.³ If such performance were consistent throughout the law school, the grades earned by ChatGPT would be sufficient to graduate with a J.D. ‘Overall, ChatGPT wasn’t a great law student acting alone, but we expect that collaborating with humans, language models like ChatGPT would be very useful to law students taking exams and to practising lawyers,’ said law professor Jonathan Choi, one of the lead professors conducting the experiment.

Uses of ChatGPT in the legal context

Aside from passing law exams, ChatGPT has already been deployed within the legal context. Having been released on 30 November 2022, it is still too early to tell whether ChatGPT can be relied upon to be regularly utilised in legal practice although, if certain media reports are to be trusted, many leading law firms see the technology as an opportunity. For example, the chief operating officer at Gilbert + Tobin, Sam Nickless, in a recent interview with the AFR, cited ChatGPT’s skill at extracting structured data from large slabs of unstructured text – for example, ‘extract all names and addresses from this text.’ Mr Nickless also explained that ‘[ChatGPT] is already assisting in marketing-type material– first drafts, turning an article into a social post, redrafting for clarity. It is brilliant for helping with Excel formulas.’⁴ Similarly, Clayton Utz has started using ChatGPT to drive efficiencies in its environmental, social and governance (ESG) practice. In a recent interview with Lawyers Weekly, Bruce Cooper, chief executive partner at Clayton Utz, explained that Clayton Utz had developed a product using ChatGPT, which assists in ‘pulling’ key information from a reliable data source, such as a judgment. ‘Once complete, ChatGPT converts that output into a well-articulated case summary. This summary and the original data source then have to be reviewed by a lawyer for accuracy and completeness, but the use of this technology is saving us time.’⁵

Examples of more adventurous and sophisticated uses are also starting to emerge. Earlier this year, Sunshine Coast lawyer Kyle Kimball experimented with ChatGPT to draft a statement of claim based upon the Mabo decision. The result was surprising: ‘If I’d asked a first-year lawyer to do it, it wouldn’t have been anywhere as good,] he said. While acknowledging the need to be circumspect with the output, Mr Kimball also saw great potential, adding that the work of most junior lawyers also needed to be carefully checked: ‘Find me a person that you’re paying to do a job that does the job 100 per cent accurately all of the time, and in the time this machine does it in,’ he said.”⁶ In possibly the first use of ChatGPT in a legal judgment, Judge Juan Manuel Padilla Garcia, sitting in the First Circuit Court in the city of Cartagena (roughly equivalent to a local court) used ChatGPT when presiding over a case involving an autistic child, where one of the key issues that arose was whether an insurer was liable to pay for the entirety of the child’s medical expenses and transport costs. Judge Garcia interrogated ChatGPT with various legal questions, such as ‘Is an autistic minor exonerated from paying fees for their therapies?’ and ‘Has the jurisprudence of the [Colombian] constitutional court made favourable decisions in similar cases?’⁷ In defending his use of the technology, Judge Garcia has been reported as explaining that ‘What we are really looking for is to optimise the time spent drafting judgments after corroborating the information provided by AI.’⁸

How can it help you as a barrister?

My own research and experimentation have led me to the view that ChatGPT certainly has the potential to assist with some aspects of barrister’s work, although I have not yet formed a conclusive view. A few examples can be used to illustrate its potential uses.

One of my first experiments was rudimentary, asking for explanations of various scientific and engineering concepts to assist in understanding a number of expert reports.

Q. Explain Miner’s rule in simple terms.

A. Miner’s rule is a simple rule used in engineering to estimate the durability of a material under repeated loading. It states that the life of a component is determined by the accumulation of damage it experiences over time. The damage is usually measured as a ratio of the applied stress to the material’s strength. [Extract only].

A quick Google search confirmed the accuracy of the response and also demonstrated ChatGPT’s distinct advantage over a typical Google search. Compared to Google, ChatGPT quickly and accurately explained a technical engineering concept in concise, plain English – obviating the need to trawl through numerous websites to achieve the same result. The efficiencies to me with such usage were obvious.

A second experiment was to use ChatGPT to summarise case law. While ChatGPT’s dataset is limited to September 2021, it can be used to analyse any text-based dataset and can access weblinks to do so. My second experiment began with the following:

Q. Summarise the court’s decision in Dasreef Pty Ltd v Hawchar [2011] HCA 21 [link to Austlii version of decision] in 100 words or less.

A. The case of Dasreef Pty Ltd v Hawchar is about an employee who suffered a back injury while working for a company. The employee claimed compensation for the injury under the workers’ compensation scheme. The employer argued that the employee’s injury was not caused by work, but by a pre-existing condition. The court found that the employer’s argument was not valid and that the employee was entitled to compensation. The court held that the injury arose out of the employment, even if it was aggravated by the preexisting condition. A technically accurate summary, although not very useful. I then started to ‘prompt engineer’ some of my questions – that is, getting to know how to communicate with ChatGPT. I then tried the following:

Q. Summarise the court’s reasoning regarding section 79 of the Evidence Act 1995 in Dasreef Pty Ltd v Hawchar [2011] HCA 21 [link to Austlii version of decision].

A. In the case of Dasreef Pty Ltd v Hawchar, the High Court of Australia considered the admissibility of evidence under section 79 of the Evidence Act 1995. The court held that section 79 did not apply to exclude evidence of a previous medical condition that was not relevant to the injury in question.

The court reasoned that section 79 only excludes evidence of a person’s character or conduct if it is sought to be used to prove that the person has a tendency to act in a certain way. The court also held that evidence of a preexisting condition was admissible to the extent that it was relevant to the question of whether the injury arose out of the employment.

Again, not very useful. My next question was ‘Summarise the court’s reasoning regarding the admissibility of expert evidence in Dasreef Pty Ltd v Hawchar [2011] HCA 21.’ ChatGPT then generated a single paragraph response, which, although, again, technically accurate, was inutile and mundane. I then prompted ChatGPT with the following: ‘Explain in detail the court’s reasoning regarding the admissibility of expert evidence in Dasreef Pty Ltd v Hawchar [2011] HCA 21.’ What followed was a six-paragraph summary accurately setting out the High Court’s reasoning regarding the admissibility of expert evidence – almost worthy of inclusion in a written submission. I say almost because the language used lacked sophistication. Nonetheless, the summary of the Court’s core reasoning was helpful.

My final experiment involved a more prosaic task. After some prompt engineering explaining that solicitors are clients to whom I regularly issue invoices with 30-day terms, I asked ChatGPT to ‘draft a polite email reminding them that the payment of my invoice is overdue.’ The response was as follows:

Certainly! Here is a draft of a polite email that you can use as a reminder to your solicitor clients:

Dear [Solicitor’s Name],

I hope this email finds you well. I am writing to kindly remind you that the payment for my services rendered on [date] is now overdue. As per our agreement, the payment was due [30 days after the invoice date].

I understand that circumstances can sometimes cause delays in payments, and I would appreciate it if you could let me know when I can expect payment to be made. If there are any issues or concerns regarding the invoice, please do not hesitate to contact me so that we can resolve the matter together.

Please let me know if there are any updates regarding the payment or if you require any further information from me. I value our business relationship and look forward to continuing to work with you in the future.

Thank you for your attention to this matter.

Best regards,

[Your Name]

While not exactly to my liking, it is by no means terrible. I suspect that if I had provided it with examples of previous correspondence, it would likely emulate my writing style.

Beware the pitfalls

As alluded to above, in using the technology, users must be cautious as to its numerous limitations and shortcomings. The most significant is ChatGPT’s susceptibility to ‘hallucinations’ – i.e., simply making things up. Where the AI has access to limited data, it will nonetheless very confidently generate seemingly valid and reliable responses that are completely fictitious. One recent example is that of the experience of a Canadian legal academic who asked ChatGPT to generate a list of academic articles on administrative monetary penalties.9 ChatGPT obliged with a list of thirteen ‘articles’ complete with citations. The problem was that not one of the articles existed. This is a well-known shortcoming and it has been estimated that the ‘hallucination rate’ for ChatGPT is about 15% to 20%. Another significant potential pitfall is confidentiality, or lack thereof. The technology is cloud-based with the usual attendant privacy risks.

A surprise twist

After completing the first draft of this article, I asked ChatGPT ‘Can you write a 3,000 word article for me explaining what ChatGPT is and how barristers can use ChatGPT in practice?’ The results can be viewed on the next page. After reading thebarticle, I am minded to tender my resignation from the Bar News Committee and instead nominate ChatGPT in my stead. BN

ENDNOTES

^{1 Farid Assaf, ‘From Countess Lovelace To Ross: A Brief Overview Of Artificial Intelligence’ [2018] (Summer) Bar News 21.}

^{2 That is, capable of predicting future value based on past values.}

^{3 Jonathan Choi et al, ‘ChatGPT Goes to Law School’ (Research Paper No 23-03, University of Minnesota Law School, 23 January 2023) 5.}

^{4 Michael Pelly, ‘Law firms say ChatGPT an ‘opportunity, not a threat’’, Australian Financial Review (online, 9 February 2023) <https://www. afr.com/companies/professional-services/law-firms-say-chatgpt-anopportunity- not-a-threat-20230208-p5cj2j>.}

^{5 Lauren Croft, ‘Clayton Utz accelerates ESG work with ChatGPT’, Lawyers Weekly (online, 20 February 2023) https://www.lawyersweekly.com.au/newlaw/36705-clayton-utz-accelerates-esg-workwith-chatgpt.}

^{6 James Purtill, ‘How ChatGPT and other new AI tools are being usedby lawyers, architects and coders’, ABC News (online, 25 January2023) <https://www.abc.net.au/news/science/2023-01-25/chatgptmidjourney-generative-ai-and-future-of-work/101882580>.}

^{7 Lauren Croft, ‘Use of ChatGPT in courts should be ‘approached with great caution’, Lawyers Weekly (online, 13 February 2023) https://www.lawyersweekly.com.au/wig-chamber/36657-use-of-chatgpt-incourts-should-be-approached-with-great-caution.}

^{8 Lauren Croft, ‘Use of ChatGPT in courts should be ‘approached with great caution’, Lawyers Weekly (online, 13 February 2023) https://www.lawyersweekly.com.au/wig-chamber/36657-use-of-chatgpt-incourts-should-be-approached-with-great-caution.}

^{9 Paul Daly, ‘Chat GPT and Legal Research: A Cautionary Tale’, Administrative Law Matters (Blog Post, 21 February 2023) <https:// www.administrativelawmatters.com/blog/2023/02/21/chat-gpt-andlegal-research-a-cautionary-tale/>.}