JD Research Assistants — A2AJ-Bench: Building an AI Legal Reasoning Benchmark

The Access to Algorithmic Justice Project (A2AJ) is seeking 10–12 JD students from Osgoode Hall Law School and Lincoln Alexander School of Law to participate in an intensive one-week sprint to build A2AJ-Bench, a new benchmark for evaluating the legal reasoning capabilities of large language models (LLMs), with an initial focus on areas of law that affect large numbers of people in Ontario.

Apply online at crs1.apps01.yorku.ca/machform/view.php?id=106913 by June 1, 2026.

About the Project

A2AJ is an interdisciplinary research initiative funded by the Law Foundation of Ontario, dedicated to advancing access to justice through the responsible development of artificial intelligence and open legal data infrastructure, with a focus on the needs of marginalized and low-income communities in Ontario.

A2AJ-Bench is a new benchmark designed to test whether AI systems can reliably perform legal reasoning tasks. For the first round of task development, we are concentrating on areas of law where unrepresented and underserved Ontarians most often need help. The benchmark will cover a broad range of areas — including landlord and tenant law, human rights and anti-discrimination law, family law, criminal law, immigration law, child protection, social assistance, employment law, mental health law, workplace injury, consumer protection, and small claims — with landlord and tenant law and human rights / anti-discrimination law as the priority focus areas at the outset. The project builds on the methodology of LegalBench, the leading collaborative benchmark for legal reasoning in LLMs, and is led by members of the A2AJ team, one of whom was a co-author on that project.

The Sprint

During the week of June 22, 2026, the team will work together in person in Toronto to design, draft, and refine benchmark questions and tasks. This is a concentrated, collaborative effort. You will work alongside A2AJ researchers, fellow students, and supervising faculty to produce high-quality evaluation datasets grounded in real-world legal problems faced by ordinary Ontarians.

Key Responsibilities

Draft and refine benchmark tasks (question-and-answer pairs, issue-spotting scenarios, rule-application problems, and similar items) in your area(s) of substantive expertise
Ensure tasks reflect the legal reasoning demands that unrepresented individuals actually face, drawing where possible on real intake scenarios from clinical, advocacy, or volunteer work
Collaborate with other team members to review, test, and validate tasks for accuracy, clarity, and difficulty calibration
Participate in daily working sessions, team discussions, and structured feedback rounds

Qualifications & Competencies

Required

JD student at Osgoode Hall Law School or Lincoln Alexander School of Law. Applications are welcome from students at any stage of the JD (i.e., students who have just completed 1L, 2L, or 3L).
Substantive knowledge in one or more of the benchmark's focus areas: landlord and tenant, human rights / anti-discrimination, family law, criminal law, immigration, child protection, social assistance, employment, mental health, workplace injury, consumer protection, or small claims
Strong legal research, writing, and analytical skills
Ability to work collaboratively in an intensive, team-based setting

Assets

Hands-on, client-facing experience through legal clinics (e.g., Parkdale Community Legal Services), caseworker or paralegal roles, summer employment with firms or clinics, or pro bono or volunteer legal work involving direct client contact
Lived experience with equity-seeking communities, or a demonstrated commitment to access to justice

Terms & Conditions

In-person sprint: Monday, June 22 to Friday, June 26, 2026
Hours: 9:00 AM to 5:00 PM daily during the sprint week (7–8 hours per day)
Location: In person in Toronto (TMU campus)
Availability: Must be available for the full sprint week
Salary: $30.00/hour (inclusive of vacation pay, less statutory deductions)
Application deadline: June 1, 2026

What You'll Gain

Hands-on experience at the intersection of AI, legal reasoning, and access to justice
The opportunity to shape a benchmark that has the potential to influence how AI systems are evaluated for use in Ontario's legal system
Acknowledgment on the resulting publication and dataset
Integration into A2AJ's research community, with networking opportunities alongside legal scholars, technologists, and access-to-justice advocates
Interdisciplinary research experience

To Apply