By Rachel McDonald, the Australian Science Media Centre

Controversial conference Agents4Science 2025 made headlines this week as it became the first open conference where all primary authors and reviewers for the featured papers were artificial intelligence. The virtual conference was held on Wednesday, with AI authorship "not only allowed, but required". The conference website states this event aims to help "drive innovation and open discussion about the role of AI in future scientific research, identifying the areas where AI models can excel and avenues for improvement.

News of the conference has been met with skepticism by Australian AI researchers, with Professor Paul Salmon, co-director of the Centre for Human Factors and Sociotechnical Systems at the University of the Sunshine Coast describing the idea as "dystopian".

"My initial reaction to the Agents4Science concept was one of horror," he said.

"I can see now though that there are wholesome intentions around understanding the capacity of AI to generate scientific insights and I acknowledge that this is important to explore. In my opinion though the conference only further highlights society’s ongoing failure to ensure that AI is safe, ethical, and beneficial."

University of Sydney AI ethics researcher Dr Armin Chitizadeh told the AusSMC that some might consider this conference a kind of "Enhanced Games" for academia - a version of the Olympics which permits performance-enhancing drugs.

"Despite potential controversy, the conference represents an important step toward a more open dialogue about AI in science. Automated Theorem Proving, a field that has used AI to prove mathematical statements since the 1950s, shows that AI has long played a legitimate role in research," He said.

"Yet, the current hesitation to acknowledge AI use mirrors the secrecy of the old alchemy era—when withholding knowledge stifled progress."

Dr Chitizadeh said his main concern was around using AI for the peer review process, especially since AI often reinforces familiar patterns and could miss genuine innovation or unnecessarily dismiss unfamiliar writing styles.

Professor Karin Verspoor, Dean of the RMIT School of Computing Technologies, agreed, saying replacing human peer reviewers with AI seemed "hugely problematic."

"Who should be the arbiter of scientific knowledge? For at least the last 40 years, peer review has been the gold standard for ensuring the quality of the scientific literature, and for engendering trust in science," she said.

She agreed AI systems have inherent biases, and are at their most unreliable at the "edge of our knowledge" - exactly where scientific research does its best work.

Professor David Powers, researcher in Computer and Cognitive Science at Flinders University, attended the conference and said authors were required to disclose the degree and nature of AI involvement in the research they presented, as well as the limitations they encountered with the AI.

This is far more useful than an outright ban on AI involvement in research, and indeed, the panel was positive about the value of AI in their individual areas of expertise, as a member of the team in some sense.

He said one paper in particular was interesting and came with useful comments about the limitations of its AI usage as well as the strengths - with the authors saying AI was able to provide "expertise" in areas the human researchers lacked, but AI tended to "lose focus" on the intent of the research.

"This was truly an excellent paper and worthy of its “best paper” award - even before taking into account the high level of AI-involvement throughout the whole process," Prof Powers said.

Dr Chitizadeh also attended the conference and noted several trends for AI's limitations, with hallucination a "recurring problem" and the AI systems struggling with "creativity and novelty" and often used in the final phases such as writing rather than conceptual stages. He said the focus on finishing touches rather than concepts was also evident in the AI-review process.

"AI-agent reviewers often overlooked core ideas and focused more on surface-level aspects," he said.

He said there was also a strong bias toward mainstream AI tools such as ChatGPT where using alternative programs may have generated a better result.

"It felt like using a hammer to drive a screw—just because it’s popular, not because it’s right for the task."

He said overall he appreciated the transparency of the conference and it "helped break the taboo around using AI systems."

Business Information Systems senior lecturer Raffaele Ciriello from the University of Sydney told the AusSMC that while there was a discussion to be had about how AI can help researchers analyse data, visualise results and improve reproducibility, when it comes to writing papers, "no human should mistake this for scholarship."

"Equating AI agents with human scientists is a profound category error," Dr Ciriello said.

"Large language models do not think, discover, or know in any meaningful sense. They produce plausible sequences of words based on patterns in past data. Granting them authorship or reviewer status anthropomorphises what are essentially stochastic text-prediction machines."

Professor Daswin De Silva, Deputy Director of the Centre for Data Analytics and Cognition at La Trobe, said this anthropomorphisation is "belittling and devaluing" the purpose of peer-reviewed research.

"The responsible practice of AI is to refrain from attributing human characteristics to AI despite its capacity to generate intelligent and human-like output," he said.

"The same applies here, conducting research and then presenting the findings at a research conference are deeply human activities of peer-reviewed research, knowledge gain, discussion, collaboration, and networking."

Dr Salmon said he hoped those participating in the conference came to the same conclusion he had - "that is that AI may provide a useful tool to assist human researchers in their work, but, in its current form at least, is not capable of creating quality research outputs."

You can read the full Expert Reaction here

This article originally appeared in Science Deadline, a weekly newsletter from the AusSMC. You are free to republish this story, in full, with appropriate credit.

Contact: Rachel McDonald

Phone: 08 7120 8666

Email: info@smc.org.au