PDFA event explores the path towards creating high impact science

On February 17th, the Postdoctoral Fellow Association (PDFA) of the RIKEN Brain Science Institute (BSI) held an event entitled “High Impact Science vs High Profile Journals,” to discuss how to scientific impact is evaluated, both in terms of the contributions made by individual researchers and the role that high profile journals, like Nature and Science, play in steering the progress of scientific fields.

Watch the video of the panel discussion

The event included introductory remarks by Charles Yokoyama, BSI’s Director of Research Administration, followed by a panel discussion with Susumu Tonegawa (BSI Director and Professor at MIT’s Picower Institute), Shun-Ichi Amari (BSI Senior Fellow and former BSI Director), and Michisuke Yuzaki (Professor, Keio University) and was facilitated by Erin Munro, member of the PDFA.

All scientists seek to find answers to questions in their field. But how they go about it—from the questions they ask to the approaches they take to find answers—varies greatly among fields and individual researchers. Today’s research environment is very competitive and the evaluation of scientists— whether applying for jobs, funding, or promotion—often centers on the number of publications they have, and which journals they were published in. However a debate has been brewing in recent years, about how to accurately evaluate scientific impact. Do high profile journals routinely capture novel and innovative science and accurately reflect the impact of an individual researcher’s science on a field? The “High Impact Science vs High Impact Journals” event was organized by the PDFA to promote discussion of this topic at BSI and to give young researchers a chance to discuss their questions and concerns with scientists who have made a successful career in science.

“Although we were concerned that discussing what qualifies as high impact science might be controversial, we decided that it is a topic that is relevant to a lot of researchers,” said Munro. “I hoped that people would come away from the workshop with the view that they can do meaningful science, a sense of how to do it and what it means in the context of publications and career.”

Charles Yokoyama, former editor of Neuron and current Director of Research Administration at BSI offered his perspective on how to create and evaluate high impact science.

Why is high impact science important? Yokoyama proposed that high impact science is a better use of society’s financial investment and of limited research time and that it is thought of as being more reliable and more ethical than low impact science due to more rigorous peer-review. He noted, however, that the definition of scientific impact varies depending on who is asking: while the Nobel committee might be looking for a study that opened a major knowledge domain with the benefit of hindsight, a journal editor is looking for studies with the potential to advance the field. Similarly, a faculty search committee or a funding agency would have different priorities and views on scientific impact.

Drawing on his experience, Yokoyama focused on the perspective of journal editors and how their decision-making on high impact science affects ongoing research. Editors evaluate the potential impact of individual papers by judging their conceptual advance, that is, whether a study advances a field in a large leap, rather than an incremental step. Yokoyama emphasized that the challenge for researchers who aim to publish in high impact journals is to collect strong data that is both grounded in existing knowledge but linked to a new idea or model. “Data alone provide the foundation for an advance but without communicating how the results fit or refute an existing model they fail to provide a conceptual advance. Communication is essential for scientific impact,” said Yokoyama.

High profile journals have pros and cons. The advantages include a strong and consistent editorial evaluation system for assessing conceptual advance in scientific research, increased visibility to scientists from other fields and the general public. The disadvantages include that without the benefit of hindsight, important but preliminary or atypical studies may be rejected, and that in competitive fields, large, well funded labs have a large advantage. However, high profile journals often provide the momentum to keep fields moving forward.

In closing, Yokoyama offered advice on how to create high impact research, suggesting that researchers study their field and identify novel questions with potential for large advances. Also, it is useful to understand one’s scientific strengths and weaknesses and seek collaborations or create teams to fill the gaps. He also suggested that at higher levels of management, institutions can facilitate high impact research by creating research teams with complementary expertise, and evaluating scientists based on their contributions to the team.

The panel discussion covered how to choose a project, discovery based vs hypothesis based science, when to give up unsuccessful projects, dealing with competition and keeping your edge in a competitive environment. The panelists also offered their views on how to succeed as a researcher. “The goal should be to produce high impact science,” said Tonegawa. “It is your science that gives high impact to a journal, not the other way around.”

The questions from the audience centered around how panelists choose projects to pursue. They agreed that one’s interests should be the guiding principle for choosing projects, although their opinions differed about what other factors are important to consider, such as research funding, available technology, and personal strengths. “Try to address a fundamental question,” advised Tonegawa. “If the project works and you publish it, how long will the conceptual discovery last? Will it appear in a textbook?”

Yuzaki added, “I think it depends on the level of your scientific career. Established lab heads can pursue more risky projects, while postdocs may need to chose more carefully. You need to balance higher risk projects with more solid contributions that can be published in a few years.” Amari added, “I like the approach of digging deep down to explore a problem, but just doing that alone is not enough, Alongside, it is good to do something easier, with a higher probability of success.”

Tonegawa continued “If you are a scientist you are the same as an artist or a musician. You chose to do something you really like, so you should continue doing what you like and not worry about how long a project is going to take.”
On the topic of competition, Tonegawa noted that “Your interest in science should outweigh your fear of competition.”

The session concluded with Tonegawa advising young scientists to work closely with someone who has done great science and observe how they do science in terms of choosing a specific question, experimentally prove their hypothesis, and the level of rigor they apply in order to conclude something. “Watch the way they do science, and try to imitate it in your own way.” Tonegawa shared his own experience working with his mentor Renato Dulbecco. “He had a vision of opening up new field. Most successful scientists do that—think about the future.”

Written by Alexandra V Terashima