Reconciling open science practices and research assessment requirements in the process of establishing a national CRIS system in Serbia

Open science promotes transparency, open access, and reproducibility in research, ensuring that research outputs are openly available and reusable by others. However, traditional research assessment systems usually emphasize quantitative indicators, such as journal impact factors and

citation counts, encouraging publication in ‘high-prestige’ journals and failing to fully capture the value of open science practices. Accordingly, a Current Research Information System (CRIS) tracking research outputs for assessment purposes does not necessarily encourage open science practices. This presentation explores the challenges and strategies involved in reconciling open science practices with research assessment requirements in Serbia during the process of establishing the national CRIS system eNauka.

Over the past decade, the Serbian research community has made significant efforts to implement the principles of open science. This process has intensified since 2018, following the adoption of the Open Science Platform, which introduced a green open mandate, leading to the development of more than 80 institutional repositories within five years, most of which ensure compliance with the FAIR principles and are harvested by international aggregators (Kosanović & Ševkušić, 2021). Research organizations have recognized the role of repositories in preserving research results and ensuring wide dissemination and improved visibility through open access. Intensive informal training provided by librarians (Đorđević et al., 2021) and community development (Open Science Community, 2021) have been instrumental in raising awareness and improving the understanding of open science practices among researchers.

In 2022, the ministry responsible for science started developing an information system to track research outputs in Serbia for reporting and assessment purposes, eNauka (eScience) portal (Kosanović, 2023). The portal provides registries of all accredited research organizations and all researchers in Serbia, as well as a database containing metadata about their research outputs, which predominantly include publications, patents and technical reports. Administrative data about organizations and researchers are curated by the organizations’ administrative staff and the ministry personnel. Metadata describing research outputs are harvested (e.g. Cobiss OPAC) or imported (Crossref, ORCID profiles) from various sources, but institutional repositories are the main data source. Metadata records in eNauka are harvested once a week and curated by officially appointed individuals from research organizations, usually librarians and repository managers. Thanks to this, the establishment of the national CRIS system additionally encouraged the development of institutional repositories. However, this process has been accompanied with some challenges and has threatened to disrupt progress in adopting open science practices.

Here is a brief overview of the major challenges:

• One of the functions of eNauka is to facilitate research assessment. Along with publications, patents, technical reports, artworks and similar outputs that can be deposited in a repository, the national regulations recognize a number of results the evidence of which (e.g. signed certificates, invitation letters, diplomas) is typically not deposited in a repository. Due to this, repository managers have faced pressure to allow depositing materials not allowed by repository policies.
• A number of research organizations established repositories with the sole idea of feeding data into the national CRIS system. In such cases, repositories provided only metadata and it was practically impossible to persuade researchers to deposit full-text content, let alone to make it open access.
• In a number of well-functioning repositories good depositing practices were disrupted, as a number of researchers refused to provide full-text content. They either only provided metadata or deposited the front pages of their publications and insisted on depositing inappropriate materials.
• A major challenge was the pressure to align institutional repositories, using standard metadata schemas and vocabularies, with the non-standard classifications used by the ministry.

The initiative to deal with these challenges came from the library community and the ministry accepted the request to establish a working group to develop guidelines for metadata curation in eNauka. It included librarians, i.e. repository managers, ministry analysts, the administrators of the eNauka portal, and ministry representatives. Their task was to harmonize the requirements of all stakeholders towards facilitating research assessment while maintaining the integrity of repositories and fostering open science practices.

The working group produced two sets of guidelines (adopted in March 2024): guidelines for metadata curation in eNauka (Ministarstvo nauke, 2024b) explaining how to describe research outputs using standardized metadata in eNauka, and guidelines for repository managers (Ministarstvo nauke, 2024a), detailing the metadata and providing clear instructions regarding full-text deposition. These guidelines make it possible to meet the assessment requirements, while empowering the managers of institutional repositories to spread awareness at the level of their organization about the multiple benefits of open science. In this respect, the guidelines are particularly valuable for newly established repositories.

The fact that the national CRIS system relies on repository data opens up many possibilities for the recognition and tracking of open research practices in research assessment. However, there are still many challenges to overcome, such as the lack of awareness and understanding of open science practices among researchers, ministry analysts and national policy makers. Many researchers are not fully aware of the benefits of open science or how to effectively implement open practices in their work. Similarly, ministry analysts lack the knowledge and tools needed to evaluate the full spectrum of research outputs that open science encompasses.

Cultural resistance within the academic community also poses a barrier. Established norms and practices in research and assessment are deeply ingrained, and resistance to change is widespread in the local research community. Researchers are hesitant to embrace open science practices if they perceive them as risky, believe that these practices might negatively impact their career prospects, or simply if they get no reward for them.

Policy reforms are crucial in integrating open science practices into research assessment. Funding agencies, universities, and research institutions should adopt policies that recognize and reward open science contributions.

KEYWORDS 

CRIS systems; institutional repositories; open science; research assessment

REFERENCES

• Đorđević, A., Nježić, I., Ševkušić, M., & Kosanović, B. (2021). Building capacity for Open Science through institutional repository training. Zenodo. https://doi.org/10.5281/zenodo.7315389
• Kosanović, B. (2023). eNauka—CRIS in Serbia. Infotheca, 23(2), 83–100. https://doi.org/10.18485/infotheca.2023.23.2.5
• Kosanović, B., & Ševkušić, M. (2021). Beyond Open Access mandates: Institutional repositories as building blocks in open scholarly communication. Zenodo. https://doi.org/10.5281/zenodo.5786272
• Ministarstvo nauke, tehnološkog razvoja i inovacija R. S. (2024a). Smernice za NIO koje informacionom sistemu eNauka dostavljaju podatke o rezultatima naučnog rada. https://enauka.gov.rs/regres/#/resource/e274ff6d-2901-4ad6-abe2-1c8399f36328
• Ministarstvo nauke, tehnološkog razvoja i inovacija R. S. (2024b). Smernice za NIO urednike. http://media.rcub.bg.ac.rs/wp-content/uploads/wp-uploads/2024/03/Smernice_za_NIO_urednike_finalna_verzija-1.pdf