Pero Šipka,  Centre for Evaluation in Education and Science, Belgrade, Serbia

Speech at the conference  ‘Technologies Transforming Research Assessment’  on March 19th, 2014 in Vilnius, Lithuania.


Presentation      video    and     PDF


The concepts of Quality Control and Quality Assurance have found its application in various production domains including journal publishing. Quality Enforcement (QE) is a recent extension of the same basic idea. The Centre for Evaluation in Education and Science (CEON/CEES), a Belgrade-based R&D observatory, made this concept a guiding principle of its efforts to assist the development of locally published journals.

QE was envisaged as a set of instruments embedded in the process of preparing journal issues that enforce actors to comply with international publishing standards and good practices. Monitoring journals impact and usage were understood as a strong integral components of QE.

In CEON/CEES’s operationalization, QE include:
(1) a tool for automatic extraction of keywords from thesauri at editors choice
(2) an application for cross-checking citations given in reference lists with in-text citations
(3) a system for normalization of cited references according to citation style selected by the editor
(4) the support for plagiarism detection,
(5) a service for citation legitimacy surveillance to prevent impact manipulation,
(6) the routines for transforming manuscripts to XML to ensure publishing in both PDF and HTML format, and
(7) a device for monitoring manuscript selection and reviewing.

The concept o QE, as well as the design of instruments to support it, were derived from the results of a long-term monitoring of the impact and quality of journals published in Serbia and surrounding countries. The low formal, bibliometric quality resulting in low visibility and trustfulness, as evidenced by minor coverage by international databases and indexes, were recognized as their main weaknesses. It was our assumption that these can be efficiently amended by strict application of QE approach.

After the two-year experience with about 50 national journals the QE model was recently implemented within a service called SEESAmE: South East European Science Ameliorated through Evaluation.

The model seems to be promising not only for the region, but for DTES countries in general.



Pero Sipka

Pero Šipka, PhD, is a trained psychologist and psychometrician. He spent most of his career in teaching statistical, methodological and informetric courses at the University of Novi Sad, Serbia, where he is still active as PhD advisor. From 2002 on, he has been concurrently serving as President at the Centre for Evaluation in Education and Science, a Belgrade-based non-for profit research observatory working for the Government, as well as independently in trying to introduce modern research evaluation in Serbia.

His main research interests are in locally published journals, with the focus on relationship between their impact and formal bibliometric characteristics, including those evidencing misconduct in publishing. From this research he derived a model for national journals monitoring, ranking, and promotion. He is the architect of applications supporting the model: SCIndeks – the Serbian Citation Index, Journal Bibliometric Report and DOPISNIcA, a CERIF-modelled system integrating R&D information at national level.

He published a lot, mainly working papers but also conference and journal articles primarily of applied character, often in polemical form, pleading for open, accountable, data-driven, and verifiable evaluation.