Thursday, August 11, 2022

How many sources are needed? The effects of bibliographic databases on systematic review outcomes

An interesting preprint, which takes articles which were identified in german systematic reviews of educational topics, and identifies which of 7 databases contain the articles (the databases were: Catalogue of the German National Library; Education Research Complete; ERIC; FIS Bildung Literaturdatenbank/ German Education Index; Google Scholar; LearnTechLib; Web of Science Social Science Citation Index).
- Keller, C., Heck, T. & Rittberger, M. (2022). How many sources are needed? The effects of bibliographic databases on systematic review outcomes. in: Aizawa, A., Mandl, T., Carevic, ., Hinze, A.,Mayr, P. & Schaer, P. (Eds). Proceedings of the ACM/IEEE Joint Conference on Digital Libraries in 2022 (JCDL '22), hybrid conference, Cologne, Germany and online, June 20 - 24, 2022. New York : Association for Computing Machinery.
"The database coverage showed high variations and clearly indicates that one source on its own does not cover a sufficient amount of relevant literature. Some databases are very similar in coverage, while national and discipline-specific databases hold publications that cannot be found elsewhere. Google Scholar outperformed all databases regarding recall. However, due to poor precision this database is considered inadequate for review purposes" - the latter sentence highlights that Google Scholar yields by far the largest % of the articles - but although they are there, you wouldn't necessarily find them if you were using a subject (rather than a known item) search. In the original search some of these items could only be found through hand searching. There are also observations on why the authors of the original systematic reviews did not find items in other databases, when the items were in fact there. This includes the poor quality of metadata, and sometimes over-specific searches.
Photo by Sheila Webber: dry heath, summer 2018

No comments: