The Literature Review Seminar

Goals and types of reviews

• Familiarize ourselves with different forms of reviews
• Dissociate goals and review types, along with their dimensions
• Appreciate the key elements of a review protocol

Forms of literature reviews

Generally, literature reviews can take different forms, including

• a standalone review paper
• a background section of a primary study (e.g., related work, theoretical background, conceptual foundations)
• a mandatory part of a funding proposal
• a part of a Ph.D. thesis
• a review protocol (e.g., published in a conference or a registry)

Our focus is on standalone reviews: an "independent paper whose purpose is to synthesize the extant literature in a field without collection of empirical data" (Templier and Paré 2018).

Literature reviews in information systems

• Early editorial by Webster and Watson (2002, MISQ, > 12.000 citations) with key suggestions

  • Rigorous search, including forward and backward searches
  • Concept matrix
  • Concept-centric writing instead of author-centric summaries

• Establishment of the Theory and Review Department at MISQ, and similar initiatives at JAIS, and JSIS

• Active discourse covering typological pluralism, systematicity, and transparency across top journals in Information Systems

Goals of literature reviews

Building on Gregor (2006), Rowe (2014) distinguishes four goals of literature reviews:

• Describing: summarizing or classifying prior research on a phenomenon with little or no contribution to theory, i.e., without discussing underlying assumptions or rationales
• Understanding: making sense of prior literature and new phenomena, working towards a conceptual understanding, and often involving an in-depth, broad, or critical discussion before drawing synthetic conclusions
• Explaining: drawing on the literature to develop a conceptual framework, or theory with testable hypotheses, i.e., statements that explain real-world phenomena, and can be tested empirically
• Theory testing: extracting data from empirical studies to assess the aggregated evidence that has accumulated

Task: Propose a research question that exemplifies each goal.

The typology of Paré et al. (2015)

• Paré et al. (2015) published the established typology of literature reviews, covering 9 distinct types of reviews (plus hybrid reviews)
• The review types can be aligned with the goals of Rowe (2014), as indicated in the table (Templier and Paré 2018)

Task: Familiarize yourself with the types of reviews in the typology of Paré et al. 2015.

Exercise: Reading circle

Task: Classify the selection of review papers according to the typology of Paré et al. (2015):

• Bélanger and Crossler (2011)
• King and He (2006)
• Kitsiou et al. (2017)
• Melville et al. (2004)
• Otte-Trojel et al. (2016)
• Petter et al. (2008)
• Powell et al. (2004)
• Seuring (2013)
• Shim et al. (2022)

Zhao and Zhu (2014) and Carillo and Beaudry (2006) may also be covered (depending on group size).

Dimensions (Cooper's taxonomy)

Central aspect: Internal coherence

Exercise: Select the review type

Form groups according to the goals with regard to theory (describing, understanding, explaining, testing).

Task: Draft the research question, select a review type and explain why it fits.

Discussion: Share your draft and discuss whether there is a high fit.

The RightReview tool

The RightReview tool provides a survey of literature review dimensions to identify the right review type for your project.

Task: Complete the survey and check your results.

• Do you agree? Are there any dimensions that are unclear?

What can we publish from a review project?

• Review protocol: Open Science Foundation (OSF), PROSPERO, conference proceedings

• Search strategy: searchRxiv

• Options to publish standalone review papers:

  • Conference proceedings (e.g., ICIS track for literature reviews)
  • Specialized journals for literature reviews (e.g., International Journal of Management Reviews, Foundations and Trends® in [Information Systems, Entrepreneurship, Management, Marketing,...])
  • Journals with theory and review departments (e.g., Journal of the AIS)
  • Review special issues (e.g., Journal of Strategic Information Systems)
  • Regular submission to journals

• Review dataset: OSF, SYNERGY datasets

Review protocols: Their purposes

• Make concrete plans for a review project
• Pilot-test the feasibility and potential contribution
• Solicit feedback from peers
• Publish the protocol to signal your work (e.g., in a registry like PROSPERO or at OSF)

Review protocols: Conceptions

• A plan that is developed a priori and fixed. The protocol regulates researchers and requires them to follow an inflexible set of rules. Fixed protocols are more common in the health sciences and are often associated with theory-testing reviews like meta-analyses.
• A “living plan”. The protocol is a living document that is used as a work log that records how the review project has evolved over time. Researchers may deviate from their original plan. Protocol development is thus iterative.

Review protocols: Structure and contents (I)

• Title (indicating that this is a review protocol)

• Plain or structured abstract (written at the end) and keywords (up to 5)

• Introduction section (WHAT and WHY)

  • Generativity statements
  • Brief introduction of the topic, phenomenon, and theory of interest
  • Rationale for the review
  • Review objectives (and research questions)
  • Scope of the review
  • Expected contributions of the review

• Background section (WHAT / if applicable)

  • Definition of key concepts/presentation of the framework to be used to organize the review, presentation of the theory at the center of the review, etc.

Review protocols: Structure and contents (II)

• Methods

  • Type of review and justification
  • Search strategy and procedures
  • Screening strategy and procedures
  • Quality appraisal strategy and procedures
  • Data extraction strategy and procedures
  • Data analysis/synthesis strategy and procedures

• Expected results:

  • Structure of the synthesis (first-level headings)
  • Proposed presentation of synthesis (e.g., in the form of a table or figure)

Review protocols: Structure and contents (III)

• Conclusion

  • Methodological limitations
  • Potential implications for research, practice, and/or policymaking

• Appendices (WHO, HOW, WHEN)

  • List of contributors and their responsibilities
  • Detailed timetable
  • List of software tools used for what purposes
  • Intended publication plan (protocol publication/registration, strategy for reporting and publishing the full review article)
  • Other declarations (funding sources, etc.)

Where can we publish a review?

Journals with (theory and) review departments and/or regular calls for reviews

• Journal of Strategic Information Systems (see cfps)
• Journal of the Association for Information Systems (see literature review category)
• European Journal for Information Systems (see review article category)
• MIS Quarterly (see theory and review category)

Conferences

• International Conference for Information Systems (see literature review track)

Journals which publish only review articles

• Academy of Management Review (link), Academy of Marketing Science Review (link)
• International Journal of Management Reviews (link), Human Resource Development Review (link)
• Annual Review of Organizational Psychology and Organizational Behavior (link)
• Foundations and Trends in … [accounting, entrepreneurship, finance, information systems, management, marketing]

Until next session

Task: Explore the literature related to your topic and identify

1. Related review papers that should be mentioned in your review (protocol)
2. A review paper that exemplifies your chosen review type, which may serve as an inspiration for your work

Prepare to give an overview of the related review papers and explain what is particularly compelling in the exemplary review paper.

References

Goals and review types

Rowe, F. (2014). What literature review is not: diversity, boundaries and recommendations. European Journal of Information Systems, 23(3), 241-255. doi:10.1057/ejis.2014.7

Paré, G., Trudel, M. C., Jaana, M., & Kitsiou, S. (2015). Synthesizing information systems knowledge: A typology of literature reviews. Information & Management, 52(2), 183-199. doi:10.1016/j.im.2014.08.008

Templier, M., & Pare, G. (2018). Transparency in literature reviews: an assessment of reporting practices across review types and genres in top IS journals. European Journal of Information Systems, 27(5), 503-550. doi:10.1080/0960085X.2017.1398880

Sample classified according to review types

Bélanger, F., & Crossler, R. E. (2011). Privacy in the digital age: a review of information privacy research in information systems. MIS Quarterly, 35(4) 1017-1041. doi:10.2307/41409971

King, W. R., & He, J. (2006). A meta-analysis of the technology acceptance model. Information & Management, 43(6), 740-755. doi:10.1016/j.im.2006.05.003

Kitsiou, S., Paré, G., Jaana, M., & Gerber, B. (2017). Effectiveness of mHealth interventions for patients with diabetes: an overview of systematic reviews. PloS one, 12(3), e0173160. doi:10.1371/journal.pone.0173160

Melville, N., Kraemer, K., & Gurbaxani, V. (2004). Information technology and organizational performance: An integrative model of IT business value. MIS Quarterly, 28(2), 283-322. doi:10.2307/25148636

Otte-Trojel, T., de Bont, A., Rundall, T. G., & van de Klundert, J. (2014). How outcomes are achieved through patient portals: a realist review. Journal of the American Medical Informatics Association, 21(4), 751-757. doi:10.1136/amiajnl-2013-002501

Petter, S., DeLone, W., & McLean, E. (2008). Measuring information systems success: models, dimensions, measures, and interrelationships. European Journal of Information Systems, 17, 236-263. doi:10.1057/ejis.2008.15

Powell, A., Piccoli, G., & Ives, B. (2004). Virtual teams: a review of current literature and directions for future research. ACM SIGMIS Database: the DATABASE for Advances in Information Systems, 35(1), 6-36. doi:10.1145/968464.968467

Seuring, S. (2013). A review of modeling approaches for sustainable supply chain management. Decision Support Systems, 54(4), 1513-1520. doi:10.1016/j.dss.2012.05.053

Shim, J. P., Warkentin, M., Courtney, J. F., Power, D. J., Sharda, R., & Carlsson, C. (2002). Past, present, and future of decision support technology. Decision Support Systems, 33(2), 111-126. doi:10.1016/S0167-9236(01)00139-7

Dimensions

Cooper, H. M. (1988). Organizing knowledge syntheses: A taxonomy of literature reviews. Knowledge in Society, 1(1), 104. doi:10.1007/BF03177550

Protocols

Paré, G., Wagner, G., & Prester, J. (2023). How to develop and frame impactful review articles: key recommendations. Journal of Decision Systems, 1-17. doi:10.1080/12460125.2023.2197701