Abstract
In the tradition of Moreno and Swales (2018), this paper presents the creation of a manually annotated resource for supporting teaching English for Legal Purposes (ELP) and for Natural Language Processing (NLP) purposes. After justifying the use of Supreme Court of the United States opinions, we define our coding scheme by adapting the move model of rhetorical structure in specialized discourse. We describe the methodology and the implementation of the annotation campaign. We analyze how our methodology and the resulting annotation scheme diverge from those described in the literature as well as the advantages that these divergences afford. In addition to the research article, we release several supplementary materials which aim to make the process transparent and serve other researchers aiming to annotate specialized discourse with the help of machine learning techniques.
References
Authors (2024a).
Authors (2024b).
Authors (In Press).
Ahmed, M., Seraj, R., & Islam, S. M. S. (2020). The k-means algorithm: A comprehensive survey and performance evaluation. Electronics, 9(8), 1295. https://doi.org/10.3390/electronics9081295
Anthony, L., & Lashkia, G. V. (2003). Mover: A machine learning tool to assist in the reading and writing of technical papers. IEEE transactions on professional communication, 46(3), 185 193. https://doi.org/10.1109/tpc.2003.816789
Artstein, R., & Poesio, M. (2008). Inter-coder agreement for computational linguistics. Computational linguistics, 34(4), 555 596. https://doi.org/10.1162/coli.07-034-r2
Asher, N., & Lascarides, A. (2003). Logics of conversation. Cambridge University Press.
Bhatia, V. K. (1993). Analysing genre: Language use in professional settings. Routledge. https://www.taylorfrancis.com/books/mono/10.4324/9781315844992/analysing-genre-bhatia
Biber, D., Connor, U., & Upton, T. A. (2007). Discourse on the Move: Using corpus analysis to describe discourse structure (Vol. 28). John Benjamins Publishing Company. https://doi.org/10.1075/scl.28
Breeze, R. (2009). Issues of persuasion in academic law abstracts. Revista Alicantina de Estudios Ingleses, 22, 11 26. http://dx.doi.org/10.14198/raei.2009.22.02
Bres, J., Nowakowska, A., & Sarale, J.-M. (2016). Anticipative interlocutive dialogism: Sequential patterns and linguistic markers in French. Journal of pragmatics, 96, 80 95. https://doi.org/10.1016/j.pragma.2016.02.007
Casal, J. E., & Kessler, M. (2023). Rhetorical move-step analysis. In Conducting Genre-Based Research in Applied Linguistics (pp. 82-104). Routledge. https://doi.org/10.4324/9781003300847-7
Cotos, E., Huffman, S., & Link, S. (2015). Furthering and applying move/step constructs: Technology-driven marshalling of Swalesian genre theory for EAP pedagogy. Journal of English for Academic Purposes, 19, 52 72. https://doi.org/10.1016/j.jeap.2015.05.004
Cotos, E., Huffman, S., & Link, S. (2017). A move/step model for methods sections: Demonstrating Rigour and Credibility. English for Specific Purposes, 46, 90 106. https://doi.org/10.1016/j.esp.2017.01.001
Dayrell, C., Candido Jr, A., Lima, G., Machado Jr, D., Copestake, A. A., Feltrim, V. D., Tagnin, S. E., & Aluísio, S. M. (2012). Rhetorical Move Detection in English Abstracts: Multi-label Sentence Classifiers and their Annotated Corpora. LREC, 1604 1609. https://comet.fflch.usp.br/sites/comet.fflch.usp.br/files/u30/RHETORICAL.pdf
Dudley-Evans, T. (2002). Genre analysis: An approach to text analysis for ESP. In Advances in written text analysis (p. 233 242). Routledge. https://www.taylorfrancis.com/chapters/edit/10.4324/9780203422656-17/genre-analysis-approach-text-analysis-esp-tony-dudley-evans
Fort, K. (2012). Les ressources annotées, un enjeu pour l’analyse de contenu : Vers une méthodologie de l’annotation manuelle de corpus [PhD Thesis, Université Paris-Nord-Paris 13]. https://theses.hal.science/tel-00797760/
Gozdz-Roszkowski, S. (2020). Move Analysis of Legal Justifications in Constitutional Tribunal Judgments in Poland: What They Share and What They Do Not. International Journal for the Semiotics of Law - Revue Internationale de Sémiotique Juridique, 33(3), 581 600. https://doi.org/10.1007/s11196-020-09700-1
Groom, N., & Grieve, J. (2019). The evolution of a legal genre. Corpus-based research on variation in English legal discourse, 91. https://doi.org/10.1075/scl.91.09gro
Han, Z. (2011). The discursive construction of civil judgments in Mainland China. Discourse & Society, 22(6), 743 765. https://doi.org/10.1177/0957926511419924
Han, Z., Bhatia, V. K., & Ge, Y. (2018). The structural format and rhetorical variation of writing Chinese judicial opinions: A genre analytical approach. Pragmatics. Quarterly Publication of the International Pragmatics Association (IPrA), 28(4), 463 488. https://doi.org/10.1075/prag.17013.ge
Joty, S., Carenini, G., Ng, R., & Murray, G. (2019). Discourse analysis and its applications. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: Tutorial Abstracts, 12 17. https://aclanthology.org/P19-4003/
Kalamkar, P., Tiwari, A., Agarwal, A., Karn, S., Gupta, S., Raghavan, V., & Modi, A. (2022). Corpus for Automatic Structuring of Legal Documents (No. arXiv:2201.13125). arXiv. http://arxiv.org/abs/2201.13125
Kanoksilapatham, B. (2008). Rhetorical moves in biochemistry research articles. In Discourse on the move: Using corpus analysis to describe discourse structure (p. 73 119). John Benjamins Publishing Company. https://doi.org/10.1075/scl.28.06kan
Kim, M., Qiu, X., & Wang, Y. (Arthur). (2024). Interrater agreement in genre analysis: A methodological review and a comparison of three measures. Research Methods in Applied Linguistics, 3(1), 100097. https://doi.org/10.1016/j.rmal.2024.100097
Kirby-Légier, C. (2005). Understanding the judicial discourse of the current United States Supreme Court. La Langue, le discours et la culture en anglais du droit. Paris: Publications de la Sorbonne, 87 110.
Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 159 174.
Larsson, T., Plonsky, L., Sterling, S., Kytö, M., Yaw, K., & Wood, M. (2023). On the frequency, prevalence, and perceived severity of questionable research practices. Research Methods in Applied Linguistics, 2(3), 100064. https://doi.org/10.1016/j.rmal.2023.100064
Le, T. N. P., & Pham, M. M. (2020). Genre practices in mechanical engineering academic articles. Ibérica, 39, 243 266. https://doi.org/10.17398/2340-2784.39.243
Mann, W. C., & Thompson, S. A. (1988). Rhetorical Structure Theory: Toward a functional theory of text organization. Text - Interdisciplinary Journal for the Study of Discourse, 8(3). https://doi.org/10.1515/text.1.1988.8.3.243
Mazzi, D. (2007). The construction of argumentation in judicial texts: Combining a genre and a corpus perspective. Argumentation, 21(1), 21 38. https://doi.org/10.1007/s10503-007-9020-8
Monreal, C. S. (2016). A move-step analysis of the concluding chapters in computer science PhD theses. Ibérica, 32, 105 132. Retrieved from https://revistaiberica.org/index.php/iberica/article/view/175
Moreno, A. I., & Swales, J. M. (2018). Strengthening move analysis methodology towards bridging the function-form gap. English for Specific Purposes, 50, 40 63. https://doi.org/10.1016/j.esp.2017.11.006
Muller, P., Vergez-Couret, M., Prévot, L., Asher, N., Farah, B., Bras, M., Le Draoulec, A., & Vieu, L. (2012). Manuel d’annotation en relations de discours du projet Annodis. http://www.irit.fr/~Philippe.Muller/perso_utf8_bib.html
Rau, G., & Shih, Y.-S. (2021). Evaluation of Cohen’s kappa and other measures of inter-rater agreement for genre analysis and other nominal data. Journal of English for Academic Purposes, 53, 101026. https://doi.org/10.1016/j.jeap.2021.101026
Salager-Meyer, F. (1992). A text-type and move analysis study of verb tense and modality distribution in medical English abstracts. English for Specific Purposes, 11(2), 93 113.
Swales, J. M. (1990). Genre analysis. Cambridge university press.
Swales, J. M. (2004). Research genres: Explorations and applications. Cambridge University Press.
Tessuto, G. (2021). Making sense of web-based European Court of Justice institutional press releases: Context, structure and replicable genres. Ibérica, 42, Article 42. https://doi.org/10.17398/2340-2784.42.219
Teufel, S., Carletta, J., & Moens, M. (1999). An annotation scheme for discourse-level argumentation in research articles. Proceedings of the Ninth Conference on European Chapter of the Association for Computational Linguistics, 110. https://doi.org/10.3115/977035.977051
Tkachenko, M., Malyuk, M., Holmanyuk, A., & Liubimov, N. (2020). Label Studio: Data labeling software. Open source software available at https://labelstud.io.
van Geel, T. R. (2009). Understanding Supreme Court Opinions (6th Edition). Routledge, Taylor & Francis Group.
Yaich, M., & Hernandez, N. (2025). Improving Accessibility of SCOTUS Opinions: A Benchmark Study and a New Dataset for Generic Heading Prediction and Specific Heading Generation.
Yang, R., Xu, L., & Swales, J. M. (2023). Tracing the development of English for Specific Purposes over four decades (1980–2019): A bibliometric analysis. English for Specific Purposes, 71, 149 160. https://doi.org/10.1016/j.esp.2023.04.004
Yu, D., Bondi, M., & Hyland, K. (2024). Can GPT-4 learn to analyse moves in research article abstracts? Applied Linguistics, amae071. https://doi.org/10.1093/applin/amae071
Authors (2024b).
Authors (In Press).
Ahmed, M., Seraj, R., & Islam, S. M. S. (2020). The k-means algorithm: A comprehensive survey and performance evaluation. Electronics, 9(8), 1295. https://doi.org/10.3390/electronics9081295
Anthony, L., & Lashkia, G. V. (2003). Mover: A machine learning tool to assist in the reading and writing of technical papers. IEEE transactions on professional communication, 46(3), 185 193. https://doi.org/10.1109/tpc.2003.816789
Artstein, R., & Poesio, M. (2008). Inter-coder agreement for computational linguistics. Computational linguistics, 34(4), 555 596. https://doi.org/10.1162/coli.07-034-r2
Asher, N., & Lascarides, A. (2003). Logics of conversation. Cambridge University Press.
Bhatia, V. K. (1993). Analysing genre: Language use in professional settings. Routledge. https://www.taylorfrancis.com/books/mono/10.4324/9781315844992/analysing-genre-bhatia
Biber, D., Connor, U., & Upton, T. A. (2007). Discourse on the Move: Using corpus analysis to describe discourse structure (Vol. 28). John Benjamins Publishing Company. https://doi.org/10.1075/scl.28
Breeze, R. (2009). Issues of persuasion in academic law abstracts. Revista Alicantina de Estudios Ingleses, 22, 11 26. http://dx.doi.org/10.14198/raei.2009.22.02
Bres, J., Nowakowska, A., & Sarale, J.-M. (2016). Anticipative interlocutive dialogism: Sequential patterns and linguistic markers in French. Journal of pragmatics, 96, 80 95. https://doi.org/10.1016/j.pragma.2016.02.007
Casal, J. E., & Kessler, M. (2023). Rhetorical move-step analysis. In Conducting Genre-Based Research in Applied Linguistics (pp. 82-104). Routledge. https://doi.org/10.4324/9781003300847-7
Cotos, E., Huffman, S., & Link, S. (2015). Furthering and applying move/step constructs: Technology-driven marshalling of Swalesian genre theory for EAP pedagogy. Journal of English for Academic Purposes, 19, 52 72. https://doi.org/10.1016/j.jeap.2015.05.004
Cotos, E., Huffman, S., & Link, S. (2017). A move/step model for methods sections: Demonstrating Rigour and Credibility. English for Specific Purposes, 46, 90 106. https://doi.org/10.1016/j.esp.2017.01.001
Dayrell, C., Candido Jr, A., Lima, G., Machado Jr, D., Copestake, A. A., Feltrim, V. D., Tagnin, S. E., & Aluísio, S. M. (2012). Rhetorical Move Detection in English Abstracts: Multi-label Sentence Classifiers and their Annotated Corpora. LREC, 1604 1609. https://comet.fflch.usp.br/sites/comet.fflch.usp.br/files/u30/RHETORICAL.pdf
Dudley-Evans, T. (2002). Genre analysis: An approach to text analysis for ESP. In Advances in written text analysis (p. 233 242). Routledge. https://www.taylorfrancis.com/chapters/edit/10.4324/9780203422656-17/genre-analysis-approach-text-analysis-esp-tony-dudley-evans
Fort, K. (2012). Les ressources annotées, un enjeu pour l’analyse de contenu : Vers une méthodologie de l’annotation manuelle de corpus [PhD Thesis, Université Paris-Nord-Paris 13]. https://theses.hal.science/tel-00797760/
Gozdz-Roszkowski, S. (2020). Move Analysis of Legal Justifications in Constitutional Tribunal Judgments in Poland: What They Share and What They Do Not. International Journal for the Semiotics of Law - Revue Internationale de Sémiotique Juridique, 33(3), 581 600. https://doi.org/10.1007/s11196-020-09700-1
Groom, N., & Grieve, J. (2019). The evolution of a legal genre. Corpus-based research on variation in English legal discourse, 91. https://doi.org/10.1075/scl.91.09gro
Han, Z. (2011). The discursive construction of civil judgments in Mainland China. Discourse & Society, 22(6), 743 765. https://doi.org/10.1177/0957926511419924
Han, Z., Bhatia, V. K., & Ge, Y. (2018). The structural format and rhetorical variation of writing Chinese judicial opinions: A genre analytical approach. Pragmatics. Quarterly Publication of the International Pragmatics Association (IPrA), 28(4), 463 488. https://doi.org/10.1075/prag.17013.ge
Joty, S., Carenini, G., Ng, R., & Murray, G. (2019). Discourse analysis and its applications. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: Tutorial Abstracts, 12 17. https://aclanthology.org/P19-4003/
Kalamkar, P., Tiwari, A., Agarwal, A., Karn, S., Gupta, S., Raghavan, V., & Modi, A. (2022). Corpus for Automatic Structuring of Legal Documents (No. arXiv:2201.13125). arXiv. http://arxiv.org/abs/2201.13125
Kanoksilapatham, B. (2008). Rhetorical moves in biochemistry research articles. In Discourse on the move: Using corpus analysis to describe discourse structure (p. 73 119). John Benjamins Publishing Company. https://doi.org/10.1075/scl.28.06kan
Kim, M., Qiu, X., & Wang, Y. (Arthur). (2024). Interrater agreement in genre analysis: A methodological review and a comparison of three measures. Research Methods in Applied Linguistics, 3(1), 100097. https://doi.org/10.1016/j.rmal.2024.100097
Kirby-Légier, C. (2005). Understanding the judicial discourse of the current United States Supreme Court. La Langue, le discours et la culture en anglais du droit. Paris: Publications de la Sorbonne, 87 110.
Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 159 174.
Larsson, T., Plonsky, L., Sterling, S., Kytö, M., Yaw, K., & Wood, M. (2023). On the frequency, prevalence, and perceived severity of questionable research practices. Research Methods in Applied Linguistics, 2(3), 100064. https://doi.org/10.1016/j.rmal.2023.100064
Le, T. N. P., & Pham, M. M. (2020). Genre practices in mechanical engineering academic articles. Ibérica, 39, 243 266. https://doi.org/10.17398/2340-2784.39.243
Mann, W. C., & Thompson, S. A. (1988). Rhetorical Structure Theory: Toward a functional theory of text organization. Text - Interdisciplinary Journal for the Study of Discourse, 8(3). https://doi.org/10.1515/text.1.1988.8.3.243
Mazzi, D. (2007). The construction of argumentation in judicial texts: Combining a genre and a corpus perspective. Argumentation, 21(1), 21 38. https://doi.org/10.1007/s10503-007-9020-8
Monreal, C. S. (2016). A move-step analysis of the concluding chapters in computer science PhD theses. Ibérica, 32, 105 132. Retrieved from https://revistaiberica.org/index.php/iberica/article/view/175
Moreno, A. I., & Swales, J. M. (2018). Strengthening move analysis methodology towards bridging the function-form gap. English for Specific Purposes, 50, 40 63. https://doi.org/10.1016/j.esp.2017.11.006
Muller, P., Vergez-Couret, M., Prévot, L., Asher, N., Farah, B., Bras, M., Le Draoulec, A., & Vieu, L. (2012). Manuel d’annotation en relations de discours du projet Annodis. http://www.irit.fr/~Philippe.Muller/perso_utf8_bib.html
Rau, G., & Shih, Y.-S. (2021). Evaluation of Cohen’s kappa and other measures of inter-rater agreement for genre analysis and other nominal data. Journal of English for Academic Purposes, 53, 101026. https://doi.org/10.1016/j.jeap.2021.101026
Salager-Meyer, F. (1992). A text-type and move analysis study of verb tense and modality distribution in medical English abstracts. English for Specific Purposes, 11(2), 93 113.
Swales, J. M. (1990). Genre analysis. Cambridge university press.
Swales, J. M. (2004). Research genres: Explorations and applications. Cambridge University Press.
Tessuto, G. (2021). Making sense of web-based European Court of Justice institutional press releases: Context, structure and replicable genres. Ibérica, 42, Article 42. https://doi.org/10.17398/2340-2784.42.219
Teufel, S., Carletta, J., & Moens, M. (1999). An annotation scheme for discourse-level argumentation in research articles. Proceedings of the Ninth Conference on European Chapter of the Association for Computational Linguistics, 110. https://doi.org/10.3115/977035.977051
Tkachenko, M., Malyuk, M., Holmanyuk, A., & Liubimov, N. (2020). Label Studio: Data labeling software. Open source software available at https://labelstud.io.
van Geel, T. R. (2009). Understanding Supreme Court Opinions (6th Edition). Routledge, Taylor & Francis Group.
Yaich, M., & Hernandez, N. (2025). Improving Accessibility of SCOTUS Opinions: A Benchmark Study and a New Dataset for Generic Heading Prediction and Specific Heading Generation.
Yang, R., Xu, L., & Swales, J. M. (2023). Tracing the development of English for Specific Purposes over four decades (1980–2019): A bibliometric analysis. English for Specific Purposes, 71, 149 160. https://doi.org/10.1016/j.esp.2023.04.004
Yu, D., Bondi, M., & Hyland, K. (2024). Can GPT-4 learn to analyse moves in research article abstracts? Applied Linguistics, amae071. https://doi.org/10.1093/applin/amae071
Copyright (c) 2025 Warren Bonnard, Mary Catherine Lavissière, Anas Belfathi, Nicolas Hernandez, Christine Jacquin, Laura Monceaux-Cachard
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Downloads
Download data is not yet available.