Fundamentals of formal properties of nonmanuals:
A quantitative approach (NONMANUAL)

In 2023-2027 my main research focus is the ERC-StG project NONMANUAL. Read more about it on the separate project page.

Whole-entity classifiers: a multiperspective approach

Whole-entity classifiers are meaningful handshapes that are used in signs describing motion and location in various sign languages. The phenomenon of classifiers lies on the intersection between language and gesture, and its properties have caused a lot of controversy in the field of sign linguistics. In this project, running in 2023 and 2023, my collaborators and me look at classifiers in several sign languages, both at their structure and meaning, and apply different frameworks and perspectives to reach a better understanding.

Example of a whole-entity classifier in NGT (Sign Language of the Netherlands)

Collaboration with: Giorgia Zorzi, Pia Simper-Allen, Jenia Khrsitoforova, Jacopo Romoli, Cornelia Loos, Calle Börstell, Ben Anible, Beyza Sumer

This project is funded by a Young CAS Fellowship from Centre for Advanced Study, Oslo, Norway.

Computational approaches to sign languages

Since 2019 I have been collaborating with a group of researchers at Nazarabayev University (Nur-Sultan, Kazakhstan), led by dr. Anara Sandygulova, on various projects involving computational approaches to sign languages.

For instance, we have done research on automatic recognition of handshapes in Russian Sign Language (read it here) and how this method can be used to answer linguistic questions, and on how automatic recognition of signs can be aided by taking non-manuals into account (read it here). In addition, we published a paper showing that eyebrow position in Kazakh-Russian Sign Language is affected by both emotions and grammar using computational approaches to quantitatively analyze the data (here, open access). In the most recent publication (August 2021) here, we applied OpenFace to extract eyebrow position in 3D from video recordings, and Machine Learning to correct for errors introduced by head tilts. This is a proof of concept for using Computer Vision for phonetic analysis of non-manual markers.

Video demonstrating application of OpenPose to study non-manuals in question marking.

Lexical variation in Russian Sign Language.

In 2020, I have collaborated with several researchers on a project initiated by the Garage Museum, Moscow, to collect and analyze a database of lexical variation in Russian Sign Language. We have collected more than 19 000 recordings for more than 100 concepts from more than 250 RSL signers. The database is openly available here, albeit only in Russian. Our first published paper using this dataset is on the very complex networks of variants that have been found in it, and it can be read here, in open access.

Network of variants for the sign GREY in RSL from the paper above.

Argument structure in three sign languages

In 2015-2018, I was a postdoctoral researcher in the project “Argument structure in three sign languages: typological and theoretical aspects”. This was a 4-year project, funded by the Dutch Science Foundation (NWO 360-70-520), and headed by dr. Roland Pfau and prof. dr. Enoch O. Aboh; two PhD students (Marloes Oomen and Vanja de Lint) also work on this project.

We are investigating argument structure of verbs in three different sign languages to compare them to argument structure described for spoken languages. For instance, we found that transitivity of verbs in the corpus of Russian Sign Language reflects the cross-linguistically based transitivity hierarchy rather well (read it here); the same is true for German Sign Language and Sign Language of the Netherlands. Furthermore, Russian Sign Language uses a number of argument structure alternations, including impersonal alternation, in a way similar to spoken languages (read it here) On the other hand, we also discovered that classifier predicates, which is a very common construction in various sign languages, have complex event and argument structures, probably extending beyond what is found in lexical verbs in spoken languages (read it here).

Examples of different types of classifier predicates in RSL.

Quotation in Russian Sign Language

As a part of a larger project funded by the Russian Science Foundation, I am investigating quotation in Russian Sign Language, in collaboration with Evgeniia Khristoforova. It has been claimed that when a signer wants quote someone else, they would lean or turn to take the role of the other person, which is known as role shift. We looked at quotations in the corpus and found that this is also the case for Russian Sign Language, but also that role shift is far from obligatory in marking quotation (read about it here). Using elicitation (that is, asking native signers for judgments) we also discussed how role shift is related to the notions of direct vs. indirect speech, and found again that the relation is not straightforward (read it here).

An example of quotation in RSL with role shift.

Information structure in sign languages

For my PhD project, I analysed how information structure is expressed in two sign languages: RSL and NGT. Specifically, I argued that topic is marked in a variety of ways in both sign languages, but that they still cannot be considered topic-prominent, and that focus in these languages is expressed primarily by manual prosody (see my dissertation for details). For a general overview of information structure in sign languages, see this chapter written in collaboration with Roland Pfau.

In addition to the basic notions of information structure, I explored a number of constructions related to expressing information structure. In particular, I looked at doubling (when a constituent appears twice within one clause) and its functions (read here or in my dissertation). Together with Anna Sáfár and Onno Crasborn, I investigated how the second hand is used in discourse and whether it is related to information structure (read this and this papers, and also this one if you are interested in formal syntax). Finally, together with Lianne Vink, I studied another construction in NGT common to many sign languages, namely question-answer pairs (when the signer asks a question and then immediately answers it). Is this construction a rhetorical question followed by an answer or something like a wh-cleft? Read this to find out.

Quantification in Russian Sign Language

Another interesting domain of inquiry that has barely been studied is how quantification is expressed in sign languages. Do sign languages use quantifiers (like all and some)? Is there a difference between distributive and collective universal quantification (e.g. each vs all)? I described basic quantificational strategies in RSL in this chapter, as well as in this paper.

Metaphor and iconicity in sign languages

Sign languages use a lot of signs that can be considered metaphoric. Together with Maria Kyuseva, Yana Lomakina, and Daria Perova, I analyzed verbs of cognition and emotions in Russian Sign Language to describe metaphoric mechanism found in these signs. We claim that these mechanisms have direct parallels in spoken languages. Read a pre-final version of our paper here.

With Anna Klezovich and George Moroz, we created a database and website of iconic patterns in nineteen sign languages and seven semantic fields in order to systematically study iconicity. The website can be found here, and read our paper about it here.

Screenshot of the website on iconicity patterns.

Formal approaches to sign language grammar

Although this is not a topic per se, I am intersted in applying a formal analysis (both syntactic and semantic) to sign language data. Several of the papers mentioned above include such analysis, but here I make a seprarte list of these papers and slides:

  • A formal syntactic analysis of weak hand holds in RSL: read here
  • Formal approaches to quantifiers in RSL: read here and here
  • An overview of information structure in signed languages, including formal approaches: read here
  • A sort-of-formal account of null arguments in RSL: read here

Other topics

In the past, I also investigated how reflexive meaning is expressed in RSL, how negation works in RSL, and whether RSL systematically distinguishes nouns and verbs (read here).

In addition, I conducted some research on spoken languages, specifically on Russian and Adyghe.