SMILE II aims to research and build advanced technology for sign language learning. The project builds on the groundwork of the SNSF Sinergia project SMILE I, which dealt with the assessment of the manual activity of Swiss German Sign Language (Deutschschweizerische Gebärdensprache, DSGS) in isolated signs produced by L1 users and L2 learners. SMILE II extends this technology to continuous sign language assessment, including both manual and non-manual components of signs, so that a DSGS learner’s sentence-level production can be assessed in an automatic manner.
The proposed goal is faced with several challenges: (a) lack of DSGS resources; (b) continuous sign language can contain both manual information (information on the hands/arms) and non-manual information (information on the torso, head, and face), and almost all work on recognition to date has focused on manual information only. Furthermore, continuous sign language recognition and continuous sign language production are still open research problems; (c) lack of standardized instruments for DSGS assessment; and (d) lack of automatic methods to assess continuous sign productions at both the manual and non-manual level.
The focus of this repository is the work carried out to address challenges (a) and (b).
