Dutch Adjective-Noun Phrase Dataset for Compositionality Tests (nld-adj-n)

If you want to use this dataset for research purposes, please refer to the following sources:

                 - Gertjan Van Noord, Gosse Bouma, Frank Van  Eynde,  Daniël  De  Kok,  Jelmer  Van  der Linde, Ineke Schuurman, Erik Tjong Kim Sang,

                 and Vincent Vandeghinste. 2013. Large Scale Syntactic Annotation of Written Dutch: Lassy. In Essential Speech and Language Technology for Dutch, pages 147–164. Springer.

                 - Corina Dima, Daniël de Kok, Neele Witte, Erhard Hinrichs. 2019. No word is an island — a transformation weighting model for semantic composition. Transactions of the Association for Computational Linguistics.

                 The dataset is distributed under the Creative Commons Attribution NonCommercial (CC-BY-NC) license.

                 The 83,392 Dutch adjective-noun phrases (58,347 train, 16,669 test, 8,376 dev) from this dataset were extracted from the Lassy Large treebank (Van Noord et al., 2013), which consists of written texts (Wikipedia, newspapers) and texts of the medical domain.

                 The train/test/dev files have the following format, the single parts are separated by tab.

                 adjective noun adj-noun phrase, where the adjective and the noun in the phrase are separated by the string _adj_n_ (e.g. politiek verlof politiek_adj_n_verlof).

For results of different composition models on this dataset see Dima et al. (2019) , No word is an island — a transformation weighting model for semantic composition.

The word embeddings were trained on the same treebank and the training corpus consists of 47.6M sentences and 700M tokens. Because adjectives and nouns are separate words, they were concatenated into a single unit (using the separator _adj_n_) for training the phrase representations.

The embeddings were learned with the skip-gram model with negative sampling (Mikolov et al., 2013) from the word2vec package. The embedding size is 200, context size is a symmetric window of 10, 25 negative samples were used and a sample probability of 0.0001.

Representations were only trained for words and phrases with a minimum frequency of 30 occurrences. The embeddings are stored in the binary word2vec format in lassy-adjn-lemmas.bin, which can be loaded by several packages (e.g. the gensim package of Řehůřek, Radim and Petr Sojka (2010)). The vocabulary contains 355,236 words.