Nadir Durrani

selected publications

1. 

   Discovering Latent Concepts Learned in BERT

   Fahim Dalvi, Abdul Rafae Khan, Firoj Alam, and 3 more authors

   In International Conference on Learning Representations Apr 2022

   Abs Bib PDF

   A large number of studies that analyze deep neural network models and their ability to encode various linguistic and non-linguistic concepts provide an interpretation of the inner mechanics of these models. The scope of the analyses is limited to pre-defined concepts that reinforce the traditional linguistic knowledge and do not reflect on how novel concepts are learned by the model. We address this limitation by discovering and analyzing latent concepts learned in neural network models in an unsupervised fashion and provide interpretations from the model’s perspective. In this work, we study: i) what latent concepts exist in the pre-trained BERT model, ii) how the discovered latent concepts align or diverge from classical linguistic hierarchy and iii) how the latent concepts evolve across layers. Our findings show: i) a model learns novel concepts (e.g. animal categories and demographic groups), which do not strictly adhere to any pre-defined categorization (e.g. POS, semantic tags), ii) several latent concepts are based on multiple properties which may include semantics, syntax, and morphology, iii) the lower layers in the model dominate in learning shallow lexical concepts while the higher layers learn semantic relations and iv) the discovered latent concepts highlight potential biases learned in the model. We also release a novel BERT ConceptNet dataset consisting of 174 concept labels and 1M annotated instances.

   @inproceedings{dalvi2022discovering,
     title = {Discovering Latent Concepts Learned in {BERT}},
     author = {Dalvi, Fahim and Khan, Abdul Rafae and Alam, Firoj and Durrani, Nadir and Xu, Jia and Sajjad, Hassan},
     booktitle = {International Conference on Learning Representations},
     year = {2022},
     month = apr,
     url = {https://openreview.net/forum?id=POTMtpYI1xH},
     area = {Latent Concepts},
   }

2. 

   What Is One Grain of Sand in the Desert? Analyzing Individual Neurons in Deep NLP Models

   Fahim Dalvi, Nadir Durrani, Hassan Sajjad, and 3 more authors

   In Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence (AAAI, Oral presentation) Jan 2019

   Abs Bib PDF

   Despite the remarkable evolution of deep neural networks in natural language processing (NLP), their interpretability remains a challenge. Previous work largely focused on what these models learn at the representation level. We break this analysis down further and study individual dimensions (neurons) in the vector representation learned by end-to-end neural models in NLP tasks. We propose two methods: Linguistic Correlation Analysis, based on a supervised method to extract the most relevant neurons with respect to an extrinsic task, and Cross-model Correlation Analysis, an unsupervised method to extract salient neurons w.r.t. the model itself. We evaluate the effectiveness of our techniques by ablating the identified neurons and reevaluating the network’s performance for two tasks: neural machine translation (NMT) and neural language modeling (NLM). We further present a comprehensive analysis of neurons with the aim to address the following questions: i) how localized or distributed are different linguistic properties in the models? ii) are certain neurons exclusive to some properties and not others? iii) is the information more or less distributed in NMT vs. NLM? and iv) how important are the neurons identified through the linguistic correlation method to the overall task? Our code is publicly available as part of the NeuroX toolkit (Dalvi et al. 2019a). This paper is a non-archived version of the paper published at AAAI (Dalvi et al. 2019b)

   @inproceedings{dalvi:2019:AAAI,
     title = {What Is One Grain of Sand in the Desert? Analyzing Individual Neurons in Deep NLP Models},
     author = {Dalvi, Fahim and Durrani, Nadir and Sajjad, Hassan and Belinkov, Yonatan and Bau, D. Anthony and Glass, James},
     booktitle = {Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence (AAAI, Oral presentation)},
     year = {2019},
     month = jan,
     area = {Neuron Analysis},
   }

3. 

   What do Neural Machine Translation Models Learn about Morphology?

   Yonatan Belinkov, Nadir Durrani, Fahim Dalvi, and 2 more authors

   In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) Jul 2017

   Abs Bib PDF

   Neural machine translation (MT) models obtain state-of-the-art performance while maintaining a simple, end-to-end architecture. However, little is known about what these models learn about source and target languages during the training process. In this work, we analyze the representations learned by neural MT models at various levels of granularity and empirically evaluate the quality of the representations for learning morphology through extrinsic part-of-speech and morphological tagging tasks. We conduct a thorough investigation along several parameters: word-based vs. character-based representations, depth of the encoding layer, the identity of the target language, and encoder vs. decoder representations. Our data-driven, quantitative evaluation sheds light on important aspects in the neural MT system and its ability to capture word structure.

   @inproceedings{belinkov-etal-2017-neural,
     title = {What do Neural Machine Translation Models Learn about Morphology?},
     author = {Belinkov, Yonatan and Durrani, Nadir and Dalvi, Fahim and Sajjad, Hassan and Glass, James},
     booktitle = {Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
     month = jul,
     year = {2017},
     address = {Vancouver, Canada},
     publisher = {Association for Computational Linguistics},
     url = {https://aclanthology.org/P17-1080},
     doi = {10.18653/v1/P17-1080},
     pages = {861--872},
     area = {Representation Analysis},
   }

4. 

   Integrating an Unsupervised Transliteration Model into Statistical Machine Translation

   Nadir Durrani, Hassan Sajjad, Hieu Hoang, and 1 more author

   In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, volume 2: Short Papers Apr 2014

   Abs Bib PDF

   We investigate three methods for integrating an unsupervised transliteration model into an end-to-end SMT system. We induce a transliteration model from parallel data and use it to translate OOV words. Our approach is fully unsupervised and language independent. In the methods to integrate transliterations, we observed improvements from 0.23-0.75 (∆ 0.41) BLEU points across 7 language pairs. We also show that our mined transliteration corpora provide better rule coverage and translation quality compared to the gold standard transliteration corpora.

   @inproceedings{durrani-etal-2014-integrating,
     title = {Integrating an Unsupervised Transliteration Model into Statistical Machine Translation},
     author = {Durrani, Nadir and Sajjad, Hassan and Hoang, Hieu and Koehn, Philipp},
     booktitle = {Proceedings of the 14th Conference of the {E}uropean Chapter of the Association for Computational Linguistics, volume 2: Short Papers},
     month = apr,
     year = {2014},
     address = {Gothenburg, Sweden},
     publisher = {Association for Computational Linguistics},
     url = {https://aclanthology.org/E14-4029},
     doi = {10.3115/v1/E14-4029},
     pages = {148--153},
     area = {Transliteration},
   }

5. 

   A Joint Sequence Translation Model with Integrated Reordering

   Nadir Durrani, Helmut Schmid, and Alexander Fraser

   In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies Jun 2011

   Abs Bib PDF

   We present a novel machine translation model which models translation by a linear sequence of operations. In contrast to the “N-gram” model, this sequence includes not only translation but also reordering operations. Key ideas of our model are (i) a new reordering approach which better restricts the position to which a word or phrase can be moved, and is able to handle short and long distance reorderings in a unified way, and (ii) a joint sequence model for the translation and reordering probabilities which is more flexible than standard phrase-based MT. We observe statistically significant improvements in BLEU over Moses for German-to-English and Spanish-to-English tasks, and comparable results for a French-to-English task.

   @inproceedings{durrani-etal-2011-joint,
     title = {A Joint Sequence Translation Model with Integrated Reordering},
     author = {Durrani, Nadir and Schmid, Helmut and Fraser, Alexander},
     booktitle = {Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies},
     month = jun,
     year = {2011},
     address = {Portland, Oregon, USA},
     publisher = {Association for Computational Linguistics},
     url = {https://aclanthology.org/P11-1105},
     pages = {1045--1054},
     area = {Translation and Reordering},
   }