Nadir Durrani

selected publications

1. 

   Discovering Salient Neurons in deep NLP models

   Nadir Durrani, Fahim Dalvi, and Hassan Sajjad

   Journal of Machine Learning Research 2023

   Abs Bib PDF

   While a lot of work has been done in understanding representations learned within deep NLP models and what knowledge they capture, work done towards analyzing individual neurons is relatively sparse. We present a technique called Linguistic Correlation Analysis to extract salient neurons in the model, with respect to any extrinsic property, with the goal of understanding how such knowledge is preserved within neurons. We carry out a fine-grained analysis to answer the following questions: (i) can we identify subsets of neurons in the network that learn a specific linguistic property? (ii) is a certain linguistic phenomenon in a given model localized (encoded in few individual neurons) or distributed across many neurons? (iii) how redundantly is the information preserved? (iv) how does fine-tuning pre-trained models towards downstream NLP tasks impact the learned linguistic knowledge? (v) how do models vary in learning different linguistic properties? Our data-driven, quantitative analysis illuminates interesting findings: (i) we found small subsets of neurons that can predict different linguistic tasks; (ii) neurons capturing basic lexical information, such as suffixation, are localized in the lowermost layers; (iii) neurons learning complex concepts, such as syntactic role, are predominantly found in middle and higher layers; (iv) salient linguistic neurons are relocated from higher to lower layers during transfer learning, as the network preserves the higher layers for task-specific information; (v) we found interesting differences across pre-trained models regarding how linguistic information is preserved within them; and (vi) we found that concepts exhibit similar neuron distribution across different languages in the multilingual transformer models. Our code is publicly available as part of the NeuroX toolkit (Dalvi et al., 2023).

   @article{JMLR:v24:23-0074,
     author = {Durrani, Nadir and Dalvi, Fahim and Sajjad, Hassan},
     title = {Discovering Salient Neurons in deep NLP models},
     journal = {Journal of Machine Learning Research},
     year = {2023},
     volume = {24},
     number = {362},
     pages = {1--40},
     area = {Neuron Analysis},
   }

2. 

   On the Effect of Dropping Layers of Pre-trained Transformer Models

   Hassan Sajjad, Fahim Dalvi, Nadir Durrani, and 1 more author

   Computer Speech and Language 2023

   Abs Bib PDF

   Transformer-based NLP models are trained using hundreds of millions or even billions of parameters, limiting their applicability in computationally constrained environments. While the number of parameters generally correlates with performance, it is not clear whether the entire network is required for a downstream task. Motivated by the recent work on pruning and distilling pre-trained models, we explore strategies to drop layers in pre-trained models, and observe the effect of pruning on downstream GLUE tasks. We were able to prune BERT, RoBERTa and XLNet models up to 40%, while maintaining up to 98% of their original performance. Additionally we show that our pruned models are on par with those built using knowledge distillation, both in terms of size and performance. Our experiments yield interesting observations such as: (i) the lower layers are most critical to maintain downstream task performance, (ii) some tasks such as paraphrase detection and sentence similarity are more robust to the dropping of layers, and (iii) models trained using different objective function exhibit different learning patterns and w.r.t the layer dropping.

   @article{sajjad2023:csl,
     address = {London, UK, UK},
     author = {Sajjad, Hassan and Dalvi, Fahim and Durrani, Nadir and Nakov, Preslav},
     issn = {0885-2308},
     doi = {https://doi.org/10.1016/j.csl.2022.101429},
     url = {https://www.sciencedirect.com/science/article/pii/S0885230822000596},
     issue_date = {January 2023},
     journal = {Computer Speech and Language},
     number = {C},
     publisher = {Academic Press Ltd.},
     title = {On the Effect of Dropping Layers of Pre-trained Transformer Models},
     volume = {77},
     pages = {101429},
     year = {2023},
     area = {Transfer Learning},
   }

3. 

   Discovering Latent Concepts Learned in BERT

   Fahim Dalvi, Abdul Rafae Khan, Firoj Alam, and 3 more authors

   In International Conference on Learning Representations Apr 2022

   Abs Bib PDF

   A large number of studies that analyze deep neural network models and their ability to encode various linguistic and non-linguistic concepts provide an interpretation of the inner mechanics of these models. The scope of the analyses is limited to pre-defined concepts that reinforce the traditional linguistic knowledge and do not reflect on how novel concepts are learned by the model. We address this limitation by discovering and analyzing latent concepts learned in neural network models in an unsupervised fashion and provide interpretations from the model’s perspective. In this work, we study: i) what latent concepts exist in the pre-trained BERT model, ii) how the discovered latent concepts align or diverge from classical linguistic hierarchy and iii) how the latent concepts evolve across layers. Our findings show: i) a model learns novel concepts (e.g. animal categories and demographic groups), which do not strictly adhere to any pre-defined categorization (e.g. POS, semantic tags), ii) several latent concepts are based on multiple properties which may include semantics, syntax, and morphology, iii) the lower layers in the model dominate in learning shallow lexical concepts while the higher layers learn semantic relations and iv) the discovered latent concepts highlight potential biases learned in the model. We also release a novel BERT ConceptNet dataset consisting of 174 concept labels and 1M annotated instances.

   @inproceedings{dalvi2022discovering,
     title = {Discovering Latent Concepts Learned in {BERT}},
     author = {Dalvi, Fahim and Khan, Abdul Rafae and Alam, Firoj and Durrani, Nadir and Xu, Jia and Sajjad, Hassan},
     booktitle = {International Conference on Learning Representations},
     year = {2022},
     month = apr,
     url = {https://openreview.net/forum?id=POTMtpYI1xH},
     area = {Latent Concepts},
   }

4. 

   On the Linguistic Representational Power of Neural Machine Translation Models

   Yonatan Belinkov, Nadir Durrani, Fahim Dalvi, and 2 more authors

   Computational Linguistics Apr 2020

   Abs Bib PDF

   Despite the recent success of deep neural networks in natural language processing and other spheres of artificial intelligence, their interpretability remains a challenge. We analyze the representations learned by neural machine translation (NMT) models at various levels of granularity and evaluate their quality through relevant extrinsic properties. In particular, we seek answers to the following questions: (i) How accurately is word structure captured within the learned representations, which is an important aspect in translating morphologically rich languages? (ii) Do the representations capture long-range dependencies, and effectively handle syntactically divergent languages? (iii) Do the representations capture lexical semantics? We conduct a thorough investigation along several parameters: (i) Which layers in the architecture capture each of these linguistic phenomena; (ii) How does the choice of translation unit (word, character, or subword unit) impact the linguistic properties captured by the underlying representations? (iii) Do the encoder and decoder learn differently and independently? (iv) Do the representations learned by multilingual NMT models capture the same amount of linguistic information as their bilingual counterparts? Our data-driven, quantitative evaluation illuminates important aspects in NMT models and their ability to capture various linguistic phenomena. We show that deep NMT models trained in an end-to-end fashion, without being provided any direct supervision during the training process, learn a non-trivial amount of linguistic information. Notable findings include the following observations: (i) Word morphology and part-of-speech information are captured at the lower layers of the model; (ii) In contrast, lexical semantics or non-local syntactic and semantic dependencies are better represented at the higher layers of the model; (iii) Representations learned using characters are more informed about word-morphology compared to those learned using subword units; and (iv) Representations learned by multilingual models are richer compared to bilingual models.

   @article{belinkov-etal-2020-linguistic,
     title = {On the Linguistic Representational Power of Neural Machine Translation Models},
     author = {Belinkov, Yonatan and Durrani, Nadir and Dalvi, Fahim and Sajjad, Hassan and Glass, James},
     journal = {Computational Linguistics},
     volume = {46},
     number = {1},
     year = {2020},
     address = {Cambridge, MA},
     publisher = {MIT Press},
     url = {https://aclanthology.org/2020.cl-1.1},
     doi = {10.1162/coli_a_00367},
     pages = {1--52},
     area = {Multilinguality},
   }

5. 

   Integrating an Unsupervised Transliteration Model into Statistical Machine Translation

   Nadir Durrani, Hassan Sajjad, Hieu Hoang, and 1 more author

   In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, volume 2: Short Papers Apr 2014

   Abs Bib PDF

   We investigate three methods for integrating an unsupervised transliteration model into an end-to-end SMT system. We induce a transliteration model from parallel data and use it to translate OOV words. Our approach is fully unsupervised and language independent. In the methods to integrate transliterations, we observed improvements from 0.23-0.75 (∆ 0.41) BLEU points across 7 language pairs. We also show that our mined transliteration corpora provide better rule coverage and translation quality compared to the gold standard transliteration corpora.

   @inproceedings{durrani-etal-2014-integrating,
     title = {Integrating an Unsupervised Transliteration Model into Statistical Machine Translation},
     author = {Durrani, Nadir and Sajjad, Hassan and Hoang, Hieu and Koehn, Philipp},
     booktitle = {Proceedings of the 14th Conference of the {E}uropean Chapter of the Association for Computational Linguistics, volume 2: Short Papers},
     month = apr,
     year = {2014},
     address = {Gothenburg, Sweden},
     publisher = {Association for Computational Linguistics},
     url = {https://aclanthology.org/E14-4029},
     doi = {10.3115/v1/E14-4029},
     pages = {148--153},
     area = {Transliteration},
   }

6. 

   A Joint Sequence Translation Model with Integrated Reordering

   Nadir Durrani, Helmut Schmid, and Alexander Fraser

   In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies Jun 2011

   Abs Bib PDF

   We present a novel machine translation model which models translation by a linear sequence of operations. In contrast to the “N-gram” model, this sequence includes not only translation but also reordering operations. Key ideas of our model are (i) a new reordering approach which better restricts the position to which a word or phrase can be moved, and is able to handle short and long distance reorderings in a unified way, and (ii) a joint sequence model for the translation and reordering probabilities which is more flexible than standard phrase-based MT. We observe statistically significant improvements in BLEU over Moses for German-to-English and Spanish-to-English tasks, and comparable results for a French-to-English task.

   @inproceedings{durrani-etal-2011-joint,
     title = {A Joint Sequence Translation Model with Integrated Reordering},
     author = {Durrani, Nadir and Schmid, Helmut and Fraser, Alexander},
     booktitle = {Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies},
     month = jun,
     year = {2011},
     address = {Portland, Oregon, USA},
     publisher = {Association for Computational Linguistics},
     url = {https://aclanthology.org/P11-1105},
     pages = {1045--1054},
     area = {Translation and Reordering},
   }