NLP problems and naive Bayes

Natural language processing: defining the problems. From syntactic to semantic problems. The text classification problem and the naive Bayesian classifier. Tf-idf weights.

Extending naive Bayes

Can we remove the naive Bayes assumptions? From classification to clustering.  From clustering to topic modeling: probabilistic latent semantic analysis.

Topic modeling

Regularised pLSA: additive regularisation of topic models (ARTM). Bayesian pLSA: latent Dirichlet allocation (LDA). LDA extensions: additional dependencies and/or additional information.

Practical session

Construct different topic models.

Language modeling

Language modeling: the problem and its importance. Language modeling with n-grams.

Other word embeddings

Word2vec variations: GloVe and FastText. Extensions of word embeddings. Examples.

Basic RNNs for NLP

Reminder: recurrent neural networks. RNNs for language modeling. RNNs for sentiment analysis.

Practical session

Improve sentiment analysis with different deep architectures.

Character-based models

Breaking words down into characters. Character-based models. Convolutional networks in NLP.

Encoder-decoder architectures

Idea of encoder-decoder architectures. Examples. Show and Tell.

Neural networks with attention

Attention in deep learning: Show, Attend, and Tell. Encoder-decoder with attention: machine translation.

Practical session

Build your own machine translation model.

Hierarchical encoder-decoder

Dialogue and conversational models. The hierarchical encoder-decoder architecture.

Practical session

Build your own chat bot.

Neural language models and word2vec

From language modeling to word embeddings: word2vec architectures.