Common NLP Interview QuestionsTokenizationQ1. Explain what is a token, a tokenizer, and tokenization.So, a token is basically the smallest piece of text the model works with. It could be a whole word,