Artificial Intelligence Exam: Basic AI and LLM Concepts

1. What is a Large Language Model (LLM)?

• A type of AI model that understands and generates human-like text, trained on large amounts of data.
• A computer program that can only perform mathematical calculations.
• An AI model designed exclusively to generate images.
• A type of AI model designed to act as text editing software.

2. What are the main differences between a traditional language model and an LLM?

• Traditional models are faster
• LLMs can handle longer contexts
• LLMs have billions of parameters
• LLMs are based on transformer architectures
• LLMs are trained on massive datasets

3. What is tokenization in the context of LLMs?

• The process of converting text to speech.
• The method for training the model.
• The process of dividing text into smaller units (tokens) that the LLM can process.
• The technique for evaluating model performance.

4. What are the main challenges in training LLMs?

• High computational cost
• Need to use only English data
• Quality of training data
• Bias in datasets
• Ethical concerns

5. What is the role of pre-training in LLMs?

• Evaluating the quality of the final model
• Training the model on a large, general dataset to learn broad language patterns
• Evaluating model performance on very specific tasks.
• Optimizing model speed

6. What is "fine-tuning" in the context of LLMs?

• Adjusting a pre-trained LLM on a smaller, specific dataset to improve performance on a particular task.
• The technique to reduce bias in the model without changing its weights.
• Creating a new language model from scratch.
• Training a model from scratch without any previous data.

7. What are the characteristics of fine-tuning?

• It is performed after pre-training
• It uses task-specific datasets
• It always requires more time than pre-training
• It improves performance on specific tasks
• It can help specialize the model

8. What is zero-shot learning in LLMs?

• The ability of an LLM to perform a task with many training examples.
• A training method that uses only a small amount of data.
• A training method without data
• The ability of an LLM to perform a task without specific training examples, based on its pre-trained knowledge.

9. What are transformers in the context of LLMs?

• Physical devices for processing data.
• Neural network architectures that use attention mechanisms to process sequences
• A method for the LLM to ignore certain words in a sentence.
• Specialized databases

10. What are the advantages of subword tokenization over word-level tokenization?

• Better handles out-of-vocabulary words
• Allows better compression of text files
• Reduces vocabulary size
• Improves model efficiency
• Captures morphological patterns

11. What are the advantages of transformers over RNNs?

• Process sequences in parallel
• Better capture long-range dependencies
• Scale better to larger models
• Consume less memory
• Allow better transfer learning

12. What is catastrophic forgetting in fine-tuning?

• A bug in the model code
• A hardware problem
• An optimization technique
• When the model loses general knowledge learned during pre-training

13. What is the risk of "overfitting" in LLMs and how can it be addressed?

• The model becomes too large to run on normal computers; addressed by reducing the number of parameters.
• The model memorizes training data and doesn't generalize well to new data; addressed using more data, regularization, or early stopping.
• The model learns too slowly; addressed by increasing the learning rate.
• The model generates outputs that are too short; addressed by adding more layers to the model.

14. What is perplexity in LLM evaluation?

• A type of model error
• A training technique
• A measure that reflects how well an LLM predicts a sequence of tokens
• A tokenization method

15. What is prompt engineering?

• An advanced programming technique
• A model evaluation method
• A neural network architecture
• The design of input instructions to guide LLM behavior without changing its weights

16. How is Natural Language Understanding (NLU) used in LLMs?

• Classifying intents, extracting entities, performing sentiment analysis, and comparing sentence meanings to understand language.
• Only to translate unknown languages.
• Creating new words for a language.
• Calculating word frequency in text.

17. What are "sparse transformers" and how do they improve LLM efficiency?

• They are Transformer models with fewer layers, making them faster.
• They reduce the quadratic complexity of self-attention by attending only to a subset of tokens, reducing computational and memory requirements.
• They are Transformers that don't use the attention mechanism.
• They are Transformer models that use less training data.

18. What is Retrieval-Augmented Generation (RAG)?

• A type of transformer architecture.
• A fine-tuning technique with internal and external information.
• A technique that combines LLMs with external information retrieval systems.
• A tokenization method.

19. What is knowledge distillation used for in LLMs?

• To clean training data
• To evaluate model quality
• To train a larger model that mimics a specialized one, maintaining performance
• To train a smaller model that mimics a larger one, reducing size while maintaining performance

20. What are characteristics of encoder-only models like BERT?

• They are bidirectional
• They excel at comprehension tasks
• They are ideal for text classification
• They process all context simultaneously
• They generate text auto-regressively