Source: 13 Functions as Data
This section discusses the concept of using functions as values and how we can represent two concepts of calculus: f(x) and d/dx. The section then implements the d/dx operation using numerical differentiation and gives an example of its implementation.
Retrieval augmentation allows for relevant information from an external knowledge base to be retrieved, avoiding LLMs’ data freshness issues. This technique is useful when internal company documents need to be interacted with via an LLM. The knowledge base necessary for retrieval augmentation consists of source knowledge (information fed to the LLM) and parametric knowledge (what the LLM has learned during training).